Magic behind closures

2020-07-02

Page content

Closure is a very important concept related to anonymous functions and lambdas. They allow a programmer to use local variables of the parent method inside a body of the inline function and then execute it at any time. The question is, how the C# compiler saves these local variables and how are they restored later - let’s check it.

Inside display classes

I’m pretty sure that a lot of programmers know the example below - it greatly shows the trap related to inline functions using parent’s local variables. We have a simple method CreateDelegates which creates a list of Action objects, and then populates it with n lambda methods, where each of them just writes a value of i variable. Our Main method is also simple and just calls every lambda created earlier.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
using System;
using System.Collections.Generic;

namespace ClosureTest
{
    public class Program
    {
        static void Main(string[] args)
        {
            var delegates = CreateDelegates(10);
            foreach (var d in delegates)
            {
                d();
            }

            Console.Read();
        }

        static List<Action> CreateDelegates(int n)
        {
            var delegates = new List<Action>();

            for (var i = 0; i <= n; i++)
            {
                delegates.Add(() => Console.WriteLine(i));
            }

            return delegates;
        }
    }
}

Somebody who sees this example the first time can think: “every lambda takes the value of i variable exactly at the time it’s created, so the program should in the result write numbers from 0 to n”. I remember my surprise a few years ago when after running similar code I’ve seen this result:

To understand what happened here, we have to know how closures feature works in .NET. I’ve decompiled this program using dotPeek to see what has been generated by the compiler (if you want to do it yourself, remember to enable “Show Compiler-generated Code” option):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
// Decompiled with JetBrains decompiler
// Type: ClosureTest.Program
// Assembly: ClosureTest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
// MVID: 32803F76-2F51-406F-99B4-9BA84B76267C
// Compiler-generated code is shown

using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices;

namespace ClosureTest
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            foreach (Action action in Program.CreateDelegates(10))
                action();
            Console.Read();
        }

        private static List<Action> CreateDelegates(int n)
        {
            List<Action> actionList = new List<Action>();
            Program.<>c__DisplayClass1_0 cDisplayClass10 = new Program.<>c__DisplayClass1_0();
            for (cDisplayClass10.i = 0; cDisplayClass10.i <= n; cDisplayClass10.i++)
            {
                // ISSUE: method pointer
                actionList.Add(new Action((object) cDisplayClass10, __methodptr(<CreateDelegates>b__0)));
            }
            return actionList;
        }

        public Program()
        {
            base..ctor();
        }

        [CompilerGenerated]
        private sealed class <>c__DisplayClass1_0
        {
            public int i;

            public <>c__DisplayClass1_0()
            {
                base..ctor();
            }

            internal void <CreateDelegates>b__0()
            {
                Console.WriteLine(this.i);
            }
        }
    }
}

We can clearly see that there are some new elements here. The most important one is a new class <>c__DisplayClass1_0 which contains our closure. It has one field i which is our iterator variable from the for loop in Main method. There is also a default constructor which basically calls object constructor and <CreateDelegates>b__0() method where code of our lambda is stored.

The second change can be spotted in our Main method. Just before the loop, the compiler creates a new instance of <>c__DisplayClass1_0 class, which will hold closure for our lambdas. for loop also has been modified - we don’t use a local variable as iterator anymore because it has been captured by our closure. This exactly explains why the invocation of every delegate will give us a 10 as a result - this is the last value after the end of for loop which is shared between all lambdas.

This effect can be easily fixed by creating a copy of iterator variable, such like this:

1
2
3
4
5
for (var i = 0; i < n; i++)
{
    var copy = i;
    delegates.Add(() => Console.WriteLine(copy));
}

Now, the compiler will generate separate display class instances for every lambda, which means that there will be 10 copies of iterator variables and each of them will contain a valid value.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
private static List<Action> CreateDelegates(int n)
{
    List<Action> actionList = new List<Action>();
    for (int index = 0; index < n; ++index)
    {
        Program.<>__DisplayClass1_0 cDisplayClass10 = new Program.<>__DisplayClass1_0();
        cDisplayClass10.copy = index;
        // ISSUE: method pointer
        actionList.Add(new Action((object) cDisplayClass10, __methodptr(<CreateDelegates>b__0)));
    }
    return actionList;
}

The program will give us the correct and desired result:

Messing with closures

Using anonymous functions and lambdas without knowledge about display classes can be very dangerous. Let’s take our previous example and add something, which will modify iterator variable i inside lambda:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
static List<Action> CreateDelegates(int n)
{
    var delegates = new List<Action>();

    for (var i = 0; i <= n; i++)
    {
        delegates.Add(() => Console.WriteLine(i++));
    }

    return delegates;
}

Somebody can think: “every lambda has its own copy of i variable, so incrementation won’t have any effect - Console.WriteLine will still write the old value”. That’s sadly not true:

Remember that every lambda shares i variable contained by the display class? Every invocation will make an incrementation and the new value will be used in the next lambda call.

Let’s make it more dangerous. Imagine that we want to display elements of the array populated with numbers from 0 to 9. After each Console.WriteLine, the array will be set to null:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
static List<Action> CreateDelegates(int n)
{
    var delegates = new List<Action>();
    var array = new int[10];

    for (var i = 0; i < n; i++)
    {
        var copy = i;
        array[i] = i;

        delegates.Add(() =>
        {
            Console.WriteLine(array[copy]);
            array = null;
        });
    }

    return delegates;
}

Again, without knowledge about how closures work in .NET, somebody will think that array is a separate reference for each lambda we created earlier. In fact, we will get the exception at the second call:

System.NullReferenceException: 'Object reference not set to an instance of an object.'

CS$<>8__locals1.array was null.

Fortunately, today’s tools are smart and detect a lot of potential bugs made by programmers even before compilation. In Visual Studio, using an i variable as in the first example will generate tooltip with the warning Captured variable is modified in the outer scope. Never ignore this - as we saw in the previous examples, bugs can be hidden at first glance and break your app after a few calls of the same lambda.

Summary

Anonymous functions and lambdas are very powerful features in C# - they allow us to write our code in a cleaner and simpler way. Additionally, closures ensure that every local variable of the parent method will be saved and safe to use at any time. Use them carefully - never ignore warnings and always think if some local variables will be shared.