Extension Methods

This post tells you all you have to know about extension methods. I'll look into why they were introduced, how to declare and use them and finally we'll explore some of the added extension methods that make LINQ possible. Let's get to it then.
I personally think that extension methods are a double edged sword. They can both add readability to code and make it obscure at the same time. As the name suggest, they extend functionality of a class. You may now be thinking that wait didn't we use inheritance for that purpose?
The answer is : well, maybe...it all depends. Inheritance is used when there is a form of specialization going on and an added state is required for all the objects of the extended class. There are times where you don't really need to add this functionality to the class since you're not specializing anything. Nonetheless, this would rarely keep you from extending that type if you own the code. But there are times when that is basically not an option.
For example, when you don't own the class's code or you are adding functionality to an interface, you can't really make any changes to legacy code. At these times you may add a static method and just pass in the class to the method for that functionality. Although this technique is widely used, this wouldn't feel that much object oriented when you look at the code since the functionality is not really called on the object itself.
In case of interfaces, when you add a method to an interface you have basically broken all legacy code that used that interface. All those classes wouldn't build anymore and would require implementation of that method. This is due to the rule that states that each implementation of an interface has to implement all the methods that are declared in that interface.
Finally Changes that LINQ needed are mostly to the interfaces. This is why extension methods were added to the language. Yes the language only. As you have probably guessed by now the changes are yet again only syntactic sugar added by the compiler. Extension methods are converted to a static class that receives each class as an argument behind the scenes. The syntax just make it look like the type now has the functionality.
Declaring extension methods is almost too simple to forget! All you have to do is adding the extended type to as a first parameter to the method and add a "this" keyword before the type too:

    
public static int MultiplyBy(this int extendedType, int multiplyBy)
{
    return extendedType * multiplyBy;
}

Now every integer has a "MultiplyBy" method. It is truly as easy as that. There are some limitation as to where the extension method can be declared. The class in which the extension method is implemented should be non-nested, non-generic and static.
Extension methods are resolved as follows:
The compiler would firstly look for an instance method with the signature defined in the caller code. If there no instance methods found in the type then all imported namespaces are searched for compatible extension methods. Compatible extension methods are those that have exactly the same syntax or have signature that can be implicitly converted. Note that if an extension method has the same signature as an instance method, the extension methods is never called. Also no warning is given by the compiler on the occurrence of said event.
Now we look at another piece of code:

    
public static bool Equals(this object obj1, object obj2)
{
    if(obj1 == null)
        return (obj1 == obj2);
    else
        return obj1.Equals(obj2);
}

Object obj1 = null;
Object obj2 = new Object();

Console.WriteLine(obj1.Equals(obj2));

What do you think the outcome of the code above is? Does it compile? Do we get a run-time error? Can we call methods on an object that is null(obj1)? We sure were not able to up to now. The code above compiles and runs without issue. The reason is that you can actually call extension methods on NULL objects ! If you think about it, it kind of makes sense. After all we talked about extension methods being syntactic sugar on top of the language. So actually what is happening is the compiler creating a static class containing the method and passing the object as an argument. There is no problem with an argument being NULL now is it ?
That's all there is to say on extension methods. Next we will look at the extension methods added to .Net 3.5.

IEnumerable<T> Extension Methods

This interface is one those extended with extension methods in .Net 3.5. A host of extension methods that work on the sequence of Ts yielded by the type. Filtering, aggregation, projection, search, grouping are just a few of the uses of these added methods. It is really fun to play around with the methods of this interface. I can't really look at them all in this post and I haven't even used all of them but I'll go through the filtering operation of Where<T>, projection in Select<T> and maybe some groupings.

Where<T> extension method

If you're familiar with SQL you can think of this method as a counterpart to the where clause in a select statement. If you aren't familiar with SQL then an example should make it clear:

Enumerable.Range(1, 10).Where(x => x < 5);

In the example above first the static method Range of the Enumerable static class is used to get the numbers between 1-10. Notice that the return value of this function is an IEnumerable. This method is defined using an iterator block which uses deferred execution. The Where extension methods is then used combined with a Lambda expression to get all the numbers that are between 1-10 and are smaller than 5. This is really where, whatever we have talked about in the previous posts all come together. In that one line of code a lot of powerful concepts can now be see. The use of iterator blocks, static classes, generics, two phase type inference and lambda expressions. These have all resulted in a lazy filtering of elements of a collection without any effort on your part. You may be thinking that Where should have a very complicated implementation. But it turns out that, that is far from the truth. You can implement your own Where method with just a few lines of code as below:

public static IEnumerable<T> Where<T>(this IEnumerable<T> sequence, Func<T, bool> predicate)
{
    if (sequence == null || predicate == null)
        throw new ArgumentNullException();

    foreach (T element in sequence)
    {
        if(predicate(element))
            yield return element;
    }
}

There is nothing to stop us from using a method group or a delegate instead of the lambda expression but usually as you are chaining these methods together you don't usually end up doing something that a lambda expression cannot handle.

Select<T> extension method

This method is also called the projection method. Basically what it does is receiving an input, project that object into another object by either selecting a part of it or adding to it, etc. For example you may have an object that carries around a lot of information and you may want to populate a grid view with the said object. One easy way of populating grid views is to assign a collection to the DataSource property. Since you can't really assign a collection of the entire object you can just select a piece of the object and add them to a collection and then set the property. In the inverse case you can create an anonymous type that contains an object:

var x = Enumerable.Range(1, 10).Select(x => new {Number = x, Inverse = 1/x});

Here I have created a new type that contains both the number and its inverse.

GroupBy<T> extension method

The GroupBy extension method has many overloads and is a very powerful tool. This extension methods would group the elements of the sequence according to a key which is designated in the first argument. The return value of this extension method is IEnumerable<IGrouping<TKey, TElement>>. The IGrouping<TKey, TElement> type is actually inherited from IEnumerable<T> and it also contains a property "Key". It is now kind of clear what this type is there for. The GroupBy method would return a enumeratable list of IGroupings which each contain a key and can also be enumerated. The simplest form can be seen in the example below.

var persons = new [] {
                        new { Name = "John", Age = "23"},
                        new { Name = "Mary", Age = "21"},
                        new { Name = "Joan", Age = "24"},
                        new { Name = "Tom", Age = "24"},
                        new { Name = "Hank", Age = "22"},
                        new { Name = "Steve", Age = "22"},
                        new { Name = "Bella", Age = "22"},
                    };

Console.WriteLine(
                    persons.GroupBy(p => p.Age)
                            .OrderBy(p => p.Key)
                );

In order to order a list by more than one field you can easily add a .ThenBy() to the end of the chain. Something to note here though is that the actual sequence's order is not changed by these commands. Most operations done in LINQ have been made to be side effect free. These operations just make copies of the sequence, make changes and pass it along.
There are some points to remember when deciding to write your own extension methods. Just like implicit typing there are pros and cons to defining extension methods. You have to realize that the code that you write in a group development environment is different than code you write for yourself. You now have to know your audience and your maintainers. You don't want to surprise any of the people working with the code. Extension methods could be really confusing for someone who's not familiar with them. They can also cause all kinds of problems. For example if you defined a method which is added to the framework in the next version, your code will break. Unless your lucky enough to get the exact same implementation of the method added. You can also cause a lot of problems if you don't define the extension methods in the right namespace. You definitely don't want to get all the extension methods of IEnumerable in your intellisense suggestions if you're not using them? Just think long and hard about defining your own extension methods and put them in the right namespace and with the right name. It may be worth it to standardize your extension methods naming scheme so that every person in the group knows it when they're calling an extension method.
Well...this is about it ! Guess what? You now know LINQ ! You just don't know that you do yet. LINQ is just syntactic sugar over all we have learned so far. It only allows a more familiar syntax for queries. Everything will fall into place with the next post which is officially about LINQ containing query expressions and LINQ to objects.
Stay tuned !

Lambda Expressions and Expression Trees

This chapter again is all about delegates. It might be a good idea to review delegates in previous posts if you think you don't remember much from them. Lambda expressions are here to make it easier and more straightforward than ever to create delegates.

Lambda Expressions

Lambda expressions are so called because Lambda calculus in Computer Science and Math deals with the definition and manipulation of functions. Other than the name I haven't seen the use of lambda calculus anywhere else so don't get intimidated by the name.
Since lambda expressions can be considered as a special case of anonymous methods and we have already covered them, I'm going to jump right into the syntax. There are many different forms of allowed syntax For lambda expressions and that is due to their wide use across different scenarios. As we get closer to LINQ, you'd see that they are used all over the place. Of course, as I said they can always be replaced by the more general anonymous methods. Here's a general form for lambda expressions :

        (explicit or implicit list of input arguments) ⇒ {code block}

As you see the most general form is quite similar to anonymous methods with the small difference of having the new ⇒ symbol and no mention of the delegate keyword. You can explicitly type the input arguments as well as have the compiler infer them. Also, another shortcut in the syntax, which is also the more important is that you can put a single expression in the {code block}. Just like anonymous methods the blocks needs to return a value. In the case that there is only one expression, this value is that expression. for example, in the Lambda expression below, the return value of the lambda expression is each employee's name :

    
    List<person> persons = new List <person>(); 
    persons.FindAll(p => p.Name == "Joe");

In the above example, the lambda expression has one input parameter(p). That is implicitly of type Person. The "=>" operator should be read as "goes to" when reading Lambda expressions. So in this example, p goes to p.Name which is the hypothetical person's name. What the above statement accomplishes then, is searching all elements of the list and then returning the element that matches the criteria. As we know FindAll expects the generic delegate Predicate<T>. The lambda expression is implicitly converted to a delegate instance and called for each element by the FindAll method. It is important to know that Lambda expressions by themselves don't represent a type. This is due to the fact that they can be converted to both delegate instances and expression trees which we'll cover in the next section.
You can imagine all the other forms that the Lambda expression can take by looking at the transformation from anonymous methods by removing/adding types/parantheses/etc. But the most popular form and the one that you will be using most is the one we just covered.
Captured variables are handled the Same way as anonymous methods here. You have to be careful about closures here just like we discussed in the post about anonymous methods.

Expression Trees

As their name suggests, expression trees are trees that have expressions as their nodes. Each node contains an expressions which after evaluation would be a child node of another expression. Expression trees and Lambda expressions are at the heart of LINQ. It is important to know why we need expressions trees in the first place. 

LINQ is here to streamline the process of querying objects, databases, XML documents, etc. It is here to provide one language and set of operators to be applied uniformely across all these inherently different topologies. The same thing that SQL in databases and VMs in cross platform development environments are doing. In order to query each specific topology you have to define and use custom queries. For example, the syntax to query a database using SQL is different than the query language used for XML(XQuery). In order to use the same syntax for all these different platforms, one can define a common series of operations possible on each platform and then translate these common operations for each respective topology. In order to do so, the program that is written using the common operations should be analyzed and translated. This means that the code itself is the data input to the translater.

The concept of using code as data then is key to creating such a mechanism. Expressions trees do just that. Then enable us to represent a piece of code in tree data structure which can later be parsed and analyzed. In .Net 3.5, the Expression class provides all the functionalities to do this. This class allows representation of actual C# code in a data structure which can then be parsed in the same or another program (in-process vs out-process). Expression trees in .Net 3.5 only allow some encapsulation of certain operations. In order to gain complete control over dynamically generated code one still has to use the CodeDom (a library to create language-independent dynamically generated code in .Net).

In the Expression namespace, each class extending the base abstract Expression class has two main attributes associated with it:

• Type: This attributes is the actual .net type of the expression. Kind of like a return value of a function.
• Node Type: Node type is selected from a defined enumeration in the Expression namespace. As we said before all different types of expressions derive from the Expression abstract base class. This is all fine and jolly but what should be we with all these different kinds of expressions that share a common structure ? We definitely want to end of with a huge inheritance chain. Here the design decision is to extend the hierarchy not in depth, but in breadth. Each general expression type as in a binary expression is grouped under a binary expression and different "Types" are defined for them. All supported node types for the binary expression class is defined here.

There is really no easy way to tell you how expression trees work unless to do this with an example. So here is an example of an expression tree that writes an output to the console.

    Type console = Type.GetType("System.Console");
    MethodInfo method = console.GetMethod("WriteLine", new[] { typeof(String) });

    //The target is null since WriteLine is a static method.
    ParameterExpression lambdaParam = Expression.Parameter(typeof(String));
    Expression methodCall = Expression.Call(null, method, new[] { lambdaParam });
         
    var rootExpression = Expression.Lambda<Action<string>>(methodCall, new[] { lambdaParam });
    var compiledMethod = rootExpression.Compile();
    compiledMethod("This was generated using an expression tree");
 
    Console.ReadKey(true);

I realize that the above code can be hard to grasp at first but bear with me for a little bit longer and soon you'd be able to get back to this code and understand it completely. In the above example, you can see how the Expression class provides you with factory methods to create expressions. Here we needed to create a single statement that would output a string to the standard output. If you think about this in .Net terms, this means calling the "WriteLine" method of the Console type.
 In order to translate this into the expression tree world, you first have to dissect the statement and see what are the elements of the code and how are they represented in the Expression namespace. Chances are that they may not be represented ! After all the Expression namespace is not here to replace the CodeDom ! At least not yet. Anyways, you need to specify a function call, specify the method used in this function call and then specify the target of the function(instance object) and the parameters.
 How do we provide information about a type or method? The Reflection namespace of course. Discussing what reflection is and how it is used is outside the scope of this article. In short, we will be able to specify a type or method's fully qualified name and get the .Net framework to search for and find the type we are talking about. This would allow specification of actual types with strings which would be searched for at run-time. If you've never heard of reflection before, this should feel very messy and counter-intuitive from an object oriented point of view, but it adds a lot of flexibility and its useful.
 Getting back to the issue at hand, we use the reflection namespace to obtain information on the method and it's parameters both of which, we supply with a string. What's left is specifying the expression types for each of the expressions and we have given the compiler all it is to know about the statement ! In the end we can ask the compiler to compile the expression tree into an element of type Expression<TDelegate>
Now things get a little bit confusing here due to a naming system used in the inheritance chain. So far we know that different types of expressions inherit from a base class called Expression. Classes like BinaryExpression, ConditionalExpression, LambdaExpression and so forth. Now there is a special generic class that extends the LambdaExpression class called the Expression<TDelegate> class. This class represents a strongly typed lambda expression and has a Compile() method which if used would create an actual Lambda expression whose type is specified by the TDelegate type parameter. In the example the type of the delegate is Action<String>. The .net library provides some premade and ready to use delegate types. This is quite handy as they are generic and have basically abolished the need to create a delegate types ever. Here we needed to specify a delegate type that accepts a string as a parameter and doesn't return anything. Action<T> and it's brothers(Action<T1, T2>, Action<T1, T2, T3>, etc) provide the types for void methods that receive 1 to 17 parameters(in .Net 4.0). What are we going to do if the method has a return type ? There is 17 generic delegates declared for that too(Func). Anyways, we use the Action delegate to represent the expression for the LambdaExpression class and call the Compile method. This method returns an actual delegate that can now be called like any other ! We have just dynamically created a delegate at run-time ! That looked like a lot of word for a rather simple task and indeed it is ! The good news is that .Net allows auto conversions of lambda expressions in their simplest case to expression trees. You just have to assign a Lambda expression to a Expression<TDelegate> instance and the compiler would do the rest. This can be seen in the example below:

    
    Expression<Action<String>> expression = (p) => Console.WriteLine(p);
    expression.Compile()("Hello");

The more complex lambda expressions which contain loops or condition blocks or even a single return statement are not supported in .net 3.5 but in .net 4.0 they were added since they were needed in the DLR framework.
The rest of the chapter in the book deals with the new rules on overload resolutions and type inference rules. In C# 2.0 each parameter was resolved independently and not much inference was going on. In C# 3.5 with the introduction of inferred types, lambda expressions and LINQ, type inference is basically going on everywhere. The rules are complicated but the gist is that in .net 3.5 type inference and parameter resolution
uses a collaborative effort among the different parameters; meaning that each parameter now can add some information to the inference process. This is needed to resolve some the method group resolutions and lambda type resolutions. I would not delve into any details on this. In case you are interested you can check out the language specifications which contains the rules or read the chapter on this in the book which goes into it in more detail.
Next up are extension methods and then we have basically covered the language side of LINQ.