LINQ and Deferred Execution using Iterators and Enumerable.Range

One of the obvious benefits of LINQ is the focus on a more declarative style of development where the code focuses more on the developer’s intent as opposed to how the end result is achieved.

Another benefit is the whole deferred execution aspect and how information is provided in a just-in-time manner.

Take the simplest, but very clear, example where we want to find all odd integers. Using LINQ, one can start to process all the odd integers immediately using Enumerable.Range and LINQ as follows:

var ints = Enumerable.Range(1, Int32.MaxValue)
 .Where(i => i % 2 == 1)
 .Select(i => i);

foreach (var odd in ints)
 Console.WriteLine(odd);

This only starts processing immediately because Enumerable.Range doesn’t sit there and wait to collect all integers between 1 and Int32.MaxValue before returning values. It instead hands off an integer one at a time as it finds them. Enumerable.Range in spirit looks like this and uses an Iterator:

public static class Enumerable
{
 public static IEnumerable<int> Range(int start, int end)
 {
 for (int i = start; i <= end; i++)
 yield return i;
 }
}

Besides the obvious benefit of processing occurring just-in-time and results being returned right away, think of the memory savings. Our Range Method doesn’t need to set-up temporary storage to store integers from start to finish before handing the entire collection back to the caller.

This even gets more awesome when you think about wanting only a subset, odd numbers in this case, of all integers between 1 and Int32.MaxValue. You really don’t want the application to first go through all the trouble of finding all integers and then having to go back through that list and only pick out the odd ones.

We can make a Where Function that in spirit embodies the intent of LINQ:

public static class MyExtensions
{
 public static IEnumerable<T> Where<T>
 (this IEnumerable<T> sequence, Func<T, bool> filter)
 {
 foreach(var item in sequence)
 if (filter(item))
 yield return item;
 }
}

This is very slick!

So maybe this still doesn’t hit one square in the noggin in terms of value. Well, let’s say we only wanted the first 200 odd integers. We obviously don’t want to waste our time getting integers past the first 200 odd integers and luckily we don’t have to:

var ints = Enumerable.Range(1, Int32.MaxValue)
 .Where(i => i % 2 == 1)
 .Select(i => i);

int count = 0;

foreach (var odd in ints)
{
 Console.WriteLine(odd);
 count++;
 if (count == 200) break;
}

Console.ReadLine();

Update: Great point by Andrey below about using LINQ .Take for getting just the top 200 items. Although I thought about using it, I don't think it really hits home the idea of delayed execution and Iterators. Nonetheless, you should of course use .Skip, .Take, etc when trying to get a page of items from a sequence as such:

var ints = Enumerable.Range(1, Int32.MaxValue)
 .Where(i => i % 2 == 1)
 .Select(i => i)
 .Take(200);

The loop I mention above would be sorta what you would find in the Take Extension Method if we made our own:

public static class MyExtensions
{
 public static IEnumerable<T> Take<T>(this IEnumerable<T> sequence, int total)
 {
  int count = 0;

  foreach (var item in sequence)
  {
   count++;
   yield return item;
   if (count == total) break;
   }
 }
}

LINQ's deferred execution and use of iterators makes this oh so sweet.

Conclusion

Hopefully this clarifies some of the value of LINQ and Iterators for those still trying to get a grip on the subjects.

David Hayden

Comments

[Andrey]

why not use .Top instead of a loop for getting first 200 odd integers?

[David Hayden]

Excellent point. In real code, that is exactly what you would do as it focuses more on the intention ( what ) rather than the how. I was thinking about that, but I am not so sure that really pushes home the point of the use of iterators and delayed execution as much as the loop.

Nonetheless, I still should have mentioned .Take as you bring up an excellent point. I will add that to the post. Thanks for the comment!