Using Enumerators in C#

Jul 19, 2019 • 6 Minute Read

Introduction

Ever since programmers started programming, there have been some basic data structures that allowed the listing or collection of specific items. These items were usually passed onto other functions to be processed or written to a remote location. These basic data structures, like lists, have a very problematic property. Let's say you would like to pass a list with 10 items to another function. The size of an integer is 32-bit (4 bytes) and this is multiplied by the amount that you want to transfer. In this case, it's just 40 bytes. As you pile more and more items into your list, it's size grows in a linear fashion. 100 items mean 400 bytes; 1 million items can consume Gigabytes of memory. Not all systems are equipped with the resources to deal with this situation. The yield return statement and the concept of Enumerators and Enumerables are here to help.

Enumerators and Enumerables

There is a nifty tool called reflector that you can use to take a look at the generated class when you are using this technique.

When we are talking about enumerators, we are simply talking about a handle that is used when we are enumerating. This points at the current element and can be moved to point to the next.

When we are talking about enumerables, we are simply talking about a class or datatype which implements some basic protocols that allow the enumerator to traverse its items.

Simple Return Statement

This function takes an argument which tells how many even numbers it should return.

      public static List<int> EvenNumbers(int max)
	{
    	var retList = new List<int>();
        int i = 0;
        while (retList.Count < max)
        	{
        		retList.Add(i += 2);
            }
    	return retList;
	}
    

This is a simple example, but, if we inspect the code, we see the bottleneck. As we increase the max argument, the list we return is getting bigger and bigger. This is due to the fact that we return the whole list at once, and do it while holding max even numbers.

Calling this for loop to print the elements shows that almost 3GB of memory was consumed by the application.

      foreach (int i in EvenNumbers(1000000000))
	{
    	Console.WriteLine($"Number {i}");
	}
    

I think you my know what I am thinking, at this point. There must be a better way. We would like to prevent our application from becoming memory-hungry.

Enter Yield Return

The yield return statement will tell our compiler that this function is only allowed to return one element per call and it will paused until another element is requested. When there are no more elements needed, it will simply get interrupted and not run to an end a.k.a. exhausted.

      public static IEnumerable<int> OddNumbers(int max)
	{
    	int i = 1;
        while (i < (max * 2))
            {
                yield return i;
                i += 2;
            }            
	}
    

Execute the following statement:

      foreach (int i in OddNumbers(1000000000))
	{
    	Console.WriteLine($"Number {i}");
	}
    

otice the power of the enumerators and the yield return statement. During execution, the RAM-used stayed around ~10MB which is a vast improvement over the old fashioned return the whole list approach, and the end result was basically the same. One notable difference was that it returned odd numbers and not the even ones like the other function.

Take It or Leave It

When we are working with enumerable datatypes, there is a function which we can apply to takea specific amount of elements from our items.

      foreach (int i in OddNumbers(100).Take(10))
    {
		Console.WriteLine($"Number {i}");
	}
    

This approach gives us only the first 10 elements of the 100 odd numbers that we have ordered from our Enumerable datatype.

The Whole App

I want to provide you with the whole console application, so you can take it for a test-drive.

Note that, in order to transform the function, you need to use the following two extra statements on top of your app.

      using System.Collections.Generic;
using System.Linq;
    

These will provide you with the "Take" function and the capability to have the return type of the OddNumbers function to be set to IEnumerable.

Without this, you will see an error stating that the compiler cannot find the appropriate type/function.

      using System;
using System.Collections.Generic;
using System.Linq;

namespace ConsoleApp
{
    class Program
    {
        public static IEnumerable<int> OddNumbers(int max)
        {
            int i = 1;
            while (i < (max * 2))
            {
                yield return i;
                i += 2;
            }            
        }
        public static List<int> EvenNumbers(int max)
        {
            var retList = new List<int>();
            int i = 0;
            while (retList.Count < max)
            {
                retList.Add(i += 2);
            }
            return retList;
        }
        static void Main(string[] args)
        {
            //The efficient Enumerable version.
            foreach (int i in OddNumbers(100).Take(10))
            {
                Console.WriteLine($"Number {i}");
            }
            // The memory hungry beast
            foreach(int i in EvenNumbers(100000000))
            {
                Console.WriteLine($"Number: {i}");
            }
            Console.Read();
        }
    }
}
    

Conclusion

We often fall into the illusion of believing that, by using ever more powerful computers, we can neglect writing efficient code. Enumerable datatypes provide you with the means to create applications that work with large amounts of data to be kept at bay. Prevent them from eating up all of your systems resources. This technique is widely used across the industry and it can be a very effective method in your programming toolset.