Code and Data in C#

Don Box's Spoutlet

Syndication

I've noticed that at least one really smart guy doesn't get how LINQ works despite my bestattempts to explain
 
Let me try again a bit more slowly this time, dissecting a simple program that elaborates on the code==data theme in both Scheme and C# 3.0.
 
// Line 1: (define f (lambda (a) (+ a 1)))
Func<int, int> f = a => a + 1;
 
// Line 2: (define e (quote (lambda (a) (+ a 1))))
Expression<Func<int, int>> e = a => a + 1;
 
// Line 3: (define elong (list 'lambda (list 'a ) (list '+ 'a 1 )))
ParameterExpression ap = Expression.Parameter(typeof(int), "a");
 
Expression<Func<int, int>> elong =
    Expression.Lambda<Func<int, int>>(
      Expression.Add(ap, Expression.Constant(1)),
      ap);
 
// Line 4: (display (car (car (cddr e)))) ; prints +
Console.WriteLine(e.Body.NodeType); // prints Add
 
// Line 5:(display (elong 4)) ; runtime error: procedure application: expected procedure, given: (lambda (a) (+ a 1))
Console.WriteLine(elong(4)); // compile-time error CS0118: 'elong' is a 'variable' but is used like a 'method'
 
// Line 6: (define f2 (eval elong (scheme-report-environment 5)))
Func<int, int> f2 = elong.Compile();
 
// Line 7: (display (f2 4)) ; prints 5
Console.WriteLine(f2(4)); // prints 5
 
The first two lines are the most important.
 
In both Lisp/Scheme and C# 3.0, there's a "quoting" mechanism that tells the front end to leave your expression as data and not turn it into some opaque yet directly executable function.
 
In Lisp/Scheme, you use the quote special form to turn off the evaluator. The quote special form is often abbreviated with an apostrophe like in my previous post).
 
In C#, the compiler looks at the context in which a lambda expression is being used. If it's being assigned to a delegate type (like Func), then the compiler emits an opaque IL-based method just like it did in C# 2.0.  If it's being assigned to an Expression where TSignature is a delegate type, then it produces an expression tree that is equivalent to the one I later construct by hand and assign to the variable elong (see Line 3).  For completeness, I show how to use Scheme's list function to do the equivalent list construction in a more verbose manner (I decided folks would freak out if I did it all with cons).
 
Note that in Line 4 of our program, we can use the standard mechanisms for traversing structured data to extract any part of the expression. In the Scheme version, I'm using the list destructuring operators car and cddr. In the C# version, I'm using the named property mechanism (LINQ expressions are structured as types with named properties rather than as raw lists).  In both the C# and Scheme versions the values of e and elong are equivalent to one another.
 
Line 5 is the one that trips up a lot of people coming to Lisp or Scheme (or LINQ).
 
When you quote an expression, you wind up with plain old data that you can slice and dice to your heart's content. What you can't do with data is directly execute it.  For that reason, Line 5 is an error that is at compile-time in C# and at runtime in Scheme.
 
To translate the data into something you can execute, both Lisp/Scheme and C# require you to take an explicit step (shown in Line 6). In Scheme, it's the evaluation of the lambda special form that translates the rest of the list into something executable (a procedure object in R5RS-speak). In LINQ, it's calling the Compile() method on the lambda expression.  Both of these operations yield an opaque value that is ready for beta reduction/direct execution.  Armed with this opaque executable value, we're off to the races in Line 7 printing out our beloved "5"
 
Both Patrick and James Robertson allude to the fact that we're marching headlong into the past with this stuff.
 
If only I could wish away a lot of the stuff that distracted/stunted the industry back in the late 80's and 1990's, but I can't.
 
All I can say is that both I and my friend the token Smalltalk weenie anxiously await the past and are doing what we can to get us there :-)

Posted May 12 2006, 05:52 AM by don-box

Comments

Sch wrote re: Code and Data in C#
on 05-12-2006 5:52 AM
I am confused. I am not an expert in LISP and Smalltalk but those have lambda (in specialized form for ST) and I do not see how this Expression object trick is a lambda.

For me, It seems more something more like macro in LISP (but without the simplicity and, then, power).
Matt Warren wrote re: Code and Data in C#
on 05-12-2006 7:05 AM
I cdr had a car, but they don't let ex cons drive.
Kurtiss Hare wrote re: Code and Data in C#
on 05-12-2006 7:15 AM
The difference between code-as-data in C# and lisp is that the representation of C#'s data is bound by it's adherence to a valid syntax tree, that is your data must always be unevaluated (uncompiled, in this case) code. (Is this true?) In lisp, your data is bound by the limitations of arbitrary trees. Some of the nodes in that tree might be purely aesthetic, or their arrangement might be customized to fit a domain-specific view of the data--because those trees can be manipulated as data (and not code). It's far more likely, then, for lisp's code-as-data mechanisms to bubble up to the surface levels of an application, because the programmer there is dealing with pure data, in the form of arbitrary trees. Let's be honest, to the programmer, line 1 is code. It's only data to the compiler and the layer of the application that has to call .Compile() to get it to run.

Aside, I certainly appreciate the strides being made to get these notions out to a mainstream crowd.
Steven wrote re: Code and Data in C#
on 05-12-2006 7:20 AM
"are doing what we can to get us there"

That's the thing--we HAVE been there, and the languages that do the things you are trying to do don't have to resort to ugly syntactic monsterisms in order to accomplish them.

This is NIH at its worst.
mike wrote re: Code and Data in C#
on 05-12-2006 8:12 AM
and people thought that c was complicated, sometimes I wonder if the benefits of these sort of things are not opaque by the time of learn them
Stefan Tilkov's Random Stuff wrote Code and Data in C# and Scheme
on 05-12-2006 8:44 AM
Great post by Don Box &#8212; if C#&#8217;s syntax were a little less ugly, this would actually be very elegant....
Kurtiss Hare wrote re: Code and Data in C#
on 05-12-2006 8:44 AM
[edit]: 3rd to last sentence should read "line 2 is code," not "line 1 is code". :)
Jason Haley wrote Interesting Finds
on 05-12-2006 11:48 AM
Michael wrote re: Code and Data in C#
on 05-12-2006 6:47 PM
This could be really cool, but some of the real power of this type of coding is using it to create DSLs. So is C# 3.0 going to have a macro like facility to do this?
John wrote re: Code and Data in C#
on 05-14-2006 8:52 AM
FWIW, here's an implementation in JScript (for WSH 5.6).

var fa = function( a ) { return a + 1; }
var fb = new Function( "a", "return a + 1" );

var e = "new Function( 'a', 'return a + 1;' )";
var f2 = eval( e );

print( fa( 4 ) );
print( fb( 4 ) );
print( f2( 4 ) );



Baczek wrote re: Code and Data in C#
on 05-14-2006 12:09 PM
That JScript soooo misses the point...
Don Box's Spoutlet wrote Eval !=
on 05-14-2006 1:55 PM
Jafar wrote re: Code and Data in C#
on 08-07-2006 6:48 AM
Very nice. How do you define a recursive function as an Expression? This would be a very powerful technique, especially if the compiler recognized tail recursion.
Lisper wrote re: Code and Data in C#
on 11-26-2006 10:30 PM
<CODE C#>
Func<int, int> f = a => a + 1;
ParameterExpression ap = Expression.Parameter(typeof(int), "a");

Expression<Func<int, int>> elong =
Expression.Lambda<Func<int, int>>(
Expression.Add(ap, Expression.Constant(1)),
ap);

Func<int, int> f2 = elong.Compile();
Console.WriteLine(f2(4));
</CODE>

<CODE Common Lisp>
(print
(funcall (compile nil
'(lambda (a) (1+ a)))
4))
</CODE>

Recursive lambda (Common Lisp):

;; Define macro to hide details
(defmacro lambda-rec ((&rest args) &body)
`(lambda (,@args)
(labels ((this (,@args)
,@body))
(this ,@args))))

;; Example of recursive lambda definition
(lambda-rec (n sum)
(if (> n 0) (this (1- n) (+ n sum)) n)

;; Macro build code:
=> (lambda (n sum)
(labels ((this (n sum)
(if (> n 0)
(this (1- n) (+ n sum))
sum)))
(this n sum))

;; Assign compiled function to variable
(setf compiled-fn
(compile nil
(lambda-rec (n sum)
(if (> n 0)
(this (1- n) (+ n sum))
sum))))

;; Apply function to arguments
(funcall compiled-fn 5 0)

=> 15
TiagoAz wrote re: Code and Data in C#
on 04-15-2007 6:47 PM
<CODE C# 3.0>
Func<int, int> f = a => a + 1;
Console.WriteLine(f(4));
</CODE>

=> 5
Mike Taulty's Blog wrote System.Linq.Expressions
on 04-24-2007 5:30 AM
One of the parts of the "story" that I attempted to tell here&nbsp;that underpins the flexibility we...

Add a Comment

(required)  
(optional)
(required)  
Remember Me?