Phil Gilmore : Programming

Anonymous Methods and Closures in Delphi 2010

Phil Gilmore — Thu, 17 Jun 2010 23:09:00 GMT

06/17/2010
Phil Gilmore

What are anonymous methods?
Anonymous methods are a new language feature introduced in Delphi 2010. To describe them in one sentence is unfair, but I'll try. They are methods which are defined inline (you might call them literal methods) and hence are not called by name. Because they are not called by name they do not have names, hence the "anonymous" title.

Like generics they are simple enough in concept, but the reasons behind them can be elusive. The application of the concept will require some real-world exposure. I will start by showing them and then addressing the questions that I think will enter the reader's mind by showing ever more realistic examples.

What do they look like?

Right off the bat, here is an example.

procedure begin writeln('Anonymous method has executed'); end

This isn't very different from a normal procedure. You'll notice that the procedure has no name. You'll also notice the missing semicolon after the final end. Without a name, we have to call them by reference. To do this, we need a reference to one.

var myAnonymousMethod: TProc;

That's easy. But what's a TProc? Just like a reference to a procedure or procedure of object, a method reference must have a type. The type must be a matching procedure or function declaration (I'll stick with procedures for now). Instead of procedure or procedure of object, anonymous methods match signatures declared as reference to procedure. Here is the definition of TProc, found in SysUtils.pas:

type TProc = reference to procedure;

Of course, they can also be functions and can contain parameters, following the same rules as normal methods. Several are defined in SysUtils.pas. They are listed below.

type TProc = reference to procedure; TProc<T> = reference to procedure (Arg1: T); TProc<T1,T2> = reference to procedure (Arg1: T1; Arg2: T2); TProc<T1,T2,T3> = reference to procedure (Arg1: T1; Arg2: T2; Arg3: T3); TProc<T1,T2,T3,T4> = reference to procedure (Arg1: T1; Arg2: T2; Arg3: T3; Arg4: T4); TFunc<TResult> = reference to function: TResult; TFunc<T,TResult> = reference to function (Arg1: T): TResult; TFunc<T1,T2,TResult> = reference to function (Arg1: T1; Arg2: T2): TResult; TFunc<T1,T2,T3,TResult> = reference to function (Arg1: T1; Arg2: T2; Arg3: T3): TResult; TFunc<T1,T2,T3,T4,TResult> = reference to function (Arg1: T1; Arg2: T2; Arg3: T3; Arg4: T4): TResult; TPredicate<T> = reference to function (Arg1: T): Boolean;

You can also define your own. I've been prone to use the traditional TGetStrProc for events, but this is a procedure of object and therefore does not match anonymous method types. I could use TProc<string>, but that's not very descriptive. If I wanted something more descriptive, I could define one like this:

type TAnonGetStrProc = reference to procedure(value: string);

Comparison to other method types

There are 3 basic types of subroutines in Delphi 2010. Going forward, it would be useful for you to be familiar with them if you aren’t already. Each of the three types are available as both procedures or as functions. The first is the traditional Pascal procedure or function. This is not technically a method, but can be used as a function pointer nonetheless, so I’ll cover it. The second type is a regular method. This is the type to which events are bound. The third type is anonymous methods.

Here is an example of the traditional Pascal procedure or function:

type TWriter = procedure; procedure CustomWriter; begin writeln('Display this text'); end; var x: TWriter; begin x := CustomWriter; x; readln; end.

Here is an example of a method:

type TWriter = procedure of object; type TCustomClass = class(TObject) public procedure CustomWriter; end; procedure TCustomClass.CustomWriter; begin writeln('Display this text'); end; var x: TWriter; begin with TCustomClass.Create do begin x := CustomWriter; x; Free; end; readln; end.

Here is an example of an anonymous method. Ignore the details for now and just pay attention to the declaration as it compares to the other two types.

type TWriter = reference to procedure; var x: TWriter; begin x := procedure begin writeln('Display this text'); end; x; readln; end.

We will cover the anonymous method example in the next section. But for now, you can see that each of these method types has a different way of declaring its type and the actual method that matches that type’s signature is also declared differently for each type. The firs two types have always been there and should be familiar to you. If not, this is something you should learn before reading further.

Anonymous methods as variables

An anonymous method doesn't have a name, so it must be used with a reference. That reference can be a variable to which the method is assigned. Here is an example.

var myAnonymousMethod: TProc; begin myAnonymousMethod := procedure begin writeln('This was written from an anonymous method'); end; // ... end.

You can now make further assignments by assigning the myAnonymousMethod to another reference. This reference can be another variable, a method parameter, a class field, etc. As long as they are all the same type, assignments can be made as needed. In this assignment, the line ends with a semicolon. This semicolon is denotes the end of the assignment line itself; it is not part of the anonymous method.

Invoking an anonymous method

Let's finish our variable reference example and invoke the method.

var myAnonymousMethod: TProc; begin myAnonymousMethod := procedure begin writeln('This was written from an anonymous method'); end; myAnonymousMethod; // Invoke the method. end.

You can see that this code is not really useful, but it illustrates the following:

Declaration of an anonymous method reference
Assignment of an anonymous method to a reference
Invocation of an anonymous method through its reference.

Because anonymous methods are so natively tied to a reference, it makes sense to use them mostly where passing references is easy to do. Method parameters are class fields are good examples. Let's expand our example to use the TAnonGetStrProc prototype and a simple logging example.

Anonymous methods as parameters

So far, our references have only been variables. Modifying the variable example, we can pass a variable reference to an anonymous method as a parameter to another method. This is far more useful than previous examples, as you'll see.

type TAnonGetStrProc = reference to procedure(value: string);

type TMyClass = class public procedure DoStuff(aLogMethod: TAnonGetStrProc); end; procedure TMyClass.DoStuff(aLogMethod: TAnonGetStrProc); var i: integer; begin try // Throws an exception. i := StrToInt('invalid integer'); except // Log the error using the method that was passed in. // We don't have to define it in this class. aLogMethod('Error encountered, logged by anonymous method'); end; end; var LogMethod: TAnonGetStrProc; begin // Log method body is defined outside of any code that uses it. LogMethod := procedure(value: string) begin Writeln(value); end; // Semicolon is NOT part of anonymous method. // Here, we execute some code and PASS a reference to the log method. with TMyClass.Create do begin DoStuff(LogMethod); Free; end; readln; end.

This is our first useful example. It shows that a method can be passed around and used by other methods in other places. These methods can then invoke this foreign code and pass it parameters if appropriate. We have therefore illustrated inversion of control, callback functionality and code reuse.

Inline declaration

In the previous example, the log method's body is defined OUTSIDE the method that calls it. Instead of the reference being a local variable, it is a method parameter. Okay, so what's so great about that? We can define regular event methods that do that, right? Remember that I said that anonymous methods are declared inline. They don't have to live inside a class at all like a real method. This means that we could change up the initialization section to get rid of the LogMethod variable altogether.

initialization with TMyClass.Create do begin DoStuff( procedure(value: string) begin Writeln(value); end); Free; end; end.

See that the first parameter is not a variable, but the procedure body itself is stuffed in the call parameter list. This is the cool thing about them. They can be declared on the spot and passed around like any piece of data. They are first-class citizens of the Delphi language. the DoStuff procedure could even pass it to another method that it calls.

Inversion of Control (IoC)

Let's focus on the essence of the logging example for a moment. The DoStuff method has the ability to log its errors. It has no knowledge of how this logging is done. A method was given to it which presumably knows what to do with a string it is given by the DoStuff method.

This method may write the string to a log file on disk, write it to the Windows Event Log, a database, or send it to a logging service like SmartInspect or CodeSite. This method may do nothing with the string at all. The point is that the DoStuff method knows nothing about that. It has no control over the logging. It can't decide whether to log to a file, the event log or a logging service. The decision lies elsewhere. This is the fundamental concept of Inversion Of Control (or IoC for short).

Inversion of control was possible using standard function pointers, events and object methods in all previous versions of Delphi. However, the concept of IoC was not in vogue until a few years ago. Anonymous methods don't do anything you couldn't do before, but they can be easier than the alternatives.

Class fields

Instead of method parameters which must be passed around and can affect existing method signatures, refactoring may be done by storing anonymous methods in class fields. Here is our previous example, converted.

type TAnonGetStrProc = reference to procedure(value: string); type TMyClass = class protected fLogMethod: TAnonGetStrProc; public constructor Create(aLogMethod: TAnonGetStrProc); procedure DoStuff; end; implementation constructor TMyClass.Create(aLogMethod: TAnonGetStrProc); begin fLogMethod := aLogMethod; end; procedure TMyClass.DoStuff; var i: integer; begin try // Throws an exception. i := StrToInt('invalid integer'); except // Log the error. fLogMethod('Error encountered, logged by anonymous method'); end; end; initialization with TMyClass.Create( procedure(value: string) begin Writeln(value); end) do // No semicolon after end begin DoStuff; Free; end; end.

In the approach above, the DoStuff signature does not have to have a method parameter for the log method. Instead, the method can use the fLogMethod field, which is initialized in the constructor. No cleanup is needed in the destructor. Now any methods that you add to the class can use the log method without a special parameter just for that. Most importantly, existing methods don't have to have their signatures modified when adding this log method functionality to an existing class.

Note too (as indicated in the comments of the above example) that the anonymous method itself is not followed by a semicolon. I've mentioned this before but it can take some reminding to get used it. It can be confusing because you have seen me repeatedly end an anonymous method with a semicolon, but if you go back and look, the semicolon is there to terminate the assignment of an anonymous method to a variable. It is NOT part of the anonymous method itself.

Generic anonymous type declarations

Before going further, I will briefly cover the generic procedure types which I mentioned earlier. In the next section, I have an example in which I use a variable declaration like this:

var ShowCounter: TProc<Integer>;

This is the notation used by generics. To make sense of it, look for the declaration of TProc<T> in SysUtils.pas. When I declare something of type TProc<integer>, it just means that I use the declaration of TProc<T>, but Delphi replaces all the Ts in the declaration with integer when I use the ShowCounter variable. In other words, given this declaration:

TProc<T> = reference to procedure (Arg1: T);

... my declaration of TProc<integer> tells Delphi that ShowCounter uses the following declaration (which the compiler creates at runtime):

reference to procedure (Arg1: integer);

This is just a way of using an existing method prototype declaration for my anonymous method instead of declaring a new one for myself. It is the same as creating a new type...

type TIntegerProc = reference to procedure (Arg1: integer);

But because TProc<T> is already declared, I can use that instead.

Closures

Before I get into closures, I want to address the constant use of the word as a misnomer. The term Closures is very frequently treated as a synonym for the term anonymous methods. Though closures are specific to anonymous methods, they are not the same concept. Anonymous methods are possible without closures. Do not use the terms interchangeably.

You may have noticed that so far none of my examples had any variables inside the anonymous methods. This is because anonymous methods have some considerations for variable scope. The first rule is that all variables which are declared inside the anonymous method are safe and will adhere to all the rules with which you are familiar. Variables declared outside the anonymous method are the ones you need to examine.

In traditional Pascal, anything in a VAR block which exists physically above the code which references is visible. These are called nested methods.

procedure OuterMethod; var upperScope: string; procedure InnerMethod; var innerScope: string; begin // Here, upperScope and innerScope are visible. // lowerScope is not visible. writeln(text); end; var lowerScope: string; begin // upperScope and lowerScope are visible here. InnerMethod; end;

This works because the InnerMethod only lives in one place and is not visible anywhere except in the OuterMethod body. But when using anonymous methods, the methods can be passed to another place where the upperScope variable is not declared. To accommodate this, Delphi provides closures. Closures create another scope which includes any symbols used by the anonymous method but which are not declared inside the method itself. This scope is then bound to the anonymous method and follows it everywhere it goes.

Without closures, You would use an an upper variable using parameters, like this:

procedure TMyClass.DoStuff; var j: integer; ShowCounter: TProc<Integer>; // We covered this type in the previous section. begin ShowCounter := procedure(value: integer) begin // j is unknown here because it is out of scope. // We output a parameter instead. writeln(IntToStr(value)); end; for j := 1 to 10 do ShowCounter(j); // Pass j as a parameter end;

You see that the anonymous method has a parameter through which I pass the current counter value. The writeln statement will output the value parameter instead of the j variable. This would be necessary because j does not exist inside of the anonymous method which we assign to ShowCounter.

We don't have to do things that way. Because Delphi provides closures, we can have direct access to j counter from with the anonymous method without using a parameter. Delphi will place j in the scope of the method and make it available to us without any work on our part.

procedure TMyClass.DoStuff; var j: integer; ShowCounter: TProc; // Method no longer has a parameter. begin ShowCounter := procedure begin // j is known here because it is wrapped in a closure // and made available to us! writeln(IntToStr(j)); end; for j := 1 to 10 do ShowCounter; // j is no longer passed end;

So now you're wondering why I just spent all this time explaining closures when it works just like a nested method anyway and you don't have to take any steps to make it work or turn it on etc. The reason is because the closure is not really direct access to the symbols in the outer scope. There are limitations. The compiler has to make decisions about where the data comes from and what it can do with that data. Sometimes it will determine that a closure is not possible and will give you an error.

For example, you cannot have a for loop which uses a counter which exists in the closure. Again, modifying the previous example...

procedure TMyClass.DoStuff; var j: integer; ShowCounter: TProc; begin ShowCounter := procedure begin for j := 1 to 10 do writeln(IntToStr(j)); end; ShowCounter; end;

This seems perfectly safe, but the compiler knows better. Some compiler optimizations for loops require the counter to live on the stack and this one lives inside a closure instead. When you run the code above, the compiler yields this error:

[DCC Error] Project1.dpr(30): E1019 For loop control variable must be simple local variable

The bad news? It won't work. The good news? There is a very good compiler error to tell you what's wrong and there is an alternate way to do the same thing. The counter must be local, as the error indicates.

procedure TMyClass.DoStuff; var ShowCounter: TProc; begin ShowCounter := procedure var j: integer; // j is now local. begin for j := 1 to 10 do writeln(IntToStr(j)); end; ShowCounter; end;

Because of such considerations, it might be a good idea to avoid closures where possible. But it is important to understand how they work so that you know when you're using them. Because they are so effectively transparent, a solid understanding is necessary to use or avoid them effectively.

Result in closures

The Result value cannot live in a closure. For example, the compiler tells you you cannot do this:

function TMyClass.AddDays(aDays: integer): TDateTime; var rc: TDateTime; p: TProc; begin rc := now; p := procedure begin // Result can't live in a closure. result := rc + aDays; end; p; end;

You get this error, again very descriptive and deliberate:

[DCC Error] Project1.dpr(34): E2555 Cannot capture symbol 'Result'

Instead, you must fix it like this:

function TMyClass.AddDays(aDays: integer): TDateTime; var rc: TDateTime; p: TProc; begin rc := now; p := procedure begin // rc can live in a closure. rc := rc + aDays; end; p; result := rc; end;

Note that this means that result cannot live only in CLOSURES; the limitation does not apply to anonymous methods themselves. If you are using a an anonymous function (discussed next), result is allowed because it is local to the anonymous function, not something which needs to be captured in the closure. Since result does not apply to procedures, it is assumed that a reference to result inside an anonymous procedure is an attempt to reference the return value in outer scope, hence it requires closure.

Anonymous Functions

So far, I have been careful to use only anonymous procedures. But as you have seen in SysUtils.pas, there are also anonymous functions. They work identically, except that there must be a return type in the type definition and in the inline method body itself.

Here is an anonymous function type definition.

type TIntegerConvertFunc = reference to function(s: string): integer; // Has a return type

Here is an anonymous function.

var myFunction: TIntegerConvertFunc; begin myfunction := function(s: string): integer begin result := IntToStr(s); end; // ...

Generic function declarations warrant mentioning. We've seen the TProc and TProc<T> declarations in SysUtils.pas. However, functions have a caveat. Because the functions have a return type, generic function declarations usually have a parameterized return type. This means that by convention, the last parameter type of any generic function declaration is the function return type, and therefore every generic function declaration has at least one parameter because a function must always have a return type.

Note that I say that this is by convention only. Non-generic function declarations obviously don't apply, and even generic function declarations may have a concrete return type. Furthermore, the type parameter order may not adhere to this convention. You'll find that VCL and .NET both adhere to this convention.

Again here is an overview of the VCL function types:

TFunc<TResult> = reference to function: TResult; TFunc<T,TResult> = reference to function (Arg1: T): TResult; TFunc<T1,T2,TResult> = reference to function (Arg1: T1; Arg2: T2): TResult; TFunc<T1,T2,T3,TResult> = reference to function (Arg1: T1; Arg2: T2; Arg3: T3): TResult; TFunc<T1,T2,T3,T4,TResult> = reference to function (Arg1: T1; Arg2: T2; Arg3: T3; Arg4: T4): TResult;

First, note that there are no TFunc declarations with no parameters. That is, they all have a return type parameter. The result parameter is always named TResult, also by convention. From top to bottom, all these function prototypes are identical except for the number of input parameters. There is a function for all input parameter counts from 0 to 4. If you need more than that, you can declare a new type or perhaps refactor to simplify things.

As an example, an anonymous method matching the last prototype might look like this:

var i: integer; myFunction: TFunc<string, string, string, string, integer>; begin myFunction := function(s1, s2, s3, s4: string): integer begin result := StrToInt(s1) + StrToInt(s2) + StrToInt(s3) + StrToInt(s4); end; i := myFunction('1', '2', '3', '4'); writeln(inttostr(i)); readln; end.

You might find, however, that using generics it is easy to simplify this example using a list.

myFunction := function(aValues: TList<string>): integer var j: integer; begin result := 0; for j := 0 to aValues.Count - 1 do result := result + StrToInt(aValues[j]); end;

Anonymous method names

I know... I said they don't have names. More specifically, the programmer does not give them names nor can someone refer to them by name. But internally, the compiler generates code for them just as any other method. When it reports an error, it tries to give you some rough idea of the method to which the error applies. It uses some information from its internal symbol tables to create this. It is useful for you to have an idea what they look like in case you get any errors and don't know what you're seeing. To see one of these, compile this code:

var f: TFunc<integer>; begin f := function: integer begin end; end.

There is no return value in the anonymous function, so you will get a warning about it. It shows you the following.

[DCC Warning] Project1.dpr(72): W1035 Return value of function 'Project1$ActRec.$0$Body' might be undefined

'Project1$ActRec.$0$Body' is the name that it gives to the method in the main body of a console application saved as Project1.dpr. There is no predicting these names and they will change in unexplained ways between projects, etc. Do not attempt to use RTTI to get a handle on these puppies by name. Let the compiler keep them to itself.

Prototype mismatch

If your anonymous method definition doesn't match the prototype of the parameter or variable to which you are assigning it, you will receive a less than perfect error. For example, here is the prototype for Generics.Defaults.TComparison<T>:

TComparison<T> = reference to function(const Left, Right: T): Integer;

Note that the parameters are declared with the const keyword. If you don't notice, you might declare a method like the following and assign it to a TComparison<string> variable.

var c: TComparison<string>; begin c := function(left, right: string): integer begin if (left < right) then result := -1 else if (left > right) then result := 1 else result := 0; end; end.

In this case, I get an internal error.

[DCC Fatal Error] Project1.dpr(16): F2084 Internal Error: AV21F2707E-R0000000C-0

We go back to see what we did wrong and find that we left out that const keyword in the function body. Furthermore, if we make this declarative mistake in other contexts, we'll get a different error. In the following case (indeed most cases), we’ll get a slightly more useful error.

type TMyClass = class(TObject) public procedure DoStuff; end; var c: TComparison<string>; { TMyClass } procedure TMyClass.DoStuff; var c: TComparison<string>; begin c := function(left, right: string): integer begin if (left < right) then result := -1 else if (left > right) then result := 1 else result := 0; end; end; begin end.

[DCC Error] Project1.dpr(33): E2010 Incompatible types: 'TComparison<System.string>' and 'Procedure'

This error could probably use improvement because the Procedure part never gets any more specific. Obviously, our anonymous method isn't a procedure; it's a function. So the first thing you may do is wonder if the error is referring to some other piece of code (a bug). No, indeed it is referring to your anonymous function. There is nothing wrong with that anonymous function by itself. It just doesn't match TComparison<string>.

It would be most useful if the error read something like this instead:

Incompatible types: 'TComparison<System.string>' and 'reference to function(string, string): integer'

... or even better ...

Incompatible types: 'reference to function(const string, const string): integer' and 'reference to function(string, string): integer'

Again, adding the const keyword fixes it up. In cases like this, copy and paste can save you some confusion. It's usually a dumb mistake or oversight like that.

Predicates

A special type of method is a predicate. A predicate is a filter condition. This means that conventionally, the implementation of a predicate is a function with any number of parameters which returns boolean. SysUtils.pas defines the following for you to use for predicates:

type TPredicate<T> = reference to function (Arg1: T): Boolean;

Unfortunately, a grep reveals that this type is used nowhere in the entire VCL. As useful as it is, I find it a tragedy that the new Generics.Collections unit isn’t riddled with methods accepting parameters of this type. That can’t stop you though. Any time you want to allow a method to filter items in a collection, a predicate is a good pattern to use.

Lambda Expressions

Unfortunately, Lambda expressions are not available in Delphi. While I am confident that they will be made available eventually, it is one of the features which I miss the most.

Because I think that it might be useful for users of other languages, I will cover them a bit here, in Prism and C# Syntax. I suspect that Delphi will adhere to the Prism syntax if this feature ever appears in a future version.

One of the questions that comes up when learning anonymous methods is how they can be useful when they're no less verbose as traditional methods of IoC and delegation. The answer is that anonymous methods don't have to be so verbose. Lambda expressions are a popular shorthand syntax for anonymous methods.

I do a lot of C# and have reaped the benefits of Lambda Expressions and anonymous methods together. You have seen how well anonymous methods play with generics. Add Lambda expressions to the mix and things get very fancy very quickly.

Given a function like this:

function(s: string): integer begin result := StrToInt(s); end;

There are three main parts of the function.
- The parameter list
- The return type
- The method body

A Lambda expression is just a shorthand expression of these three elements. The first element is a parameter list. Fundamentally enclosed in parentheses, the parentheses can be omitted if there is only one parameter.

s: string

The next is the method body. Because there is only one line and it is the result assignment, the assignment to result can be omitted, leaving just an expression to be evaluated. Hence, the "result :=" is removed, leaving:

StrtoInt(s);

Put them together using the "hat" operator, which is read out loud as "such that" or "is a function of" (or whatever else you want), you get this in Prism and C#, respectively:

// Prism s: string -> StrToInt(s) // C# string s => Int.Parse(s)

In C# and to some extent in Prism, the type of the parameters and the function result can be determined very effectively by the compiler based on context, alleviating the need to specify these types. This is called "Type Inference" and further simplifies the code:

// Prism s -> StrToInt(s)

// C# s => Int.Parse(s)

Multiple statements and parameters can be used, but much of the simplification goes away. Here are examples again in Prism and C#, respectively.

// Prism (s1, s2) -> begin var i := StrToInt(s1) + StrToInt(s2); result := i; end

// C# (s1, s2) => { int i = Int.Parse(s1) + Int.Parse(s2); return i; }

While functions are usually used with Lambda expressions, procedures can be used too. If the expression does not evaluate to anything, it generally requires that the result not be assigned to anything.

Parameterless functions and procedures can also be used. Usually, an empty parameter list is provided with a pair of parentheses, ( () –> … ).

Phil Gilmore (www.interactiveasp.net)

VDUG: Delphi Generics Presentation available for download

Phil Gilmore — Wed, 19 May 2010 19:45:00 GMT

05/19/2010

Phil Gilmore

I presented Delphi Generics for the May Virtual Delphi User Group. A recording with tips ‘n tricks by Robert Love is available as a streaming video(Flash) or as a full download (MP4) at the VDUG website:

http://video.vdug.org/2010/May/VDUG%20-%20May%202010.html

Phil Gilmore (www.interactiveasp.net)

Getting started with MyGeneration, a primer and tutorial

Phil Gilmore — Fri, 04 Dec 2009 00:36:00 GMT

12/03/2009

Phil Gilmore

About MyGeneration

MyGeneration is a developer's code-generation tool that gives you access to your database’s schema. It is useful for building O/RM entity classes based on your database schema.

MyGeneration is mature, free and open-source. It supports many databases. It supports scripting in many languages (C# script VB.NET script, jscript, vbscript, etc.). It supports output into any language. You can write your own templates or use someone else’s.

There are many templates available. Some come with the installer, but many more can be browsed and downloaded from the internet right in the MyGeneration main window.

I found it easy enough to get started using someone else's downloadable templates, so I will only take a brief moment to describe that. I will then spend more time explaining how to write your own templates.

The MyGeneration main page is here and has the latest download link to download.com.

http://www.mygenerationsoftware.com/portal/default.aspx

As the this writing, the download link points here. This download is an NSIS installer package. I recommend installing all components.

When you first run MyGeneration, you are presented with the Default Settings screen. The purpose of this window is to give you a place to define your database connection.

Using the OLEDB configuration wizard (1, see image below), you can easily produce a working connection string. You can also get a generic one here to get your started if you have trouble. Or, enter the string manually if you have the aptitude for it. However you do it, the point is to get the connection string into the connection string field (2).

Once you test your connection string (3) and can connect, type a name for your connection in the box above (4) and save it (5). Now you can always refer to this connection string by selecting it in the dropdown list (4) and clicking the load button. Set the language mapping to your preference (i.e. C#). The other settings are usually left as default. Click the Save button at the top of the page and you may continue.

MyMeta

The MyMeta library is the body that fetches metadata from your database engine. It’s the part that must be aware of your particular database engine in order to support it. You can see what MyMeta can see by opening the MyMeta browser window and viewing its tree contents. If you don’t see your database schema in the MyMeta browser, your connection isn’t set up correctly or you don’t have permissions to see the things you’re looking for. In any case, the data won’t be accessible for code generation unless it is visible in the MyMeta browser.

Templates

There are two ways to do code generation in MyGeneration. The first is to use someone else’s template. The second is to write your own. Most folks who are using a mainstream O/RM such as NHibernate and a mainstream language such as C# can pick a template that works with both and be done in a few minutes. Others can write some templates in a few hours for any systems they want to use, as long as MyMeta supports their database engine.

Using someone else’s templates

In case you want to just get some code generation done and be done with this, I’ve put this here so you don’t have to wade through the rest of this document to get your work done.

MyGeneration’s installer will give you a handful of templates that you can use. These are limited and probably don’t have what you want.

One of the nice things about MyGeneration is that you can browser for more templates online without leaving the application. Click the globe icon in the template browser to find more templates online…

Now we’re talking! You now have the latest templates. It isn’t always easy to find what you’re looking for, but usually they’re categorized pretty well. You may look for a template by the language you want to generate, or you may search by the database engine or data access framework you’re using, such as NHibernate.

Double-click a template to see some pretty good detail about it if you need to know more about it. When you’ve found a template you like, click the save button and continue for more templates or close the Online Template Library window when you’re done.

The templates you saved will now appear in your regular Template Browser. You can right-click on the desired template in the Template Browser, click EXECUTE from the popup menu, and code generation will begin.

What happens at that point depends on the template. Many templates will need you to give them some input parameters to do their job. When this is the case, they will prompt you for parameters with their own popup form. When it is complete, you will have output in the output window, or the output will be saved to a set of files. This is also specific to the template. Most templates that output to files will prompt you for an output location.

Modifying someone else’s template

If you followed the steps above in “Using someone else’s templates”, you know how to get your hands on a template that you can modify. Take any template in your template browser and double-click it. The template can be modified right inside the MyGeneration application. If you’re modifying someone else’s template, I cannot give you very specific direction. From here on, I will only give direction about how to create a new template.

Creating your own template

Relax. This really is easy. You can pick from several languages and probably won’t have to learn a new language to do this. There is code insight, useful error messages and bugs are easy to fix.

Here some more information about MyGeneration that you will need to know. First, MyGeneration has several systems. We’ve already covered MyMeta, MyMeta Browser, Template Browser, and Online Template Browser. There are separate help files for different systems too. This is useful to know because you will be using these help files from time to time. The help files really have good information, but it can be hard to find what you’re looking for, especially if you don’t know which help file to look in.

The systems you’re primarily concerned with are Zeus, and MyMeta. Unless you are using any of the other systems listed under Help.

Zeus is the scripting engine that MyGeneration uses. It is the engine that takes your templates, executes them to create a program for code generation and then executes that code generation program and either passes the output back to MyGeneration or outputs it to a file (if the script tells it to). Zeus can use several languages. It supports C# Script, VB.Net Script, VBScript and JScript. I will proceed under the assumption that you are using C# Script.

MyMeta, as we have discussed, is a set of objects that your Zeus script can ask for when it needs information about your database schema. There are many object types in this library and you will probably want to just browse over the help file for it to familiarize yourself with some of the interfaces that you will be using. They are mostly self-evident.

The Zeus engine has an output object that to which the generated code will be written. This output would be sent to the Output window by default. However, many templates will redirect this output to a file. If that happens, the output may not appear in the output window.

The Zeus engine has an input object that can be queried for parameters that the user enters are runtime. Runtime is the time at which the Zeus script is executed. Using the input object is optional.

To create a new template, click File, then New –> C# Zeus Template. The template will appear in a new tab on the main MyGeneration panel. Inside that “Untitled” panel are 4 more tabs.

Tab	Description
Template Code	You write this script. This script builds the Template Source script.
Interface Code	You write this script. It builds a UI for gathering parameters from the user.
Template Source	This script is generated and you do not touch it. This is the code generation program. This is the output of the Template Code script. Look here if you have errors if your Template Code script.
Interface Source	This script is generated and you do not touch it. As the TemplateCode generates the Template Source, so too does the Interface Code generate the Interface Source. Use this to debug your Interface Code.
Output	If the output from the Template Source is not redirected to a file, you will see it here when you execute the template.

As you can see, you will do your editing primarily in the template code tab. If you build a UI, you will also use the interface code tab.

In its simplest form, a template can be comprised simply of template code which accesses the MyMeta object as input and the output object as output. Next, you will want to redirect output to files, then finally build a UI. The data gathered in the UI is made available to the template code via the input object.

Let’s start with something simple. In the Template Code and Interface Code tabs, you will see some default code. Keep it. You will need it. Run the template (F5). You see the output in the output tab. Great! Back at the Template Code window, find the Render() method. This is where you will write your code. It is your entry point. You can write your own methods outside of it and call them from the Render method.

Server tags

<% and %> tags define blocks of code within the Template Code. As in ASP or JSP, this is technically a code file, but anything outside the server tags is output as a literal. Note that in the Template Code window, there is literal region that looks like this:

        %>        
You can toggle in and out of script like this         
<%

Take a look at the Template Source tab. You’ll see that this literal has been converted to this:

output.writeln("You can toggle in out of script like this");

To keep things simple for now, I removed indentation of this literal in the template code window. You will find that line breaks and tabs will be adhered to in these literal regions. For finer control, you can use direct output in the Template Code like this:

        output.writeln("You can toggle in out of script like this");        
%>         
You can toggle in out of script like this         
<%

The output becomes:

        output.writeln("You can toggle in out of script like this");        
output.writeln("");         
output.writeln("You can toggle in out of script like this");

Of course, if you only did your literal output in this way, your template code would be identical to your template source. But until you get used to it, you can do this to fine tune your whitespace.

You can also dereference a single symbol from your code without using a call to output.writeln() using the <%= %> tags. For example:

        int keyCount = input.Keys.Count;        
int j = 0;         
foreach (string inputKey in input.Keys)         
{         
    j++;         
    string inputValue = input[inputKey].ToString();         
    %>Input key <%=j.ToString()%><%         
    %> of <%=keyCount.ToString()%><%         
    %>: <%=inputKey%><%         
    %> = <%=inputValue%><%="\r\n"%><%         
}

The Dereference Symbol tag pair is equivalent to the output.writeln. Notice that you never put a semicolon between these tags. To do so would generate something like this in the Template Source. So this tag:

        <%=j.ToString();%>

Would generate this:

        output.write(j.ToString(););

While it’s obvious to see how the premature semicolon breaks the generated code, it is still sometimes hard to remember to omit your trailing semicolons when using the Dererence Symbol tags. Understanding that the text is output verbatim to an output.writeln() call makes it easier to keep things straight.

Using MyMeta

Okay, so we could output anything we want now, but we don’t have much to do without some input. The input is going to start from our database schema. This comes from the MyMeta object. MyMeta is a global object instance of the MyMeta library and is available throughout your script. Have a look at what it can do…

The databases property will give you a collection of databases. From there, you can drill down into tables, columns, keys, stored procedures, etc. To get a single database, first you have to access it by index from the MyMeta.Databases collection.

MyMeta.Databases["Northwind"]

Type the line above in the Render method. This references the Northwind database. For RDBMS servers like Oracle, there may be one or many databases. For local databases, this may not apply.

Now that we are referencing a database, you should be able to reference schema information for the tables on that database. You can drill down from the database reference to its tables collection. But lo, press the period key at the end of that line, and no code insight comes up. The code insight is useful but limited. It does not show you types, and sometimes it doesn’t know the type of the subject and can’t show its members.

Workaround – If you assign that reference to a strongly typed variable, the variable gives the code insight enough information to show you its members. There’s just one problem. You don’t know the type either. You’ll have to guess. This sounds terrible, but it’s not that bad. Remember I said you need to be familiar with the help files? Open the MyMeta help file from the MyGeneration help menu.

Right there in the root you can see a list of interfaces provided by MyMeta. You can easily guess which type our reference is. It is an IDatabase.

Now we have the next level of code insight. This is only required for code insight. Instead of this (which provides code insight):

        IDatabase db = MyMeta.Databases["Northwind"];        
ITable tbl = db.Tables["table1"];         
IColumn col = tbl.Columns["id"];         
output.writeln(col.Name);

This would work too. But you have to be more familiar with MyMeta object hierarchies.

        output.writeln(        
    MyMeta         
        .Databases["Northwind"]         
        .Tables["table1"]         
        .Columns["id"].Name         
);

Most collections in MyMeta have named members and can be accessed by ordinal index or by named index:

        MyMeta.Databases[0];        
MyMeta.Databases[“Northwind”];

You now have all the tools you need to build and run a useful code generation template using MyGeneration.

Redirecting output

Expect to be executing your templates repeatedly during development of the template itself and periodically during development of whatever project consumes your generated code. It becomes cumbersome to copy the contents of the output window from the clipboard (MyGeneration automatically copies it to the clipboard when it writes to the output window) over to an editor to overwrite previous output. It would be convenient to write output directly to a file on disk. The output object can do this. It can also write to multiple files.

For more detail on the output object, see the Zeus help files under IZeusOutput and ZeusOutput class. But here is a basic use of output to files.

        // At first, our output buffer is empty.  Add something to it.        
output.writeln(“Something”);         
        
// Save the current buffer to 2 different files.         
output.save(path1, “o”);         
output.save(path2, “o”);         
        
// Start a new buffer and save it to a third file.         
output.clear();         
%>Something else<%         
output.save(path3, “o”);         
        
// Start a new buffer and since we don’t clear it, it shows up in the output window.         
output.clear();         
%>Wrote to: <%=path1%><%         
%>Wrote to: <%=path2%><%         
%>Wrote to: <%=path3%><%

You may have noticed that I’m mixing %>literals<% (outside the server code) with explicit writeln.output() calls. Remember that these are interchangable and you’ll want to shoot for consistency most of the time. I just mixed them up to show some variety in my examples.

The second parameter of the save() method (“o”) is not documented under IZeusOutput very well. They are found under the ZeusOutput class, but poorly. I’ll document them here to save you time.

Value	Description
“o”	Overwrites an existing file.
“b”	Overwrites existing file but backs it up first.
“a”	Appends an existing file.
“d”	Do not overwrite (silent failure).

Building a UI for your template.

If you need parameters for your template execution they can be gathered at runtime by a UI which you build into your template. The UI code is written in the editor under the Interface Code tab. Again, the default sample code should be kept to get you started. The Setup() method is your entry point. In here, you have a global object named ui. It is of type Zeus.UserInterface.GuiController. You will find help for it in Zeus help under Zeus.UserInterface.

In a nutshell, you have some Addxxxx methods to create UI elements such as buttons and comboboxes. These methods return objects of the corresponding type (IGuidButton, etc.). Those objects then have layout properties such as top, bottom, width, etc. To keep things simple, the IGuiController ui is very good as choosing default layout parameters for you. if you add an element to the UI, it will appear in vertical order relative to other elements you have added. It will fill the form horizontally. While this isn’t always pretty, it makes it easy to get started.

First thing’s first. The interface is disabled for now. Uncomment the contents of the Setup() method to see what happens.

        public override void Setup()        
{         
  ui.Width  = 100;         
  ui.Height = 100;         
  GuiLabel lblDemo = ui.AddLabel("lblDemo", "Demo", "Demo Tooltip");         
  ui.ShowGui = true;         
}

That wasn’t hard. The code sizes the form (lines 1 and 2), adds a label (line 3) and displays the form (line4). To expound on this, we’ll keep lines 1, 2, and 4. We’ll increase the size of the form and replace line 3 with our customized form code.

        public override void Setup()        
{         
  ui.Width  = 500;         
  ui.Height = 300;         
   
  // Customize things here...         
   
  ui.ShowGui = true;         
}

Next we’ll add some controls that are actually going to be useful in this sort of project. We’ll add a database selection combo box and an output file selector (with labels). I’ll add them to their own method to keep the Setup() method cleaner. Remember that the ui object creates the controls for us. This is a bit easier than MyMeta. These functions have code insight that tells us the return type.

Easy enough? This won’t take long.

        public void CreateControls()        
{         
  // Database selection drop-down.         
  GuiLabel lblSelectDatabase =         
   ui.AddLabel("lblSelectDatabase", "Select a database:", "");         
  GuiComboBox cboSelectDatabase =         
   ui.AddComboBox("cboSelectDatabase", "Select a database");         
    
  // Output file picker         
  GuiLabel lblOutputFile =         
   ui.AddLabel("lblOutputFile", "Output filename: ", "");         
  GuiTextBox txtOutputFile =         
   ui.AddTextBox("txtOutputFile", "", "Output filename");         
  GuiFilePicker fpOutputFile =         
   ui.AddFilePicker("fpOutputFile", "...", "", "txtOutputFile", false);         
}

Notice that when passing a control as a parameter to an Addxxx() method, it expects the control name as a string, not the object reference for that control. To keep things straight, I always make the string ID parameter which I pass to the add method identical to the object reference to which I assign the ui element (lblOutputfile = ui.AddLabel(“lblOutputFile”…) for example).

Well, that creates the form controls, but they have no data behind them. You will frequently need to bind MyMeta elements to your UI elements. We will want to do so with our database selection combo box.

        cboSelectDatabase.BindData(MyMeta.Databases);

How easy was that? The layout is still a little ugly, but we now have a working UI. The next step is to use it from the Template Code. We’ll get to that in a moment.

UI Layout

Right now, let’s fix up the layout a bit for sanity’s sake. This is easy too. Just set the width, height, top and left properties of each UI element. You can calculate them in conjunction with the width and height properties of the ui object.

Here is the finished UI. As simple as it is, it may be all you need for some projects.

        public class GeneratedGui : DotNetScriptGui        
{         
    // Declare UI elements in class scope.         
    GuiLabel lblSelectDatabase;         
    GuiComboBox cboSelectDatabase;         
    GuiLabel lblOutputFile;         
    GuiTextBox txtOutputFile;         
    GuiFilePicker fpOutputFile; 
          public GeneratedGui(ZeusContext context) : base(context) {} 
          //-----------------------------------------        
    // The User Interface Entry Point         
    //-----------------------------------------         
    public override void Setup()         
    {         
        ui.Width  = 500;         
        ui.Height = 300;         
        ui.Title = "Building our first UI!";         
        // Customize things here...         
        CreateControls();         
        LayoutControls();         
        DataBindControls();         
        ui.ShowGui = true;         
    } 
          public void CreateControls()        
    {         
        // Database selection drop-down.         
        lblSelectDatabase =         
            ui.AddLabel("lblSelectDatabase", "Select a database:", "");         
        cboSelectDatabase =         
            ui.AddComboBox("cboSelectDatabase", "Select a database"); 
              // Output file picker        
        lblOutputFile =         
            ui.AddLabel("lblOutputFile", "Output filename: ", "");         
        txtOutputFile =         
            ui.AddTextBox("txtOutputFile", "", "Output filename");         
        fpOutputFile =         
            ui.AddFilePicker("fpOutputFile", "...", "", "txtOutputFile", false);         
    }         
    public void LayoutControls()         
    {         
        // Form         
        ui.Width  = 500;         
        ui.Height = 150;         
        ui.Title = "Building our first UI!";         
        int margin = 12;         
        // Position dropdown and label.         
        lblSelectDatabase.Width = 100;         
        lblSelectDatabase.Left = margin;         
        lblSelectDatabase.Top = margin;         
        cboSelectDatabase.Width = ui.Width         
            - lblSelectDatabase.Width - margin * 3;         
        cboSelectDatabase.Top = lblSelectDatabase.Top;         
        cboSelectDatabase.Left = lblSelectDatabase.Left         
            + lblSelectDatabase.Width + margin ;
              // Position file picker        
        lblOutputFile.Width = lblSelectDatabase.Width;         
        lblOutputFile.Left = lblSelectDatabase.Left;         
        lblOutputFile.Top = lblSelectDatabase.Top         
            + lblSelectDatabase.Height + margin;         
        fpOutputFile.Width = 25;         
        fpOutputFile.Top = lblOutputFile.Top;         
        fpOutputFile.Left = ui.Width - fpOutputFile.Width - margin;         
        txtOutputFile.Width = ui.Width         
            - fpOutputFile.Width - lblOutputFile.Width - margin * 4;         
        txtOutputFile.Top = lblOutputFile.Top;         
        txtOutputFile.Left = cboSelectDatabase.Left;         
    } 
          public void DataBindControls()        
    {         
        cboSelectDatabase.BindData(MyMeta.Databases);         
    }         
}

Consuming the UI, Method 1: Input

You have a working UI now but what good is it? What does it actually do? By itself, it lets you specify parameters but those parameters have to be consume by the Template Code script. It does so in two ways. I’ve mentioned before that the first way is through the input object. The input object is also a global instance. Once again, see the Zeus help file under IZeusInput and ZeusInput. The input

object provides the values for the UI elements as selected by the user at runtime. It is more convenient than the second method, which is to access the elements themselves. It is more convenient because every element has a single scalar value. For example, a dropdown list has a list of elements, a selected value, a selected index, etc. The input object contains a single string for that control, the selected value.

The input object is a collection of values, each with a name corresponding to a control on the UI form (more specifically the string ID property used in the construction of said control) and a value as entered or selected in that control at runtime. Objects retrieved from this collection must be cast to specific types, usually string.

Here’s an interesting exercise. Put this code in your Template Code’s Render method. It will output all the items in the input collection. Now you’ll know exactly what’s available to you. You’ll see the elements from your UI as well as several fields defined by MyGeneration. MyGeneration input key names begin with double underscores.

          output.writeln("Input fields: ");        
  foreach(string s in input.Keys)         
  {         
   output.write(s);         
   try         
   {         
    output.write("=");         
    output.write((string)input);         
   }         
   catch { }         
    
   output.writeln("");         
  }

You’ll also see an entry for cboSelectDatabase. It is as you selected in the UI. You can now use this in your template code like this:

        IDatabase selectedDB =        
    MyMeta.Databases[(string)input["cboSelectDatabase"]];         
output.writeln("Selected database: " + selectedDB.Name);         
output.writeln(""); 
      foreach (ITable t in selectedDB.Tables)        
{         
    output.writeln("\tTable: " + t.Name);         
    foreach(IColumn c in t.Columns)         
    {         
        output.write("\t\tColumn: name=" + c.Name);         
        output.write(", dbTargetType=" + c.DbTargetType);         
        output.write(", DataType=" + c.DataType);         
        output.write(", DataTypeName=" + c.DataTypeName);         
        output.writeln("");         
    }         
    output.writeln("");         
}

That script will output basic schema for all tables in your database.

Notice that when retrieving the value of the cboSelectDatabase combo box, we did not have to drill into a SelectedValue property to get the selected value. The input object gives us a single string for cboSelectDatabase when we pass its name as a key. The string is its selected value.

Consuming the UI, Method 2: UI

The second method is discouraged. You can use the ui object to retrieve the UI elements directly and query their properties for the values you want.

In the previous example, we might have changed this line:

        IDatabase selectedDB =        
    MyMeta.Databases[(string)input["cboSelectDatabase"]];

and made it work like this:

        IDatabase selectedDB =        
    MyMeta.Databases[((GuiComboBox)ui["cboSelectDatabase"]).SelectedValue];

When I change that line, the rest of the template code still works.

I hope this has given you everything you need to get started. It certainly won’t cover everything you need to learn to achieve your particular task, but my goal was to get you started so that you would know where to look and have the tools to troubleshoot your work. If I’ve left out something fundamental, please let me know.

Phil Gilmore (www.interactiveasp.net)