code dojo

LCD Image Persistence

2006-04-25T08:24:00.000-07:00

Most of us have long forgotten about CRT burn-in that plagued early monitors. Did you know LCDs have a similiar problem? It happened last night on my high-end 20" LCD. Image persistence results from displaying a static image on your screen for an extended period of time. In my case, it was a black color scheme for Windows. The icon and title 'persisted' on my screen. I usually work in maximized mode meaning the windows title bar for any application is at the top. I'm stumped, however, why my taskbar doesn't have the same symptoms. (I autohid it to check).

Fortunately this state is reversible in most cases. I am following Apple's advice since I need to use my computer:

Avoiding image persistence on Apple LCD displays

I went beyond the white background and switched to a Milk scheme for windows. I've noticed an improvement over night. There's still a slight hint of persistence which I hope will disappear in the next couple of days.

The other recommended solution is to shut off the LCD for several days. I'm not sure why this would be more effective than say shutting it off for one day. I'll try it if Apple's recommendation doesn't remedy the problem.

XmlUnit

2006-04-15T20:15:00.000-07:00

XMLUnit is a utility class to aid unit testing of XML processing. I've used it before but gave up on it too quickly mainly because it didn't ignore whitespace. I'm inundated with XML work. XMLUnit's differencing is too strict and is not very useful for me in its current state. So I downloaded the source, made it compliant with NUnit 2, modified a few unit tests so they run successfully from TestDriven.NET and finally added a property to ignore whitespace differences.

These two XML are now considered equal:

string control = "<a color=' red'><empty/>APPLE ORANGE</a>";
string test = "<a color=\"red\"><empty>        \t\r\n</empty>          APPLE   ORANGE          </a>";
XmlAssert.AreSimiliar(control, test);

Notice the library treats these as similiar (not identical):

"<empty/>" == "<empty> <empty/>"
"APPLE ORANGE" == " APPLE ORANGE "
"red" == 'red' (use of single or double quotes for attributes)

I no longer have to worry about my xml editor reformatting my files and finding out later all my unit tests are broken. I'm considering starting a new project based on XMLUnit so I may enhance it and also rename methods to align it more closely with NUnit.

XSLT: How to autonumber all elements and attributes.

2006-04-15T20:03:00.000-07:00

I'm doing more XSLT transformations and there's a lot of mapping to be done. I want to automate as much of the testing as I can. The first thing in order is how do I easily create a control sample so I can automate all attributes and fields mapped? Another stylsheet of course.

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:fo="http://www.w3.org/1999/XSL/Format">
    
    <xsl:output indent="yes" encoding="utf-8"/>
    <xsl:template match="/">
        <xsl:apply-templates />
    </xsl:template>

    <xsl:template match="text()">
        <xsl:number level="any" format="0001" count="text()|@*"/>
    </xsl:template>
    
    <xsl:template match="@*">
        <xsl:attribute name="{name()}">
            <xsl:number level="any" format="0001" count="text()|@*"/>
        </xsl:attribute>
        
    </xsl:template>
        
    <xsl:template match="*">
        <xsl:copy>
            <xsl:apply-templates select="@*|node()"/>
        </xsl:copy>
    </xsl:template>
</xsl:stylesheet>

This simple stylesheet replaces the value of all attributes and elements with a sequential number from 0001 to n. The result looks like this.

<recipes>
    <recipe name="0001" prep_time="0002" cook_time="0003">
        <title>0004</title>
        <ingredient amount="0005" unit="0006">0007</ingredient>
        <ingredient amount="0008" unit="0009">0010</ingredient>
        <ingredient amount="0011" unit="0012" state="0013">0014</ingredient>
        <instructions>
            <step>0015</step>
            <step>0016</step>
            <step>0017</step>
        </instructions>
    </recipe>
</recipes>

This line needs a little explanation.

<xsl:number level="any" format="0001" count="text()|@*"/>

It says, create a counter for element text (text()) or attribute value (@*) encountered at any level, then format the number returning a four-digit number with leading zeros. The rest of the stylesheet is a modified identity template.

XslTransform and xsl:document() woes

2006-04-13T07:50:00.000-07:00

This little .NET feature consumed half my day (sarcasm intended). XslTransformation and document(''). .NET XSL transformations may behave unexpectedly when a stylesheet invokes the xsl:document() function. The same stylesheet results in two different outputs. If the stylesheet is loaded from a URI, such as a file, transformations produce expected results. Loading the stylesheet from a stream or a string, however, requires the workaround as described in the article above.

Why would one opt to load a stylesheet from a Stream or string? I prefer to embed test files and test stylesheets into an assembly by selecting the Embedded Resource action in Visual Studio. This is a neat trick if building with both NAnt and Visual Studio. Most NAnt projects build assemblies into a build, dist or bin directory. Visual Studio defaults to bin\debug and bin\release. An embedded resource always has the same path regardless where your assembly is located.

Loading a stylesheet from a non-URI location may reduce I/O as hinted in the article above. XSLT is ubiquitous. As sectors move towards standardization stylesheets become larger and larger. Most developers laboriously map one element to another. I treat XSLT as a cool programming language with a funky syntax. XML is structured and soon a resourceful developer realizes there's a pattern to the transformation. I tend to use lookup tables with the xsl:document() function. Resolving document('') to memory instead of a file reduces I/O.

I created a utility function to load stylesheets that use xsl:document().

XmlReader stylesheet = XmlUtils.LoadNonUriStylesheet(IOUtils.OpenEmbeddedResource("res/test.xls"));

LoadNonUriStylesheet loads the stylesheet assigning it an arbitrary URI. The URI is resolved by a custom resolver during transformation. OpenEmbeddedResource converts the res/test.xls path to a fully qualified namespace path then opens a Stream for the resource.

Test-Driven Development to the Rescue

2006-04-11T16:58:00.000-07:00

My first project at my current job was to write some business rules against XML. At first, it seemed liked a daunting task. To write and test a rule required knowledge of the backend framework, the servers, many applications and patience to push a document through the framework. There was no way I could push through 420 test scenarios doing that. I spent several days analyzing what was in place to isolate my task, refactoring existing code etc (though i did not merge it back) to create a test harness. I even fibbed to the project manager on my progess knowing that I would make up the time later. That's how much confidence I have in Test-Driven Development. At first, things were slow-going. The pressure was mounting. Once I had my test harness in place, I'm guessing not only did I do this task faster than anyone before me, I also had reproduceable tests. To top of it, I had code generators to automate SQL scripting, etc. Testing was smooth as butter.

Thinking Dependency Injection

2006-04-11T16:32:00.000-07:00

I'm creating components to convert proprietary source files into a standardized XML format. The legacy backend business components get their settings from disparate sources. Some settings are retrieved via a custom configuration object while others are read using stored procedures. Writing a unit test requires knowledge of how to retrieve those settings.

Point 1 - Components should be focused on business logic not infrastructure logic.

Here's a contrived example.

class XmlConverter {
    void Convert(int orderId)
    {
        // get settings from a database
        DbSettings dbSettings = GetDbMetadata(orderId);

        // get from config object
        string targetFile = OurConfig.GetPath("StandardStore") + dbSettings.OrderNumber + ".xml";
        
        string sourceXmlFile = dbSettings.XmlFile;
        string stylesheet = dbSettings.Stylesheet;
            
        string standardXml = XmlUtils.XslTransform(sourceXmlFile, stylesheet);
        
        using (TextWriter tw = new StreamWriter(targetFile))
        {
            tw.Write(standardXml);
        }
    }
}

The preceding code looks normal for most of us. And if you are the developer who wrote the code you know how to setup the database and you could select a valid order id so everything works when you test it. In addition, you know how the configuration object works and you can change its value blindfolded. It's so darn easy. Or is it?

Along comes poor me. I'm assigned to the project to fix an issue with the stylesheet. I hold a black-belt in stylesheet code-fu. Puh-leeze, I'll knock this task out in less than an hour. Then, why did it take me all day? Well because all that information you have in your head is too much to grasp for someone new to a project. The object knows too much. You know too much.

Point 2 - Objects should be a on a need-to-know basis.

How do we fix this with dependency injection? First, think backwards. Give the component the information it needs, instead of that component retrieving what it needs. That is the basic paradigm shift of Dependency Injection. A component is injected with the dependencies it needs. Let's rewrite this.

class XmlConverter {
    void Convert(string sourceXmlFile, string stylesheet, string targetFile)
    {
        string standardXml = XmlUtils.XslTransform(sourceXmlFile, stylesheet);
        
        using (TextWriter tw = new StreamWriter(targetFile))
        {
            tw.Write(standardXml);
        }
    }
}

Now the component is focused on what it has to do---converting a file and saving it to some known location. I can go in there and fix the stylesheet in 15 minutes and everyone is happy. What about configuration? First, let's look at the advantage of what just happened. To write a unit test, I simply pass in source XML file, a stylesheet file and the absolute path to where the transformed file is persisted. If everything works, I'm 99.9% sure my fix will close the ticket. I didn't have to know about infrastructure nor could I have affected it.

What about the settings? Someone has to retrieve it and inject it into the object. If you're the original coder, you would have already done this. If you're a maintainer, don't worry about it. You don't need to know to do your job. The infrastructure code is separate from business component logic. The application is easy to maintain.

Dependency Injection containers help in this effort by allowing you to wire objects together declaritively. For example, why build a SqlConnection object when you can ask the container for pre-built one?

SqlConnection connection = Container.GetObject("sqlconnection");

The container will cache it, or return a singleton or create a prototype when asked. Object instantiation is controlled declaritively either through a configuration file or a compiled script. The same kind of things you're doing now. Now it's done in one place and you don't have to build any infrastructure code. Use the container.

Markdown vs. Markup

2006-04-09T23:04:00.000-07:00

Today I stumbled upon Markdown.NET. I've been using my own homegrown text to HTML converter, MGutz.Markup, for a while now. I created a WinForms utility to convert text to HTML on-the-fly giving immediate feedback. I prefer the utility for the generation of plain HTML documents over DreamWeaver. I've changed the utility to support both Markdown and MGutz.Markup.

My disappointment with Markdown is it lacks markup for quick tables. MGutz.Markup supports nested tables as well as nested elements. Overall, Markdown has a cleaner syntax. MGutz.Markup's syntax tries to avoid escaping of characters used in code like '[', ']', '*', '_', '-' and '+'. As a consequence, MGutz.Markup source files are not as eye-friendly.I like Markdown and will probably use it for most day-to-day documents but I use tables a lot. I may change the grammar for MGutz.Markup since most of my code is usually eclosed in <pre></pre> blocks.

I have security concerns about Markdown. It's not suitable for use on Wiki's or CMS web sites as-is. It accepts raw HTML elements which could allow someone to insert javascript. MGutz.Markup, by default, does not allow the insertion of raw HTML elements. The escape sequence in MGutz.Markup tells the lexer to not evaluate the rule. Escaped tag sequences are still processed by HtmlEscape() . MGutz.Markup allows insertion of raw HTML tags only if a developer explicitly sets the EnableRawXml property. Likewise, links will not be generated unless EnableLinks is explicitly set.

Here's a quick comparison of the two:

html output

Markdown

MGutz.Markup

this is a para

HTML special chars: & < >

this is
  a
    para

HTML special chars: & < >

this is
  a
    para

HTML special chars: & < >

google

[google](http://www.google.com)

<<google, http://www.google.com>>

apple
orange
pear

- apple
- orange
- pear

// newline not requierd
@- apple 
@- orange 
@- pear

one
two
three

1. one
2. two
3. three

@+ one @+ two @+ three

one
- apple
two

@+ one @> @- apple @+ two

column heading

languages

right

C#, Boo

left

centered

sub-table

[[* column heading || languages ]]
[[> right || C#, Boo ]]
[[ left ||| centered [[sub-table]] ]]

bold italic code italic bold

_italic_ **bold** `code` _**italic bold**_

**bold** //italic// ::code:: //**italic bold**//

![alt text](http://buttons.blogger.com/bloggerbutton1.gif "Title")

@[http://buttons.blogger.com/bloggerbutton1.gif]

one two three

> one
> two
> three

@bq
one
two
three
bq@

Merging XML with XUpdate

2006-04-05T23:27:00.000-07:00

I'm currently working on a backend process that receives two separate XML documents from different sources. The backend process must first transform each document, using XSLT, to conform to the company's standard format. The process then merges the two transformed documents into a single document. The merging operation requires some thought as the two documents share elements. Some elements are added while others are updated.

I wanted to automate this as much possible. The easy way out is to write some C# code and merge the documents using two XmlDocuments. But surely someone must have already thought of a better approach to this problem. My thought was to transform the first document like normal but transform the second document into an update document to be applied against the first document. The second document will contain embedded instructions on how to update the first.

After some googling, I found three solutions:

Microsoft XML Diff Language v.1.0 Beta

XUpdate Working Draft

Merging two XML Documents [using XSLT] by Oliver Becker

The Microsoft XML diff approach defines an XML diffgram language to describe the operations necessary to update a target document. It's not very readable using code numbers, etc for operations.

XUpdate follows a similiar approach defining basic commands and operations to update a target document. The language is much more readable and there exists several implementations for non-.NET languages. As a bonus I found a decent .NET implementation on Xinjie ZHANG's Homepage. The working draft, however, has not been touched for many years. I updated the DTD as well as the XUpdate class to support <xupdate:document-fragment/>.

Oliver Becker's approach is pure XSL. While I consider myself pretty darn good at XSL, XSL is just too much damn work. And trying to decipher someone else's XSL stylesheet takes at least three times the effort of trying to understand say C# code.

Here's an example using the XUpdate approach:

<!-- baseline.xml: target document -->
<recipes>
    <recipe name="unsalted bread" prep_time="5 mins" cook_time="3 hours">
        <title>Basic bread</title>
        <ingredient amount="3" unit="cups">Flour</ingredient>
        <ingredient amount="0.25" unit="ounce">Yeast</ingredient>
        <ingredient amount="1.5" unit="cups" state="warm">Water</ingredient>
        <instructions>
            <step>Mix all ingredients together, and knead thoroughly.</step>
            <step>Cover with a cloth, and leave for one hour in warm room.</step>
            <step>Knead again, place in a tin, and then bake in the oven.</step>
        </instructions>
    </recipe>
</recipes>

<!-- diffgram.xml: contains an append and update command -->
<xupdate:modifications version="1.0" xmlns:xupdate="http://www.xmldb.org/xupdate">
    <!-- append the Salt ingredient -->
    <xupdate:append select="/recipes/recipe[@name='unsalted bread']">
        <xupdate:document-fragment>
            <ingredient amount="1" unit="teaspoon">Salt</ingredient>
        </xupdate:document-fragment>
    </xupdate:append>

    <!-- change the recipe name to 'Salted Bread' -->
    <xupdate:update select="/recipes/recipe[@name='unsalted bread']/@name">Salted Bread</xupdate:update>
</xupdate:modifications>

// unit test
XmlDocument baseline = new XmlDocument();
baseline.Load(IOUtils.OpenEmbeddedResource("Xml/res/baseline.xml"));

XmlDocument diffgram = new XmlDocument();
diffgram.Load(IOUtils.OpenEmbeddedResource("Xml/res/diffgram.xml"));

XUpdate update = new XUpdate(baseline, diffgram);
update.Execute();

<!-- result of XUpdate.Execute() -->
<recipes>
    <recipe name="Salted Bread" prep_time="5 mins" cook_time="3 hours">
        <title>Basic bread</title>
        <ingredient amount="3" unit="cups">Flour</ingredient>
        <ingredient amount="0.25" unit="ounce">Yeast</ingredient>
        <ingredient amount="1.5" unit="cups" state="warm">Water</ingredient>
        <instructions>
            <step>Mix all ingredients together, and knead thoroughly.</step>
            <step>Cover with a cloth, and leave for one hour in warm room.</step>
            <step>Knead again, place in a tin, and then bake in the oven.</step>
        </instructions>
        <ingredient amount="1" unit="teaspoon">Salt</ingredient>
    </recipe>
</recipes>

MSDN: Stored Procedure Object Interface Layer

2006-03-29T11:03:00.000-08:00

MSDN has an article on a stored procedure data layer using attributes.

Stored Procedure Object Interface Layer

As usual, the open source community has been doing this sort of thing for a while—iBATIS. It can handle both stored procedures and straight SQL. iBATIS is much more flexible. There are some codegen tools that will do the mappings for you.

iBATIS

I'm Back

2006-02-13T10:05:00.000-08:00

Geez, it's been a while. I moved back to sunny San Diego and started a new contract today. Im familiar with the area as I went to college here and lived here for many years. It really is the finest city in the US. You can't beat it for weather and there's everything around.

I'm working on .NET stuff as usual. The company I work for is using all three versions of Visual Studio---2002, 2003 and 2005 which means I'm restricted to using C# 1.0 features. FUN FUN FUN. This is common in the real world. While it's neat to read and play with the latest software technologies, most companies are content using an older version against which legacy applications are built. There is a huge amount of resources deploying, regression testing enterprise applications. Definitely no generics, etc.

Is Boo dead? I dont see much activity.

Object

2005-12-06T13:02:00.000-08:00

It's good practice to override Equals() and GetHashCode() in your classes. Well, that's what they say anyway, I rarely do. But, with generics and org.apache.commons.lang.builder.HashCodeBuilder (one of my java ports) it's easy to do. I created a generic base Object class:

public class Object<T> {
    public override int GetHashCode() {
        return HashCodeBuilder.reflectionHashCode(this);
    }

    public bool Equals(T obj) {
        if (obj == null) {
            return false;
        }
        return obj.GetHashCode() == GetHashCode();
    }

    public override bool Equals(object obj) {
        if (obj == null) {
            return false;
        }

        if (obj is T) {
            return GetHashCode() == obj.GetHashCode();
        }
        else {
            return false;
        }
    }
}

HashCodeBuilder.reflectionHashCode(this) introspects all the fields of the class and creates a HashCode. To create a class with this functionality, derive from Object<T>:

public class Book : Object<Book> {
    string author = null;
    public string Title = null;
    public string Author {
        get { return author; }
        set { author = value;}
    }
}

This trick is great for unit testing where you need to compare the inner value of classes. Unfortunately you can seldom use this trick in C#, a single-inheritance language, unless you're building a framework and you derive from a single base. For performance, use HashCodeBuilder's append functions instead of reflection.

I Miss Java

2005-12-06T00:37:00.000-08:00

While I have a lot more experience with C# and .NET, I have good experience with Java. I like both C# and Java. I especially like Java for open-source. There's so many well-designed open-source Java projects. Whatever open source projects there are for .NET, there's bound to be a multitude of projects that are stable and more mature on the Java side. And there are many Java projects that simply do not have a counterpart on the .NET side. One library I really miss is Jakarta Commons, a comprehensive set of utility classes for your day to day coding. I'm porting functions/classes as I need them. Fortunatley, the MS Java Conversion assistant is helping in this effort, doing most of the work.

Switching to LGPL

2005-12-04T23:51:00.000-08:00

After reading the history of AJAX.NET it really bothers me that open-source on the .NET is greedy. Many open-source .NET projects that have become successful have turned commercial leaving behind many developers who contributed, tested and offered ideas while it was open source. What a shame. There are many projects that are close-sourced and proclaim to be free for .NET. You're a SUCKER if you're contributing your time to one of these projects.

I'm switching all my open-source projects to LGPL, GNU Lesser General Public License. This means you can use the library as-is in a commercial application without any issues and give credit in a credits.txt file. You can't create a derivative of it by changing the source, recompiling and using it commercially without my permission. If you fix bugs or enhance it, then send a diff of those modifications and you are granted automatic permission to use the libary commercially again. I don't care about your proprietary code, only changes/enhancements to the library. I think this is more than fair. I'm not sure if LGPL is the right license for all I said, but it's a good start.

JSON Serializer/Deserialier

2005-12-04T23:15:00.000-08:00

I have a working JSON (JavaScript Object Notation) serializer/deserializer for .NET. I can marshall complex types between the browser and server using JSON. This is for my AJAX Behaviour.NET library but like all my stuff you're free to use under the terms of the license. This was not an easy task. JSON is typeless which makes it very difficult to deserialize into strongly typed language like C#. I resorted to encoding the type of an object or array (collection) into the document itself:

{
    "__objtype__" : "Test.Amelgy.Ajs.Json.Person, Test.Amelgy.Ajs",
    "Name" : null,
    "Sex" : "\u0000",
    "__arrtype__Pets" : "System.String[], mscorlib",
    "Pets" : [
        "sam-sam",
        "chili"
    ],
    "__arrtype__Books" : "System.Collections.ArrayList, mscorlib",
    "Books" : [
        {
            "__objtype__" : "Test.Amelgy.Ajs.Json.Book, Test.Amelgy.Ajs",
            "Author" : "Tom Archer",
            "Title" : "Inside C#"
        },
        {
            "__objtype__" : "Test.Amelgy.Ajs.Json.Book, Test.Amelgy.Ajs",
            "Author" : "GOF",
            "Title" : "Design Patterns"
        }
    ],
    "Weight" : 160
}

This has potential for a security risk, but I'll give the end developer an option to encrypt the type and assembly. Deserializing is a matter of building C# code and using the CSharpCodeDom to compile it. Here's a sample of C# code generated by the deserializer. Still very readable, good for debugging.

namespace nsc2fc87b2d1d5450a8080dd6f0168da50 { class Foo { public object GetObject() {
Test.MGutz.Json.Person object0 = new Test.MGutz.Json.Person();
object0.Name = null;
object0.Sex = "\u0000";
System.String[] array1 = new System.String[2];
object0.Pets = array1;
array1[0] = "sam-sam";
array1[1] = "chili";
System.Collections.ArrayList array2 = new System.Collections.ArrayList();
object0.Books = array2;
Test.MGutz.Json.Book object3 = new Test.MGutz.Json.Book();
array2.Add(object3);
object3.Author = "Tom Archer";
object3.Title = "Inside C#";
Test.MGutz.Json.Book object4 = new Test.MGutz.Json.Book();
array2.Add(object4);
object4.Author = "GOF";
object4.Title = "Design Patterns";
object0.Weight = 160;
return object0; }}}

The only remaining issue is nested anonymous arrays. That'll be ugly cuz I think I'm too far in the parser and I'm using my JsonTextWriter which is not random access. A slight challenge is all.

{
    "__arrtype__nested" : "ArrayList, mscorlib : int[], mscorlib"
    "nested" = [
        [1, 2, 3]
    ]
}

ASP.NET aaargh!

2005-12-04T12:40:00.000-08:00

I'm deciding how to expose server-side methods to AJAX clients of non-Gamba developers. I want Behaviour.NET to be usable outside of Gamba. AJAX is easy for Gamba or any MVC2 framework for that matter. A controller can expose any combination of views or methods. In Gamba's case, a server-method's logic is as simple as changing the response content type to Content.TextPlain, Content.TextXml or the default Context.TextHtml then return data directly bypassing the usual logic of rendering views.

In ASP.NET you don't get a controller, you get a stupid page. I have some options. I can derive from the ASP.NET Page to create a BehaviourPage. Developers then change their code behind to inherit from BehaviourPage. This lets me create a page controller for pages taking advantage of Behaviour.NET. Remote call would need be encoded in the query string of that page. But, what if a shop already has custom page in place? This folks is one of the many reasons why the ASP.NET page framework sucks!

The other option is to commandeer the .ashx extension and create an MVC framework strictly for AJAX methods like web services does with .asmx. I could use a different extension but .ashx is supported natively by ASP.NET. Using a different extension would prohibit its use in a hosted environment where you can't change IIS mappings. Even worse, .ashx may already be in use.

Yet another option is to hook into the lifecycle events. Seems too manual for me, but it maybe the only option I have.

If anyone has any other suggestion please speak up. It's why I think aloud on my blog. It's a think-tank.

I've told some people interested in this library it would take about a week to create a standalone version of Behaviour.NET. I forgot about the pains of ASP.NET. I'll get it out guys. I want to make it simple for you, so don't shoot me if it's not out in a week.

Behaviour.NET - Real Open Source AJAX for .NET

2005-12-03T21:11:00.000-08:00

Real open source meaning Apache License 2.0 instead of the mess with AJAX.NET.

So I thought I had discovered a better way to do AJAX. Well, it looks like the MS Atlas folks and I think alike. I haven't really looked Atlas, and today I saw a screenshot of it with the word behaviour. What's funny is behaviours are the foundation of Behaviour.NET, a standalone AJAX library for my Gamba framework. I'd have to look at some samples to verify. What is that old adage? "Great minds think alike." I won't go that far, but this seems a better approach than what I've seen.

If you remember my previous article Helpers or Components? I could not decide whether to create an AJAX API using helper functions or a lightweight component framework. After some contemplation it's my humble opinion helper functions are evil. We're back to ASP spaghetti again. Let's compare the two approaches, here's the helper function style:

class Test(AjaxController):
    def Foo():
        ViewMethod = FooView

    def SayHello() as string:
        ContentType = Content.TextPlain
        return "Hello world!"

<!-- Test.FooView.tpl -->
<%= CreateButtonDoRemote("Invoke Remote", Urlize("SayHello")
        , {"onsuccess" : "$('input1').value = request.responseText;"
        , "onfailure" : "alert('Unable to get information from server.')"}
    )
%> &nbsp; <input id="input1"/>

It's typical ASP coding. Here's how it's done with Behaviour.NET, with behaviours code residing in the controller.

class Test(AjaxController):
    def Foo():
        button = Button("button1")
        with method = RemoteMethod(Test, "SayHello"):
            OnSuccess += Prototype.SetValue("input1")
            OnFailure += Prototype.MsgBox("Unable to get information from server.")
        button.OnClick += Prototype.InvokeRemote(method)

        Behaviours.AttachBehaviour(button)
        ViewMethod = FooView

    def SayHello():
        ContentType = Content.TextPlain
        return "Hello world!"

<!-- Test.FooView.tpl -->
<input id='button1' type='button' value="Invoke Remote"/> &nbsp; <input id="input1"/>

It's a little bit more coding, but here are some advantages off the top of my head

It's readable and easy to understand.
Errors are minimized. Event names are checked at run-time
Separation of code and view (developers vs. designers).
Developer's are shielded from the underlying AJAX engine.
The generated javascript is as lean and syntax correct the first time.

One disadvantage is a slight performance loss, but that's minimal. I've looked at some implementations of AJAX generic helper functions and they use a maze of functions to generate a script. These objects are very light and for now all they do is encapsulate some javascript code. This lightweight component framework coincidentally fits in very well with Gagamba, my plans for a fully componentized version of Gamba. I wanted to do a basic core framework first before starting Gagamba.

Oh, I almost forgot. Thanks to Ben Nolan, the author of the Behaviour JavaScript library for inspiration.

Gamba: A New Beginning

2005-12-02T14:34:00.000-08:00

Hmm, maybe Boo is a language for three developers (quote from somewhere). Noone seems to be interested in a web framework for Boo. Eventually I'd like to enlist people to contribute but I dont see much interest. That's a little disconcerting, but time to move on. Maybe I'll contribute to Boo in other ways, but for now I'll concentrate on C#. Boo's architect, bamboo, has some good ideas with Boo but I don't think people realize how things like AstAttributes let you be more expressive and creative. Code injection, for example, is very powerful. You can get close with an MSBuild preprocessor and partial classes with C# 2.0 but you don't get the AST. I've only touched the surface, you could change the behaviour of a class instead of just mixing in other behaviour. Anyway, it's all for not.

If you're interested in Boo for web stuff you still have Brail and MonoRail.

I still have plans for Gamba, which as of today starts anew with focus on ASPX pages, MS Enterprise Blocks and C#. My real experience lies in enteprise development and so it's my comfort zone. Sadly, I can only do a pseudo-MVC2 approach but it fits in well with how most Microsoft developers do web development.

Gamba: Helpers or Components?

2005-12-02T10:45:00.000-08:00

I can't decide how Gamba should present AJAX to the end developer. After analyzing the prototype javascript library, which is the foundation for most AJAX functionality these days, should Gamba provide helper functions or create a component model? AJAX lends itself to the component model. You're forced to deal with synchronous/ asynchronous events. Events fired by one element may result in the update of another element. For example, clicking a button calls a remote function updating an input box.

Gamba can follow MonoRail and provide helper functions. This requires the developer to learn the prototype javascript library. Gamba can shield the end developer by creating a bunch of helper functions as MonoRail does. Here's an example:

<%= CreateButtonDoRemote("Invoke Remote", Urlize("SayHello")
        , {"onsuccess" : "$('input1').value = request.responseText;"
        , "onfailure" : "alert('Unable to get information from server.')"}
    )
%> &nbsp; <input id="input1"/>

It's just not simple, which goes against the KISS principle. The other option is to create a lightweight component model with pseudo-events. Here's the equivalent behaviour as above:

<%
text = TextBox('input1')
with button = Button(Caption: 'Invoke Remote'):
    OnClick += Client.CallRemote(Urlize('SayHello'))
    OnSuccess += Client.SetValue(text)
    OnFailure += Client.MsgBox('Unable to get information from server.')
%>

${button} &nbsp; ${text}

SVN access for Reflex and CodeInjection

2005-11-30T23:11:00.000-08:00

I've moved the source for the CodeInjection and Reflex library to ckknight's SVN box. NovellForge was too much of a pain with passwords. I'll import the rest of the Gamba framework to there once I feel the core interfaces and implementations won't change. The libraries above may be used separately from Gamba and are licensed under APL 2.0 .

Here's the address: svn://ckknight.no-ip.org/mgutz/trunk

There isn't much documentation, well because I've been lax and Boo doesn't have a formal documentation tags syntax. Your best bets are the articles I've written in this blog and the examples or unit tests.

Gamba: First-Class Branding

2005-11-30T22:48:00.000-08:00

More refactoring today. I've promoted the branding idea to a full pledged object model. The entry point (after front controller) into Gamba will not be a controller as in traditional MVC2, rather a brand. The importance of this is a brand controls access to a set of controllers and also encapsulates look-and-feel. For example, an executive brand of a corporate website allows the use of the FinancialsController and CorporateStrategiesController as well as any other controllers it inherits through derivation.

If developers do not wish to use brands, then the DefaultBrand will simply be a pass-through to a controller as in normal MVC2. Here's a simplified UML diagram. (There's actually a few classes between dispatchers and brands like IActionProxy).

Gamba: Purpose

2005-11-30T22:45:00.000-08:00

I should have formally announced what Gamba is. Gamba is a web framework built on a well understood design pattern, MVC2. MVC2, unlike ASP.NET which is a simple page controller model, has a front controller directing actions to sub controllers. The duty of the front controller is to further separate the concerns of what a page has to do from what an infrastructure should provide. Every page should not be concerned with user authentication or how to handle exceptions. In the ASP.NET model where users are accessing the page directly, you're forced to handle it on the page or by inheritance. Ideally a page should only be concerned about rendering data its given, nothing more. The front controller also insulates your users from changes in the backend since URLs are logical paths not physical paths as in ASP.NET. I would highly recommend googling front controller or MVC2 to get a better understanding of this pattern.

My goal for Gamba is to create a web framework suitable for consulting work. Gamba has first class infrastructure support for things like branding. The HTML templates will be editable in design tools like DreamWeaver. The data templates will use normal SQL queries or stored procedures making it easier to integrate with legacy data. Gamba is based on Boo because I believe and have shown it to be more flexible than any other .NET language. In the end, I'll show how you can build a blog in less time than Ruby on Rails and have a fully compiled application with no templates. It will be more scalable and faster.

KISS and the 80/20

2005-11-30T03:30:00.000-08:00

I design with two principles in mind, KISS (keep it simple stupid) and the 80/20 % rule. Ckknight asked a question today in IRC about the data layer using hardcoded SQL. What if you want to switch databases? My reply was something to the effect "if you're not using generic SQL then you'll have to change templates. There's no way to get around it." The Gamba data layer implements the basic DAO design pattern, using SQL queries is by design and may be changed without affecting the rest of the application.

In all my years of development, I've only had to change the backend database once on a production application! Thankfully stored procedures were used and the DBAs did most of the work. Our DAOs worked pretty much as is, except SQL Server returns a dataset whereas Oracle returns a cursor. Most companies simply do not switch out databases because one is better. If something works, maintain it.

On a sidenote, what a mistake switching from MS/SQL Server to Sun/J2EE/Oracle was. No big gains at all. The new CIO had a thing against Microsoft. Funny how we spent literally a million dollars on servers and J2EE/Oracle licenses. Guess which applications are still handling millions of transactions? VB6 MTS components running in a web farm on cheap $5000 Windows servers compared to $30000 Sun boxes. The other is the C# application I architected. This was the application we switched to Oracle to supposedly gain performance. In Oracle's defense, we had to use Microsoft's Oracle drivers. At that time Oracle's driver was still in beta. The moral is no fancy J2EE distributed gumbo. Each core application relied on MSMQ with a 20 or so server farm hiding behind a round-robin TCP-IP dispatcher. KISS at the enterprise level.

Back on topic. Gamba will be flexible, but not to the point where it becomes over-architected, over-engineered, over-OOPsed as I like to say. It will be simple and 80% of what AJAX web developers need will be very easy to do. The other 20% will require the usual manual effort by the developer. What you get is a fast, light framework without the bloat. Features will be added as needed not what I think the future will need. Who knows, AJAX may be gone in a few years when the next big thing hits.

Gamba: Data

2005-11-29T22:27:00.000-08:00

As I thought more about a data access layer for Gamba, my ever flickering idea light came on and the solution was right in front of me. One of the goals of Gamba is to use Boo for everything, configuration, templates and so on. Why not use the CodeInjection library to simplify data access?

No matter which data access library is used, the library has to be told via a metadata file what data is queried and how to map those results to objects. (The exception being code generators which may need as little info as a connection string.) With the magic of CodeInjection metadata files can be transformed into managed code. And thus the genesis of Boo Data Templates.

Some advantages:

Fields/Properties are checked at compile-time
No performance hit for dynamic run-time mapping to classes
Supports both plain and dynamic SQL queries as well as stored procs
Code is cleaner. The code is not marked with a ton of attributes.
Classes are not overwritten by a code generator.
Less XML configuration!

Disadvantages:

No runtime query language which most ORMs provide. It's nice to be able to query a result set.
You have to write SQL queries. Most ORMs shield you from this.

Boo Data Templates

Here's the bird's eye view of Boo Data Templates:

Each template contains a query and possibly a map definition if the result of the query is to be mapped to properties of the class. The templates are injected into Person via the MixinDataTemplates attribute. As an example, the Person.SelectAll.bdt template contains this:

query:
    Select = "select person_name, person_gender from person"

map:
    Name = person_name
    Gender = person_gender

The class which uses the template looks like this:

[MixinDataTemplates]
class Person:

    [property(Name)]
    _name = ""

    [property(Gender)]
    _gender = "M"

After compilation, the class looks like this:

class Person:
    [property(Name)]
    _name = ""

    [property(Gender)]
    _gender = "M"

    def SelectAll() as (Person):
        people = ExecuteQuery(queryFromTemplate)

        for p in people:
            person = Person()
            person.Name = people['person_name']
            person.Gender = people['person_gender']

    def Find(key) as Person:
        ...

    def Update():
        ...

Next Steps

Cameron Kenneth Knight (ckknight) is taking this idea forward. This will be a very nice data layer. More details will come as the library progresses.

Just Data Please

2005-11-28T23:06:00.000-08:00

Why is it so difficult to do generic data access with .NET? Why not follow JDBC? Universal data access, remember that MS technology, not in .NET. Abstract as they might, ADO.NET still sucks. Is it that hard for MS to swallow their pride and implement a JDBC model? Or, is being able to create a SQL Server specific driver the reason ADO.NET sucks? MS knows how to do it they refuse to in ADO.NET. Every other data access library from Perl, Python, Ruby ... Java does it right.

I need a nice Data Access library for Gamba. I like iBATIS.NET mapper but it's doing more than I want. iBATIS has a lower level dData access library which may suit my needs. My ideal data access would be the MS DAAB (Data Access Application Block) but a truly generic version without needing any of the other MS blocks like configuration, logging. I don't find ORMS to be appealing as they assume 1) you have control of the database, 2) I prefer metadata and 3) stored procedures are good.

Gamba: Branding

2005-11-28T17:16:00.000-08:00

Gamba, the Boo MVC2/Rails framework is coming along nicely. As I add features to Gamba I'll write a short article explaining the usefulness of the feature. This article focuses on branding, the ability for a multi-user website to appear and behave differently based on the user.

Motivation

A very simple type of branding is achieved by most blog hosts. They allow users to specify a different template for their personal blog. For enterprise websites branding requires more than a new 'skin'. Sub applications of an enterprise-level web application may access a different database or perform specific business logic tailored for the client.

Example

In this example, we'll create a simple web app which changes behavior based on the brand's URL. Since Gamba is an MVC2 framework, pages are not directly accessible as they are in ASP.NET. Don't let this dissuade you. Creating a view is as easy as deriving from DefaultController and a template for each view.

// Foo.boo - controller
[MixinTemplates]
class Foo(DefaultController):
    def Hello():
        ViewMethod = HelloView

    def Name():
        Model["name"] = "mario"
        ViewMethod = NameView

// Hello.HelloView.tpl - template for HelloView
<p>Hello world!</p>

// Hello.NameView.tpl - template for NameView
<p>Hello ${Model['name']}</p>

Gamba's IController has many views whereas an ASP.NET page has one view, the .aspx page itself. Gamba may choose not to call any views of a controller if, for example, the user has not logged in. To view the views, compile the class above and drop the assembly into Gamba's bin folder, then browse:

Let's do something different for any user of the boo sub application. A sub application is accessed via a different URL prefix - http:://localhost:8080/gamba/boo. Gamba will automatically reuse the controller above if we don't do any modifications. Let's create a new layout and also create a controller for the boo sub application. The layout is basic, we'll just change the background to green. I don't use a template here just to show you don't have to use templates at all:

class BooLayout(DefaultLayout):
    override def Execute() as List:
        htmlTop = "<html><head></head><body bgcolor='green'>"
        htmlBottom = "</body></html>"

        return [htmlTop, _child.ViewMethod(), htmlBottom]

Let's assume Foo has a lot of functionality in it and we want to modify one view. Using OOP derive a controller from Foo.

[MixinTemplates]
class Bar(Foo):
    override def Name():
        Model['names'] = ['mgutz', 'bamboo', 'ckknight']
        ViewMethod = NamesView

# Bar.NamesView.tpl - template for NamesView
<% for name in Model['names']: %>
<p>Hello ${name}</p>

<% end %>

Again, compile the classes and drop into Gamba's bin folder. Browse the two pages noting the use of 'boo/bar':

Conclusion

This article described how Gamba solves one of the challenges facing web developers, branding. In particular, Gamba uses object-oriented programming to alter behaviour and appearance unlike some other frameworks which resort to using a template hierarchy.

BTW, as someone commented, I use FireFox. I used IE so I could capture the address bar in a small window.

Delaying Code Execution with Reflex

2005-11-25T03:10:00.000-08:00

I've already found a nice benefit of using my Reflex library. I can delay the execution of an entire block of code. Here's the scenario. A user browses this address:

http://www.mgutz.org/gamba/person/show?id=1

To Gamba (Boo web framework) it means invoke Person.Show(). When the URL above is received, Gamba forwards the URL to ReflexDispatcher for dispatching. ReflexDispatcher returns a ReflexActionProxy object which may be executed at any time. Delaying execution of the action makes Gamba flexible. The action could be chained with pre and post interceptors, it could be queued, or completely discarded by a filter in the pipeline. Here' the pseudo-logic in ReflexDispatcher.

    def Execute():
        klass = ParseClass(_context)
        method = ParseMethod(_context)

        script = """
            ${klass}.new() {
                Context = $0
                ${method}()
                return this
            }
        """

        return ReflexActionProxy(script, _context)

How does it work? First remember Gamba has to parse the class and method from URL. Gamba creates a Reflex script to avoid doing any reflection manually. The script is then packaged into an action proxy along with argument $0 (_context object). The script above, after string interpolation looks like this:

    Person.new() {
        Context = $0
        Show()
        return this
    }

Create Person() controller
Assign the HttpContext
Invoke Show()
return the instanstiated Controller

To do all of this, ReflexActionProxy simply evaluates the script.

    return Reflex.Eval(script, context)

You may be wondering is Reflex using CodeDom? DEFINITELY NOT! Reflex is optimized for small, reflection-based scripts. Reflex caches types, methods and properties to minimize performance loss.

VS2005 Tip - How to Rename a Class (not so obvious)

2005-11-21T01:22:00.000-08:00

If you rename a class by refactoring, VS2005 does not prompt to rename the file to match the new class name. It's good practice for the file to be named after the class.

Here's the trick. Rename the file in Solution Explorer first and VS2005 will ask to rename the class and update all references.

XmlDocument for AST?

2005-11-20T11:36:00.000-08:00

I played with Coco/R (lexer/parser generator) last week and was a little disappointed Coco doesn't have a quick way to build an AST like ANTLR. For small utilities, the built-in AST library of ANTLR is sufficient. ANTLR's parser grammar uses a '^' character to indicate the root of the AST node for the production rule. Very simple. To do the same with Coco, you add attributes to each element that becomes part of an AST in your production rules. The grammar file quickly becomes a hybrid of EBNF fules interlaced with C# code. To be fair, ANTLR requires interlacing of C# when creating a grammar of any complexity, but the amount of C# code is much less.

On the other hand, Coco is lightweight and doesn't require a run-time library. The reason for trying it in the first place. Coco is supposedly faster. Parsing does seem faster but I haven't built a complex parser with it.

I went back to ANTLR. Then, this little programmer demon in my head would not succumb to such an easy defeat. I'm now back on Coco for this project. Yep, i'm a flip-flopper. Since .NET doesn't provide a tree collection, I decided to use the XmlDocument for my AST instead of spending a few minutes to create an AST Node. This has many advantages:

Quick hierarchical representation of the tree. Console.WriteLine(tree.InnerXml);
Employ XPath to search the tree.
Employ XSL to convert the tree into another tree.
Create AST externally using your favorite XML editor. (for experimenting with optimizations, etc)

I'm not a compiler guy but I see this approach having many advantages (and disadvantages such as performance). Imagine being able to convert from one language to another using XSLT. Or, to simplify the use of AstMacros or NRefactory refactoring? Most of us know how to manipulate XmlDocument and even write XSLT. There is huge potential here.

The idea came from an article I read on the web. Something to do with running multiple languages using common XML ASTs on an XML virtual machine. Interesting read, I wish I saved the link.

Why not MonoRail?

2005-11-18T21:04:00.000-08:00

I almost have the core libraries to start a Boo web framework. Someone in the IRC asked why not build on MonoRail or use Brail? The main reason is the use of NVelocity as the official view engine. Brail emulates NVelocity. They completely missed the point of Ruby on Rails.

NVelocity is a full blown language. Why learn another language? NVelocity isn't object-oriented so you have to do something like this:

<head>
$AjaxHelper2.GetJavascriptFunctions()
$Effects2Helper.GetJavascriptFunctions()
</head>

<body>
Auto completion:<br>
$AjaxHelper2.InputTextWithAutoCompletion( "name", "nameautocompletion.rails", null, null )
</body>

MonoRail pushes two objects into the NVelocity context. These objects provide functions to aid in the creation of AJAX controls. I would prefer to have a controller derived from AjaxController that adds the javascript functions to the header as well as provide additional functions.

NVelocity's purpose is to be a template engine. It interprets a template at runtime to render a view. It might be worth incurring a performance hit if NVelocity provided some advantage. I've shown in a previous article how Boo can be used as a template compiler. In short, the article shows how Boo's compiler attributes let you transform a template into managed code.

How would I approach all of this using Boo? First, using OOP I would derive AjaxController from DefaultController:

class AjaxController(DefaultController):
    // add javascript functions to html
    private def AddJavascriptFunctions():
        pass

    // create an AJAX input text control with auto copmletion
    def InputTextWithAutoCompletion():
        pass

To create an actual controller:

[Template('person.tpl', 'RenderFind')]
class Person(AjaxController):
    def Find():
        RenderFind()

    def Auto():
        // logic to return matches for autocompletion

<!-- person.tpl, compiled as RenderPerson() -->
Auto completion:<br>
$InputTextWithAutoCompletion("name", "person/auto", $Model.Name)

The Template compiler attribute transforms person.tpl into a RenderFind() method. You can add multiple Template attributes. To render a person, call Person.Find(). There is no need to send variables to a template. The template is part of the class and has access to any property, field or method of the class. We're not setting up some context and passing it to an engine like NVelocity does. Person is derived from AjaxController giving this class AJAX functionality by inheritance. As an added bonus, you can debug through the generated code of the template. (Boo does not yet have a #line directive. When Boo does, you'll be able to debug the template directly.)

Are templates the way to go? It's an outdated idea for web frameworks. I may go down the road similiar to XAML, instead I'll use Glade markup. Imagine being able to draw your GUI in Glade and having it run as a web app?

If you want to create an innovative web framework, you have to start with an innovative language and take advantage of its features. That's what ROR did. I think the same can be done using Boo.

Mint-Green Tea Theme for Blogger

2005-11-17T11:14:00.000-08:00

I became bored of not doing any web stuff so I created a mint-green tea theme for Blogger. Actually, I modified the old orange theme I was using. I'm not that good of a designer but I have a utility called ColorImpact (Windows). It lets you pick a main color then based on different formulas (complements, triad, monochrome, etc) ColorImpact presents a palette of harmonius or complementary colors. I simply used the colors it gave me. I'm a little concerned about not using web safe colors so please let me know if the site looks funky!

I cleaned up the blogger template to maximize use of space, to focus on content. Nobody likes looking at dead space or scrolling just to read an entry. Most developer's use a theme good for general thoughts but not very good for describing and presenting source examples. I even indented the post bodies ala Boo. I don't know if that's functional but it says this site is about coding. I also de-empasized dates and static links on the sidebar. I removed extraneous information like "posted by Mario Gutierrez". Yeah, it's my blog who else posted it? The only thing left to do is to float the links sidebar and have a minimize button to shrink it. I could not make it work in both IE and FireFox. I used to be a die-hard IE fan, not anymore. Get Firefox!

It wasn't just a color update! If the layout is distracting, seems crammed or if you have any other suggestions, please leave a comment. I keep these things in mind when building web pages.

SharpDevelop, NRefactory Doing it Right

2005-11-16T10:23:00.000-08:00

If you're like me you probably didn't like SharpDevelop 1.1 too much. It was prone to crash. Since I've been using Boo lately, I gave SharpDevelop 2.0 a look. I build it manually from SVN. What have these guys been drinking? I want some. It's a drastic improvement over the previous version. You still encounter errors here and there but mostly for unimplemented features. It's not ready for prime time as it's a work-in-progress but it's very usable.

Daniel Grunwald is a machine. In addition to being a mad coder on SharpDevelop team, he's created tutorial videos on how to create add-ins and NRefactory. Check out this video on NRefactory. Wow! It's only a matter of time before SharpDevelop 2.0 rivals Eclipse in refactoring. Personally, I'd rather have Eclipse-like refactoring and integrated unit-testing than the new stuff in VS2005. I've also heard good comments from the Boo masters the add-in architecture is well thought out. Maybe SharpDevelop will start a trend like Eclipse did for Java. BIG KUDOS to the SharpDevelop team.

Reflex Syntax Change

2005-11-15T23:56:00.000-08:00

After some tinkering with Reflex's grammar, I decided on this (very Groovy-like):

01: #ref "../../MGutz.Test.dll"
02:
03: new Person() {
04:     Name = 'mario'
05:     Weight = 160
06:     Hobbies = new string['scuba', 'biking']
07:     ID = GetUserID(Name)
08:     Vehicle = new Car('Infiniti', 'G35') {
09:         Color = 'Silver'
10:         Horses = 260
11:     }
12:     return this
13: }

In English, just in case it doesn't make sense:

Load "../../MGutz.Test.dll"
Create a new Person object using default constructor.
Set Name, Weight and Hobbies property
Create a new string array having two values ('scuba' and 'biking') then assign to instance.Hobbies.
Call a method to get information from the database.
Create a new Car object of make 'Infiniti', model 'G35' through the constructor, and assign to vehicle.
Set car's Color, Horses property
Return the root object (if return is omitted the root object is returned). return is used to return the value of a method or property.

Again, Reflex is not meant to be a full blown scripting language. It's a quick, easy way to create objects and invoke methods and properties using reflection with simple syntax. I plan to use it to map database tables and stored procedure results to objects and to map URLs to classes and methods. And, anywhere else I need to do reflection. (A Boo Web 2.0 Framework is coming).

// example in Boo

name = "mario"
weight = "160"
hobbies = "'scuba', 'biking', 'rugby'"

snippet = """
    #ref '../../MGutz.Test.dll'
    new Person() {
        Name = '${name}'
        Weight = ${weight}
        Hobbies = new string[${hobbies}]
    } 
"""

p = Reflex.Eval(snippet) as Person

// example in C#

string name = "mario";
string weight = "160";
string hobbies = "'scuba', 'biking', 'rugby'";

string template = @"
    #ref '../../MGutz.Test.dll'
    new Person() {
        Name = '{0}'
        Weight = {1}
        Hobbies = new string[{2}]
    }
";

string snippet = string.Format(template, name, weight, hobbies);
Person p = Reflex.Eval(snippet) as Person;

Reflex is white space agnostic (except for #ref pragma). This is equivalent:

#ref '../../MGutz.Test.dll'
new Person() { Name = '{0}' Weight = '{1}' Hobbies = new string[{2}] }

VS 2005 Annoyance

2005-11-15T22:26:00.000-08:00

Here's an annoying bug. Look where I'm setting the breakpoint with the mouse. VS2005 isn't even close. This is an intermittent issue. I've always been fond of MS products but i'm not at all impressed with VS2005. VS2003 + Resharper was a much better product. Maybe Resharper spoiled me.

Reflex Reflection Library

2005-11-14T14:29:00.000-08:00

Introduction

Reflex, short for reflection expressions, is a library in the works to simplify the use of the .NET reflection library. Simply put a consutrctor, method and property class in a string and let Reflex do the work:

// instantiate Math using default constructor, then invoke Add(1,2)
string expression = "Math()::Add(1,2)";
int result = (int) Reflex.Eval(expression);

Without Reflex you would have to get the type, find the constructor, invoke the constructor, cast the the returned type, get the method, build the parameters, then invoke it. What if you already have an instance?

Math m = new Math();
rx = new Reflex(m);
expression = "Add(1,2)";
int result = (int) rx.Invoke(expression);

Expressions

`[../../some.dll] SomeClass()::Method()`	Loads `some.dll`, instantiate `SomeClass` using default constructor, invoke `Method()`
`SomeClass::Method()`	Invoke static method `Method()`
`Languages["Boo"].Url`	Get the `"Boo"` entry from dictionary `Languages`, then get the `Url`property from the object.
`Languages.Boo.Url`	Same as above, dictionary entries are treated as properties.
`Languages[0].Url`	Get the first entry from dictionary `Languages`, then get the `Url`property

Require Attribute

In some scenarios, Reflex should be prohibited from blindly invoking methods. For example, mapping a URL to classes and methods is a huge security hole. Create an attribute to mark the methods you wish to expose, then use the overloaded Eval() or Invoke():

// throws an exception, Fibonacci is not marked with [Expose]
Reflex.Eval("Math::Fibonacci(3)", typeof(Expose));

class Math {
 [Expose]
 public int Add(int a, int b) {
  return a + b;
 }

 public int Fibonacci(int n) {
 ...
 }
}

Usage

Map a data row to a class.

string template = @"
My.Person()::
    Name = ""{0}""
    Age = {1}
    Height = {2}
";
expression = string.Format(template, row.Name, row.Age, row.Height);
Person p = (Person) Reflex.Eval(expression);

Map url to execute method.

string queryString = "Math/Add?a=1&b=2";
expression = ToExpression(queryString); // convert to "Math()::Add(1, 2)"
int result = (int) rx.Eval(expression);

Instantiate an object from Web.Config setting.

// retrieve from config
string value = "MGutz.Person, Person";
expression = ToExpression(value); // convert to "[MGutz.Person] Person()"
IPerson person = (IPerson) Reflex.Eval(expression);

ne paroli Esperanto

2005-11-12T11:25:00.000-08:00

i don't speak Esperanto...

I remember chatting with ckknight in the IRC and he brought up the Esperanto language. He typed a few phrases and translated. Wow! I didn't really say that but it was interesting. The moral is I didn't know anything about the language, and with the help of a translator I was able to comprehend the sentence.

Boo has a fantastic extensible compiler. The InjectCode attribute I've been writing about has many possibilities. Yet, I doubt it will get much use outside my projects. If the C# compiler allowed something like InjectCode, there would be some interesting projects around it. C# has partial classes which functionally is what InjectCode does. It builds a partial class in memory at compile time. The important difference is by using InjectCode a source file becomes multi-lingual in the computer language sense. C# only understands C#. InjectCode has the potential to understand any language whether it be in XML, XMI (UML), SQL, etc with the appropriate translator (ICodeBuilder).

No programming language or approach has yet solved the dilemma of how to get every member of a team---DBAs, visual designers, coders, architects, analysts ...---to seamlessly work together. Some, like MDA zealots, believe that creating one visual language is the solution. A picture may be worth a thousand words but it takes a thousand pictures to explain the low-level details of an application. I may be reaching here but I think a multi-lingual language that allows each specialist to speak in their domain language may be a big stepping stone in harmonious development. For example, imagine injecting an analyst's use-case constraints directly into a class or method? Yes, some UML tools do that but it's an external, not integrated process. Good IDEs come close to being a multi-lingual language as they can draw forms, create HTML, etc but they're still well-coordinated individual pieces.

Hmm...maybe it's time to go back to college and do some graduate work.

Use XSLT Transformations to Ease Glade#

2005-11-10T16:40:00.000-08:00

Summary

This article describes how to use the XslCodeBuilder to simplify cross-platform Gtk# application development.

Introduction

In a previous article, I described the implementation of XslCodeBuilder to inject code generated by an XSLT transformation. In this article we'll apply XslCodeBuilder to ease Gtk# development.

There are at least two ways to build a Gtk# application. The first is to use the Gtk# library as is, manually creating widgets and compose them in a hierarchy to build your GUI. The easier way is to use the Glade# library. Draw your GUI using the Glade 2 utility then save the project creating XML metadata in a .glade file. The .glade file is loaded by the glade-sharp library to render your GUI.

Glade development is summed up in these steps:

Draw your GUI - use the glade utility to generate a .glade file
Declare variables for widgets - each widget in a .glade file has a unique id
Set event handlers - each widget has a set of 'signals' or events
Load the .glade file and run

The rest of the article describes the use of XslCodeBuilder to semi-automate these steps.

XslCodeBuilder and the Glade stylesheet

A .glade file is simply an XML file. It's simple enough to create a stylesheet to generate Boo code to perform the steps described above:

01 <xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
02  <xsl:output method="text" />
03  <xsl:template match="/">
04      <xsl:apply-templates  select="glade-interface/widget[1]"/>
05      def RunGlade():
06              RunGlade(null)
07      end
08
09      def RunGlade(init as callable):
10          Application.Init()
11          gxml = Glade.XML("view.glade", "window1", null)
12          gxml.Autoconnect(self)
13
14          if init != null:
15              init()
16          end
17
18          Application.Run()
19      end
20  </xsl:template>
21
22  <!-- build variables -->
23  <xsl:template match="widget">
24      [Widget]
25      <xsl:value-of select="@id"/> as <xsl:value-of select="substring(@class, 4)"/>
26      <xsl:apply-templates select="child/widget" />
27  </xsl:template>
28 </xsl:stylesheet>

Line 03 is the entry point into the stylesheet. The apply-templates statement starts a recursive descent into the XML file invoking any template that matches the current XML node. In this example, there is a template defined for match="widget". The template emits a declaration like this:

[Widget]
id as widgetClass

After all the widgets are found, processing continues at line 05 which emits overloaded methods of RunGlade(). The XSL-savvy will realize this stylesheet only acts on the first widget from the root and all its children. This is just a sample stylesheet. In all honesty I'm a Glade# NoOB. I need a better understanding of Glade to create a robust stylesheet.

Anyway, here is the source of a basic Glade# application using XslCodeBuilder and the stylesheet:

import Gtk from "gtk-sharp"
import Glade from "glade-sharp"
import MGutz.CodeInjection from "MGutz.CodeInjection"

[InjectCode(XslCodeBuilder, "view.glade;glade.xsl")]
public class GladeView:
    def Run():
        RunGlade()

InjectCode generates this code and stores it in a file so you can step through it in the debugger if needed:

    [Widget]
    window1 as Window
    [Widget]
    vbox1 as VBox
    [Widget]
    toolbar1 as Toolbar
    [Widget]
    openButton as ToolButton
    [Widget]
    quitButton as ToolButton
    [Widget]
    scrolledwindow1 as ScrolledWindow
    [Widget]
    textview1 as TextView

    def RunGlade():
            RunGlade(null)
    end

    def RunGlade(init as callable):
        Application.Init()
        gxml = Glade.XML("view.glade", "window1", null)
        gxml.Autoconnect(self)

        if init != null:
            init()
        end

        Application.Run()
    end

Create and compile a simple controller then run it:

o = GladeView()
o.Run()

If you click on either button nothing will happen. We need to hook events and display a FileChooserDialog when the Open button is clicked.

[InjectCode(XslCodeBuilder, "view.glade;glade.xsl")]
public class GladeView:
    def Run():
        RunGlade(Init)

    def Init():
        quitButton.Clicked += OnQuitButtonClicked
        openButton.Clicked += OnOpenButtonClicked

    def OnQuitButtonClicked():
        Application.Quit()

    def OnOpenButtonClicked():
        fcd = FileChooserDialog(Title: "Select File")
        fcd.AddButton("OK", ResponseType.Accept)
        fcd.AddButton("Cancel", ResponseType.Cancel)
        try:
            response = cast(ResponseType, fcd.Run())
            if (response == ResponseType.Accept):
                file = fcd.Filename
                using reader = StreamReader(file):
                    textview1.Buffer.InsertAtCursor(reader.ReadToEnd())
        ensure:
            fcd.Destroy()

The overloaded RunGlade(callable) is used to set event handlers. The handlers are self-explanatory. Run the controller, then click open

File loaded

Conclusion

I've shown how to use XslCodeBuilder to transform Glade's XML metadata to inject Boo code into a class.

Glade masters are probably thinking this doesn't save much time. I agree somewhat. The point is it does save time. When partial classes are merged into Boo event handlers can be autogenerated into a partial file to survive round-trips between regeneration. I didn't attempt to do that in this example as it's bad kharma to modify the source of the decorated class.

How to Inject Code from XSLT Transformations

2005-11-10T13:33:00.000-08:00

Summary

This tutorial describes how to implement an ICodeBuilder class to inject code from the result of an XSLT transformation.

Introduction

Application frameworks and GUI builders create and store metadata in XML format. We can take advantage of this metadata to autogenerate code. We can not only generate code, we can attach metadata to a class by utilizing the MGutz.CodeInjection library. When the metadata changes the booc compiler will update the class during the compilation phase.

The compiler cannot perform this magic on its own. We must create a class derived from AbstractAstAttribute to manipulate the AST nodes the compiler works with. The MGutz.CodeInjection.InjectCodeAttribute does exactly that, it uses 'lower-level' operations to inject code. InjectCodeAttribute is flexible in the sense that it can use any ICodeBuilder object to build a string containing Boo code.

The rest of the tutorial describes the steps to implement XslCodeBuilder, an ICodeBuilder derived object that uses the result of an XSLT transformation to inject code.

Implementing XslCodeBuilder

We have two options to process XML metadata: 1) create a dedicated ICodeBuilder to process specific XML metadata with XmlTextReader and Boo code, or 2) use XSL transformations. The first option is the most flexible as you have the entire framework class library at your disposal. The second option is an all-purpose solution using XSL stylesheets. I'll take the second approach for this article. The end goal is to decorate a class with an attribute that says take this xml file, transform it to Boo code then inject the code into the class during compile time. In other words,

[InjectCode(XslCodeBuilder, "metadata.xml;test.xsl")]
class XslTest:
    pass

To create a code builder implement the MGutz.CodeInjection.ICodeBuilder interface. The source for XslCodeBuilder is fairly simple.

01 public class XslCodeBuilder(ICodeBuilder):
02  TargetParser as BooParserType:
03      get:
04          return BooParserType.WSABooParser
05
06  def BuildCode([required] arg as string, [required] klass as ClassDefinition) as string:
07      // parse xml and xsl file name
08      args = arg.Split(char(';'))
09      if args.Length < 2:
10          raise ArgumentOutOfRangeException("arg", "Expected '<xml-file>;<xsl-file>', received: " + arg)
11      xml = args[0].Trim()
12      xsl = args[1].Trim()
13
14      // files are located in same folder as class
15      xml = Path.Combine(Path.GetDirectoryName(Path.GetFullPath(klass.LexicalInfo.FileName)), xml)
16      xsl = Path.Combine(Path.GetDirectoryName(Path.GetFullPath(klass.LexicalInfo.FileName)), xsl)
17
18      // transform to WSABoo code
19      using reader = StreamReader(xml):
20          xmlDoc = XPathDocument(reader)
21          stylesheet = XslTransform()
22          stylesheet.Load(xsl)
23          writer = StringWriter()
24          stylesheet.Transform(xmlDoc, null, writer, null)
25          return writer.ToString()

At line 02, TargetParser informs InjectCode that XslCodeBuilder is building WSABoo (white space agnostic) code. It's very difficult to control indentation in an XSL file. Don't waste any time making it pretty. This is a shortcoming of languages like Boo that rely on indentation. Fortunately, the Boo masters realized this and provide a WSABooParser class to handle the situation.

At line 16, InjectCode calls BuildCode() to build code. It has two arguments. arg is the argument from the attribute declaration, e.g. "metadata.xml;test.xsl". klass is the AST node of the decorated class.

The rest of the code converts the arguments into absolute paths then performs the XSLT transformation. The transformation result is returned to InjectCode as a string.

Using XslCodeBuilder

This example is found in the example/xsl directory.

First, decorate the class with InjectCode attribute. The example specifies test.xml as the metadata and test.xsl as the stylesheet.

# XslView.boo
import MGutz.CodeInjection from MGutz.CodeInjection

[InjectCode(XslCodeBuilder, "test.xml;test.xsl")]
class XslTest:
    pass

# test.xml - metadata
<def name="KR" comment="white book">
    print "Hello world!"
</def>

# test.xsl - stylesheet to transform metadata into WSABoo code
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output method="text"/>
    <xsl:template match="/">
        <xsl:apply-templates />
    </xsl:template>

    <xsl:template match="def">
        # <xsl:value-of select="@comment"/>
        def <xsl:value-of select="@name"/>():
            <xsl:value-of select="."/>
        end
    </xsl:template>

</xsl:stylesheet>

Create a simple controller to use the XslTest.

o = XslTest()
o.KR()

Compile it all together, then run it. The build file for the example creates an executable xsl.exe:

> xsl
Helo world!

Conclusion

This article described the implementation of XslCodeBuilder, an ICodeBuilder derived class to perform XSLT transformations on XML metadata generating Boo code. The Boo code is injected into a decorated class each time the class is compiled syncing the class with the metadata.

A future article applies XslCodeBuilder to Glade#/Gtk# application development.

Dependency Injection and Scripted Configuration for Enterprise Development - Part 2

2005-11-09T01:45:00.000-08:00

Nano and a Configuration Script - Thinking Outside the Box

In this part of the article series, I discuss the external configuration file NanoContainer compiles and invokes to create a managed container of objects. An application retrieves objects from this managed container to perform business logic. These objects are not necessarily instantiated. They may be lazy-loaded for performance and resource considerations.

To get a better idea how Nano integrates with an application, consider this sequence diagram:

Administrator starts an application.
Application asks Nano to create a container.
Nano compiles the configuration script.
Nano sets IScript.Parent to a default container.
Nano asks the script to Compose() the container.
Script returns a container.
Nano returns the container to the application.
Application requests objects from the container and uses them to perform busines logic.

Pretty simple, eh? It's worth reiterating that Nano can use a script written in practically any .NET supported language---Boo, C#, VB, JS, JavaScript, VBScript or even an XML configuration file. Pick your poison. One benefit of using Boo (other than it's more elegant than the other .NET languages) is that a Boo install has an interpreter which may be used to run the script independent of the application! Nano gives a team the full power of any .NET language to configure an application. At first this may not seem to be much of an advantage. Think outside of the box! What if a a configuration file could test itself? What if a configuration file could perform a check on its environment before running an application? What if a configuration file could test for missing property values? What if a configuration file could swap in mock objects? A scripted configuration file's behaviour is only limited by your coding skills. Before we go into the actual configuration file, consider the class diagram for this configuration block.

Class	Purpose
ConfigBase	Is the base class for all configuration files. The class contains methods to check for missing keys, test settings any other future features. It also contains a `Settings` hash to hold objects derived from `SettingBase`.
SettingBase	Is the base class for all settings. This class defines an abstract test method that must be implemented.
RequiredSetting	Defines a setting that is required. When `Test()` is executed it always throws an exception. Must be replaced by another setting.
StringSetting	Defines a string setting. `Test()` simply returns with no side effect
SafeCreateDirSetting	Safely creates a directory. `Test()` creates the directory if it does not exist. It's assumed `Test()` will be called once.
SqlConnectionStringSetting	Defines a SqlConnection string. `Test()` will attempt to open a connection to the database.
BooConfig	Is the external script file. It does not reside in the `ConfigBlock` assembly.

The complete source of the configuration file follows:

# BooConfig.boo - compiled by NanoContainer when an application starts.
01 class BooConfig(ConfigBase):
02     # Gets or sets check mode.
03     [property(IsCheck)]
04     _isCheck = false
05
06     # Populates the master settings hash.
07     override def PopulateSettings():
08         Settings = {
09             "name" : StringSetting("mgutz"),
10             "connectionString" : RequiredSetting(),
11             "logDirectory" : SafeCreateDirSetting("/temp/log")
12         }
13
14     # Set the environment for DEV.
15     private def DEV():
16         Settings["connectionString"] = SqlConnectionStringSetting("Server=(local);Database=devapp;uid=boodev;pwd=boodev")
17
18     # Set the envrionment for TEST.
19     private def TEST():
20         pass
21
22     # Set the environment for PROD.
23     private def PROD():
24         Settings["connectionString"] = SqlConnectionStringSetting("Server=(local);Database=prodapp;uid=booprod;pwd=booprod")
25
26     # Composes the object container used by an application.
27     # This method is invoked by an application.
28     override def Compose() as IMutablePicoContainer:
29         _container = DefaultPicoContainer(Parent)
30
31         # change to match environment, could be automated by pinging a unique serverr
32         EnvironmentMethod = DEV
33         ConfigureEnvironment(_isCheck)
34
35         # register an instance of an object
36         _container.RegisterComponentInstance("name", Settings["name"].ToString())
37
38         # lazy-load a connection object by registering its implementation not instance
39         _container.RegisterComponentInstance("dbConnectionString", Settings["connectionString"].ToString())
40         params = array (IParameter, (ComponentParameter("dbConnectionString"),))
41         _container.RegisterComponentImplementation("dbConnection", SqlConnection, params)
42
43         # return the container
44         return _container
45
46 # A check on the environment is performed when this script is called from
47 # booi interpreter. When this script compiled by Nano, this method is not invoked.
48 # These are the same steps taken by the NanoBoo contaienr except the .IsCheck
49 # attribute is not set.
50 def main():
51     config = BooConfig()
52     config.Parent = DefaultPicoContainer()
53     config.IsCheck = true
54     config.Compose()
55
56 main()

The Boo file contains a single class BooConfig derived from ConfigBase. Line 32 must be changed based on which environment the application is deployed. This could be automated by pinging a unique server in each environment. If the server pings back, use the appropriate environment method. Additionally, the file contains a main() method that is executed when the file is run from the command line:

> booi BooConfig.boo

So, what is happening?

First, excuse the sequence diagram. Apparently my UML CASE tool can't handle wierd sequences. The lifeline of the methods should be continous as all of this is actioned in one call to Compose().

Nano, if application is running, or booi, if run from command line, invokes Compose() to create a container.
Script file creates a container to hold objects it builds.
Script file sets the environment method callback.
Script file asks ConfigBase to configure the environment.
ConfigBase in turn asks the script file to populate the Settings hash.
ConfigBase then invokes the environment method, say DEV() to use settings specific to the DEV environment.
If checkEnvironment was true, ConfigBase checks for missing keys by iterating over the hash looking for null or RequiredSetting() objects.
If checkEnvironment was true, ConfigBase iterates over the Settings hash invoking SettingBase.Test() on each object. This tests database connections, missing directories, web service connections, etc.
Flow returns to the script file, where it nows registers implementations and instance of objects storing them into the container previously built. See lines 35-41 where objects are inserted into the container.
The container is returned ending the sequence diagram.

As you can see, a script configuration file can be much more than static key and value pairs. With a little bit of thought, it's very easy to create a configuration block of your own that goes far beyond the Microsoft Enterprise Configuration Block.This article described a simple, yet powerful configuration block that:

Adapts an application to different server environments
Validates the existence of required settings
Performs test based on the type of setting
May be edited and tested by an administrator before running an application
Inserts object instances and implementations (lazy-loading) into a managed container

The block could easily be extended by creating additional SettingBase-derived classes. Some examples include:

PingSetting - ability to ping expected servers
FtpSetting - checks an FTP connection and perhaps drop files into it
HTTPSSetting - ensure SSL certificates are accepted
EmailSetting - validate mail servers
etc., etc., etc.

To be continued...

Indeed this is just the tip of the iceberg. I intend to create a Utility class to faciliate use of PicoContainer. It's just too much typing and looks more complicated than it really is. In the next part, I'll cover a few scenarios describing the tangible benefits of NanoContainer in testing and refactoring.

C#, VB to Boo Converter in SharpDevelop

2005-11-08T15:45:00.000-08:00

I just did an svn update on SharpDevelop. Daniel Grunwald imported a standalone C#/VB to Boo converter. Awesome! I was about to do this myself. Anyways, it's as simple as this:

    StandaloneConverter -o boofiles d:\temp\someCsProject

This recursively converts any C# or VB source file found under d:\temp\someCsProject into a Boo file into boofiles. The directory structure is maintained. A huge time saver! Wish it had a simpler name though like csvb2boo.

This has been a good week for updates: TestDriven.NET, GhostDoc, StandAloneConverter

Dependency Injection and Scripted Configuration for Enterprise Development - Part 1

2005-11-08T11:16:00.000-08:00

Summary

This article describes how script-like configuration files and the Dependency Injection pattern (DI) prevent many of the issues encountered when building an enterprise-level application. Two relatively unknown and powerful open source technologies are used: NanoContainer.NET IoC framework and the agile .NET language Boo.

Application Configuration

An application typically configures itself by retrieving settings from a static configuration file in the form of XML, property files, registry values or some other text format. The setting values are used to instantiate objects, then objects are wired together to perform the functions of the application. If implemented poorly, this infrastructure logic is the source of many bugs and wasted man hours. Even when implemented properly the infrastructure logic is intertwined with business logic deviating from the Separation of Concerns principle.

The dependency injection pattern (DI), in practical terms, is the opposite. DI injects dependencies into an application instead of the application pulling and building its dependencies in code. DI injects pre-wired, ready-to-use objects into an application. The application simply asks for a database connection from a managed container instead of retrieving a connection string then instantiating a database connection object. An application simply uses a pipeline object containing wired pipes and filters without having any knowledge of sequence or how to instantiate pipes and filters. The developer focuses on business logic. Factories, singletons become redundant. Unit testing is easier as mock objects are easily injected without any change to the application.

I use the term DI very loosely. DI is not a design pattern in the sense that it can described within a UML class diagram. DI is a methodology. DI is the fundamental principle behind Inversion of Control (IoC) frameworks. These frameworks not only build and instantiate objects, they provide additional benefits such as interceptors, object pooling, objects lifecycles, etc.

Application Deployment

Most medium to large teams deploy aplications to different server environments as an application matures in the development cycle. Environments have a similar network infrastructure and run the same version of software as the actual production environment. Each environment serves a specific purpose whether it be for integration, testing or staging for actual deployment. Here are some common environments utilized within a development cycle:

`PERSONAL`	Individual developer box with isolated database, web server, etc. Virtual machine software is an indispensible tool for emulating the servers found in production.
`DEV`	An application is deployed to DEV when a developer has met all requirement for his task including successful unit tests. Developer's are usually given admin access to servers in this environment to facilitate diagnosis of issues that surface. Preliminary black-box testing is performed against this environment in preparation for formal testing. (Try to run the same tests as QA). Developers must resist the urge to deploy a single web page or assembly. Fully automated installations are a must.
`TEST`	The quality assurance (QA) team's environment. Developers do not have server access to this environment. Installation, database script issues, trivial bugs should have been caught in the `DEV` environment. The QA team tests the application at a much higher level and more than likely different than developers ensuring the application meets business requirements. It behooves the development team to build or reuse a logging framework to not only log diagnostic messages but also have a website/tool to view and query the logs.
`PROD`	The exact same application bits passed in `TEST` are deployed to `PROD`. The logging threshold is increased to log info and errors.

What can go wrong? Many things, and it will! Some of the culprits are access permissions, invalid or missing configuration settings, DLL hell (Java and .NET too), different software on the servers, bad network connection, missing keys in the database, human error, etc---so much can go wrong.

To be continued...

In the next article I'll detail how configuration scripts, written in any .NET supported language (Boo, C#, VB, J#, JavaScript and VBScript), and the NanoContainer.NET IoC framework alleviate configuration and deployment issues. I'll cover Boo specifically as it provides additional benefits over C# and VB.NET.

When is a PDB not a PDB?

2005-11-02T09:42:00.000-08:00

I have been trying all week to debug the CodeInjection library using the .NET SDK's CLR debugger clrdebug.exe. I was able to step through several examples but not the library itself. I checked for the .PDB file and it was there! WTF is going on? I searched the web for a PDB dumper/lister/viewer, nothing. I had no choice but to break out a hex editor. Voila. I don't know what the heck I was looking at but I noticed the executables I could debug had the source filenames embedded into them, my library did not.

I was building them differently. It turns out booc, the command line version, has Parameters.Debug set to true by default, and booc, the NANT task has it false by default even though it generates a .PDB file. Make sure to set debug to true if you want to use clrdbg.

    <booc output="${build.dir}/MGutz.CodeInjection.dll" target="library" debug="${debug}">
        ...
    </booc>

Here's another tip. If you want the debugger to come up automatically, put this in your code.

    // Windows will prompt to start the debugger (if you have SDK installed)
    System.Diagnostics.Debugger.Break()

CodeInjection uploaded to boo-contrib

2005-11-01T12:02:00.000-08:00

I've uploaded the CodeInjection library to boo-contrib. The SVN adress:

svn+ssh://anonymous@forgesvn1.novell.com/svn/boo-contrib/trunk/mgutz
password: anonymous

You'll have to enter the password several times if you're using TortoiseSVN.

You'll need the SVN version of Boo. I'm using a helper method in CompilerErrorFactory not found in the current binary release.

Brail, the genesis

2005-10-31T12:41:00.000-08:00

A Jason fellow left a comment about the simliarities of TemplateCodeBuilder to Ayende's Brail. Funny he should mention that. I have been thinking about creating an open-source project for quite some time. I initially wanted to create a .NET Web MVC framework but stumbled upon Castle. Why build yet another web framework?

Castle has multiple view engines. I found Brail intriguing because of Ayende's choice of language, Boo. I knew Ayende was a keen guy based on the ideas written on his blog, so Boo was worth a looksie. Wow!

One day I was coding and it hit me ... In one hand I had just discovered a great .NET language, Boo, built on an exensible compiler architecture. In my other hand I was creating a SQLite provider for CodeSmith. Why not create an open source code generation framework? Why not extend Boo to convert templates into code so you don't need CodeSmith or MyGeneration? Why not generalize the idea and transform any metadata into code? Why not indeed.

It comes as no surprise then that my first example is Brail like. It's funny how things come full circle.

CodeInjection next steps

2005-10-30T19:48:00.000-08:00

I'm close to releasing the CodeInjection library. The only issue I have left is how to force the debugger to step into the intermediate files InjectCode generates. I'll chat with the Boo masters on IRC and get some ideas.

So, what's next? I'll write yet another article on how to use this new facility to fill in your classes from an iBATIS.NET data mapping.

iBATIS.NET is a data access framework. If I had to categorize it, it's somewhere between Microsoft's Enterprise DAAB and NHibernate. Most enterprise projects I've been involved with forbid me or any developer from touching the database. (They eventually let me since I'm pretty good with T-SQL). In general, DBAs are responsible for creating stored procs and permissions. No direct SQL to the database. iBATIS is perfect for this since it can map any SQL including stored procedures to objects. It doesn't try to do what NHibernate does but it does map to objects which the DAAB doesn't. With code injection your code will always be in sync with your data mapping file. Of course, you could take it a step further and generate your data mapping file using AspTemplateCodeBuilder.

FYI: The ANTLR packaged with Boo generates Boo code. NICE! I found a couple of bugs and entered them into JIRA but they're minor. I used it to parse the ASP-like template files.

Boo as a configuration language

2005-10-30T19:22:00.000-08:00

I wrote an article for CodeProject.com extolling the benefits of scripting (Boo, C#) over static XML, INI files for application configuration. I never submitted and now I can't find it! It explained how a script file can perform a precheck of the environment before starting the application. For example, the configuration file could check a database connection, pull down a WSDL to validate a web service connection, etc. Boo is perfect for this since an administrator can change a value in the Boo script and simply run it again with booi. The application uses the script file to configure itself without having to do any casts to strings, etc. Moreover, scripts can detect missing key values across different environments---development, test and production.

I'll write it up again. If you're wondering, it's Boo + dependency injection, a potent combination.

Boo is mixin it up

2005-10-29T17:44:00.000-07:00

What is a mixin? A mixin is an interface with implemented methods. When a class includes a mixin, the class implements the interface and includes, not inherits, all the mixin's attributes and methods. They become part of the class during compilation. Interestingly enough mixins don't need to implement an interface. The advantage of implementing an interface is obvious so that the class may be passed to methods requiring that interface.

Ayende's idea about using code injection to implement static mixins motivated me. How would you define a mixin? You could try this, but Boo wouldn't like it. It's not an interface:


interface IMath:
 def Add(a as int, b as int):
  return a + b

Here is what I've come up with. To create a mixin define the interface as usual, then define mixin attributes and methods in a separate file and inject them during compilation.

# IMath.boo - interface to our math mixin
interface IMath:
 def Add(a as int, b as int) as int:
  pass

# IMath.mxn - implementation of the interface with a bonus
[property(Result)]
_result as int

def Add(a as int, b as int) as int:
 _result = a + b
 return _result

The first file is self explanatory. The second is interesting in that it implements the IMath.Add() method and a Result accessor. To mix this in with a class use the InjectCode attribute using MixinCodeBuilder.


# english.boo
import MGutz.CodeInjector from "MGutz.CodeInjector"

[InjectCode(MixinCodeBuilder, "IMath.mxn")]
class English(IMath):
 def construtor():
  pass

 def Noun() as string:
  return "Boo"

When booc compiles english.boo, MixinCodeBuilder comes alive and parses the code in "IMath.mxn" and adds Add() and Result to English. Here is the test:


mixin = English() as IMath
print mixin.Add(2, 3)

o = English()
o.Add(6,6)
print o.Result

As you can see, English implements IMath as well as has a Result accessor. I like this approach as you have the option to implement the mixin methods by simply not injecting the mixin code. The downside is declaring the interface and injecting the code are two steps.

This wasn't that difficult to implement. To be honest the CodeInjector is probably less than 400 lines of code and that may be an exaggerated number. Boo just does so much. OK, I haven't really tested other scenarios but so far, so good!

Must-have free Win utilities

2005-10-28T23:19:00.000-07:00

AutoHotKey: Program your keyboard to do anything. Use it to map or remap your entire keyboard. For example, I have Win+V start Visual Studio, Win+space to start a command window, PrtScrn to start Snagit. More powerful than any commercial keyboard utility, guaranteed.

Autoruns: These guys probably know more about Windows internals than Microsoft. This utility shows you *every* exe, dll that runs at startup with the option to prevent them.

CDX: A change directory utility for the command line. When you type cdx an Explorer dialog pops up to select your folder. Use the arrows key to select your folder. Right arrow to expand. Left arrow to contract. Hit enter. You don't need no stinkin' mouse and the focus remains on the command line. It's faster than Tab. The link is direct to the download since I don't read Japanese. Winrar will open the LZH file.

KeyNote: Jot your ideas down in a tree-type (outline) metaphor. Use it to store code snippets, URLs or whatever.

Regulator: A regular expression builder, tester focused on .NET regular expressions. This was featured on "Ten must have tools...". I don't mention the others in that article, you probably use or know of them.

Smart Paster: Grabs text from the clipboard and pastes it into Visual Studio as a StringBuilder, string, comment, etc. Saves you from having to escape newline, tabs, quotes, etc.

I think a generic taskbar applet that has the functionality of Smart Paster would be neat. This way it could be used in any editor, IDE. Hmmm ... maybe my first GUI-based Boo app. Of course it would have to be extensible so you could do things like convert to HTML.

Visual Studio 2005 Gotcha

2005-10-28T19:50:00.000-07:00

I guess I'm old school in that I always partition my drives into an operating system drive (C:) and an applications/data drive (D:). My small form factor PC allows for a single drive. When I partitioned the drive, I figured 6 gigs would be enough for the Windows operating system.

Apparently not! Microsoft and other companies tend to install parts of its application on the C: drive even if you tell it to install in D:\program files. The .NET Framework for example will always install itself in the Windows directory. I finally finished downloading Visual Studio 2005 and tried to install it. Bam!

You cannot change the installation location for these products. These products have a dependency on a file already installed in another location, which prevents these products from being installed in a custom directory. The products you are trying to install are dependent on the following programs: * Microsoft .NET Framework 2.0 SDK - ENU

What? I installed the .NET 2.0 SDK on my D: drive earlier in the day. Further reading tells me I should first uninstall then reinstall the Framework to the drive where I want to install Visual Studio. Where do you think it is dummy?! VS needs 2.6 GB of space and my C: drive has 3.0 GB of free space. That doesn't leave Windows enough room to download and apply upgrades, etc. Been there, done that. Aaargh. I have an old version of Partition Magic, I hope it works on XP SP2. Otherwise, I'll have to run to the nearest software store to buy partitioning software. I bet this eats up one day. Visual Studio 2005 better be worth it!

Boo as a template compiler

2005-10-27T00:51:00.000-07:00

Lazy Style Code Fu master (ahem) me presents Boo the template compiler.

If you read "custom preprocessor idea for Boo" then now you'll see the start of one of the projects I had in mind. Thanks to the Boo newsgroup I discovered Boo has compiler attributes that have the behaviour I want. I started working on it yesterday and code generation works.

Say you want to generate classes for tables in your databases. First retrieve the database metadata. I represent the metadata returned in a Hash named model. model contains the tables for the database as well as the name and columns for each table.

# controller.boo
import test from "test"

model = {
    "tables" : [
        {   "name": "user",
            "columns": ["user_id", "username", "password"]
        },
        {   "name": "item",
            "columns": ["item_id", "name", "description"]
        }
    ]
}

f = View(model)
print f.RunTemplate()

Don't let the Hash scare you. If you plan to do any AJAX, get used to the JSON format. I much prefer it over XML. You would probably build stongly typed classes for the metadata. JSON is just a quick way to represent objects. Back on topic, staying within the MVC pattern, create a view for model.

# view.boo
import MGutz.CodeInjector from "MGutz.CodeInjector"

[InjectCode(TemplateCodeBuilder, "view.tpl")]
class View:
    [property(Model)]
    _model as Hash

    def constructor(model as Hash):
        _model = model

View doesn't do much but accept a model. The magic happens in the compiler. InjectCode is a special attribute instructing the Boo compiler, "invoke me when you compile this class." When the atribute is invoked, it instantiates TemplateCodeBuilder to process the contents of the view.tpl file. TemplateCodeBuilder converts the file into Boo code and it becomes the body for a RunTemplate() method. InjectCode then injects RunTemplate() into the class View.

Look at the view's template using the model.

# view.tpl - template for the view
<% for table as Hash in Model['tables']: %>
class <%= table['name'] %>:

    <% for column in table['columns']: %>
    [property(<%=column%>)]
    _<%=column%> as string

    <% end %>
<% end %>

It's ASP(.NET) syntax. Notice how Model is accessible here. You could access the instance variable _model just as well. Remember this template is injected as a function into View, it has access to private methods, private fields, ... like any member of the View. Also note the end keyword. This is an alternate Boo syntax when using it in markup scripts. end is a required keyword in your templates.

Compile this baby. view.boo + view.tpl merges into a single class into test.dll.

> booc -t:library -out:test.dll view.boo

Run the controller.

> booi controller.boo

class user:
        [property(user_id)]
        _user_id as string

        [property(username)]
        _username as string

        [property(password)]
        _password as string

class item:
        [property(item_id)]
        _item_id as string

        [property(name)]
        _name as string

        [property(description)]
        _description as string

Voila! Instant classes for your database. Remember, the template is compiled. Think of a template as a function in an abstract format. You now have a compiled class that executes your template without any run-time parsing, merging, etc. This is ideal not only for code generation but also for views in a web framework.

I'll finish up by saying Boo is such a wonderful language. It's what VB.NET should have been. Even down at the compiler pipeline level you just sense how solid Boo is.

If you're wondering what the generated code for View.RunTemplate() looks like.

def RunTemplate() as string:
        out = System.Text.StringBuilder()
         for table as Hash in Model['tables']:
out.Append("class ")
out.Append( table['name'] )
out.Append(":\r\n")

 for column in table['columns']:
out.Append("\t")
out.Append("[property(")
out.Append(column)
out.Append(")]\r\n")
out.Append("\t")
out.Append("_")
out.Append(column)
out.Append(" as string\r\n\r\n")

 end
 end
out.Append("\r\n\r\n")

        return out.ToString()
end

custom preprocessor idea for Boo

2005-10-25T10:18:00.000-07:00

I remember reading the possibility of adding language directives to Boo. If they do, then I propose a feature which I have not seen in other compilers---the ability to invoke a custom preprocessor.

Imagine the possibilities. You could include an ORM XML mapping file, preprocess it to automatically generate accessors within the class. You could include a XAML file and bind events. There are so many more neat things you could do.

Here's an example applied to my yet to be released MVC framework. A view is comprised of a template and code-behind much like ASP.NET. What's the advantage? There is no run-time compilation, no parsing, no reflection, etc. The view and template merge and compile into a single type. I realize ASP.NET gets compiled as well but it's a separate framework, this becomes a first class feature of the language.

# MyTemplate.html - maintainted by a designer. Rows between <%=* %> and <%*%>
# are omitted when rendering. It's an aid for designers.
<html>
    <table>
        <%=* Cycle([EvenRow, OddRow], Model['Items']) %>
        <tr class='even'><td>even row omitted when compiled</td></tr>
        <tr class='odd'><td>odd row omitted when compiled</td></tr> 
        <tr class='even'><td>even row omitted when compiled</td></tr>
        <%*%>
    </table>
</html>

# MyView.boo - code-upfront (code-behind)
class MyView(ViewBase):
    def OddRow(s as string) as string:
        return "<tr class='odd'><td>${s}</td></tr>\n"

    def EvenRow(s as string) as string:
        return "<tr class='even'><td>${s}</td></tr>\n"
        
    #include "MyTemplate.html", TemplateProcessor

The line above means preprocess MyTemplate.html using a custom preprocessor TemplateProcessor. Here's the pseudocode:

#converts an html template into a boo snippet
class TemplateProcessor(Boo.IPreProcessor):
    def Process(filename as string) as string:
        // read the contents of the file
        // convert html template into code
        // return BuildOutput() code

After preprocessing the class contains the generated BuildOutput()

# class with generated function
class MyView(ViewBase):
    ... 
    
    // called by ViewBase.Render()
    def BuildOutput() as string:
        out = StringBuilder()
        out.Append("html\n\t<table>\n")
        Indent("\t\t")
        out.Append(Cycle([OddRow, EvenRow], Model['Items'])
        Dedent()
        out.Append("\t</table></html>")
        return out.ToString()

The view simply cycles between OddRow() and EvenRow() applying values from Items resulting in an HTML table with alternating row styles.