/// Wednesday, November 30

SVN access for Reflex and CodeInjection

I've moved the source for the CodeInjection and Reflex library to ckknight's SVN box. NovellForge was too much of a pain with passwords. I'll import the rest of the Gamba framework to there once I feel the core interfaces and implementations won't change. The libraries above may be used separately from Gamba and are licensed under APL 2.0 .

Here's the address: svn://ckknight.no-ip.org/mgutz/trunk

There isn't much documentation, well because I've been lax and Boo doesn't have a formal documentation tags syntax. Your best bets are the articles I've written in this blog and the examples or unit tests.

Gamba: First-Class Branding

More refactoring today. I've promoted the branding idea to a full pledged object model. The entry point (after front controller) into Gamba will not be a controller as in traditional MVC2, rather a brand. The importance of this is a brand controls access to a set of controllers and also encapsulates look-and-feel. For example, an executive brand of a corporate website allows the use of the FinancialsController and CorporateStrategiesController as well as any other controllers it inherits through derivation.

If developers do not wish to use brands, then the DefaultBrand will simply be a pass-through to a controller as in normal MVC2. Here's a simplified UML diagram. (There's actually a few classes between dispatchers and brands like IActionProxy).

Gamba: Purpose

I should have formally announced what Gamba is. Gamba is a web framework built on a well understood design pattern, MVC2. MVC2, unlike ASP.NET which is a simple page controller model, has a front controller directing actions to sub controllers. The duty of the front controller is to further separate the concerns of what a page has to do from what an infrastructure should provide. Every page should not be concerned with user authentication or how to handle exceptions. In the ASP.NET model where users are accessing the page directly, you're forced to handle it on the page or by inheritance. Ideally a page should only be concerned about rendering data its given, nothing more. The front controller also insulates your users from changes in the backend since URLs are logical paths not physical paths as in ASP.NET. I would highly recommend googling front controller or MVC2 to get a better understanding of this pattern.

My goal for Gamba is to create a web framework suitable for consulting work. Gamba has first class infrastructure support for things like branding. The HTML templates will be editable in design tools like DreamWeaver. The data templates will use normal SQL queries or stored procedures making it easier to integrate with legacy data. Gamba is based on Boo because I believe and have shown it to be more flexible than any other .NET language. In the end, I'll show how you can build a blog in less time than Ruby on Rails and have a fully compiled application with no templates. It will be more scalable and faster.

KISS and the 80/20

I design with two principles in mind, KISS (keep it simple stupid) and the 80/20 % rule. Ckknight asked a question today in IRC about the data layer using hardcoded SQL. What if you want to switch databases? My reply was something to the effect "if you're not using generic SQL then you'll have to change templates. There's no way to get around it." The Gamba data layer implements the basic DAO design pattern, using SQL queries is by design and may be changed without affecting the rest of the application.

In all my years of development, I've only had to change the backend database once on a production application! Thankfully stored procedures were used and the DBAs did most of the work. Our DAOs worked pretty much as is, except SQL Server returns a dataset whereas Oracle returns a cursor. Most companies simply do not switch out databases because one is better. If something works, maintain it.

On a sidenote, what a mistake switching from MS/SQL Server to Sun/J2EE/Oracle was. No big gains at all. The new CIO had a thing against Microsoft. Funny how we spent literally a million dollars on servers and J2EE/Oracle licenses. Guess which applications are still handling millions of transactions? VB6 MTS components running in a web farm on cheap $5000 Windows servers compared to $30000 Sun boxes. The other is the C# application I architected. This was the application we switched to Oracle to supposedly gain performance. In Oracle's defense, we had to use Microsoft's Oracle drivers. At that time Oracle's driver was still in beta. The moral is no fancy J2EE distributed gumbo. Each core application relied on MSMQ with a 20 or so server farm hiding behind a round-robin TCP-IP dispatcher. KISS at the enterprise level.

Back on topic. Gamba will be flexible, but not to the point where it becomes over-architected, over-engineered, over-OOPsed as I like to say. It will be simple and 80% of what AJAX web developers need will be very easy to do. The other 20% will require the usual manual effort by the developer. What you get is a fast, light framework without the bloat. Features will be added as needed not what I think the future will need. Who knows, AJAX may be gone in a few years when the next big thing hits.

/// Tuesday, November 29

Gamba: Data

As I thought more about a data access layer for Gamba, my ever flickering idea light came on and the solution was right in front of me. One of the goals of Gamba is to use Boo for everything, configuration, templates and so on. Why not use the CodeInjection library to simplify data access?

No matter which data access library is used, the library has to be told via a metadata file what data is queried and how to map those results to objects. (The exception being code generators which may need as little info as a connection string.) With the magic of CodeInjection metadata files can be transformed into managed code. And thus the genesis of Boo Data Templates.

Some advantages:

Disadvantages:

Boo Data Templates

Here's the bird's eye view of Boo Data Templates:

Each template contains a query and possibly a map definition if the result of the query is to be mapped to properties of the class. The templates are injected into Person via the MixinDataTemplates attribute. As an example, the Person.SelectAll.bdt template contains this:

query:
    Select = "select person_name, person_gender from person"

map:
    Name = person_name
    Gender = person_gender

The class which uses the template looks like this:

[MixinDataTemplates]
class Person:

    [property(Name)]
    _name = ""

    [property(Gender)]
    _gender = "M"

After compilation, the class looks like this:

class Person:
    [property(Name)]
    _name = ""

    [property(Gender)]
    _gender = "M"

    def SelectAll() as (Person):
        people = ExecuteQuery(queryFromTemplate)

        for p in people:
            person = Person()
            person.Name = people['person_name']
            person.Gender = people['person_gender']

    def Find(key) as Person:
        ...

    def Update():
        ...

Next Steps

Cameron Kenneth Knight (ckknight) is taking this idea forward. This will be a very nice data layer. More details will come as the library progresses.

/// Monday, November 28

Just Data Please

Why is it so difficult to do generic data access with .NET? Why not follow JDBC? Universal data access, remember that MS technology, not in .NET. Abstract as they might, ADO.NET still sucks. Is it that hard for MS to swallow their pride and implement a JDBC model? Or, is being able to create a SQL Server specific driver the reason ADO.NET sucks? MS knows how to do it they refuse to in ADO.NET. Every other data access library from Perl, Python, Ruby ... Java does it right.

I need a nice Data Access library for Gamba. I like iBATIS.NET mapper but it's doing more than I want. iBATIS has a lower level dData access library which may suit my needs. My ideal data access would be the MS DAAB (Data Access Application Block) but a truly generic version without needing any of the other MS blocks like configuration, logging. I don't find ORMS to be appealing as they assume 1) you have control of the database, 2) I prefer metadata and 3) stored procedures are good.

Gamba: Branding

Gamba, the Boo MVC2/Rails framework is coming along nicely. As I add features to Gamba I'll write a short article explaining the usefulness of the feature. This article focuses on branding, the ability for a multi-user website to appear and behave differently based on the user.

Motivation

A very simple type of branding is achieved by most blog hosts. They allow users to specify a different template for their personal blog. For enterprise websites branding requires more than a new 'skin'. Sub applications of an enterprise-level web application may access a different database or perform specific business logic tailored for the client.

Example

In this example, we'll create a simple web app which changes behavior based on the brand's URL. Since Gamba is an MVC2 framework, pages are not directly accessible as they are in ASP.NET. Don't let this dissuade you. Creating a view is as easy as deriving from DefaultController and a template for each view.

// Foo.boo - controller
[MixinTemplates]
class Foo(DefaultController):
    def Hello():
        ViewMethod = HelloView

    def Name():
        Model["name"] = "mario"
        ViewMethod = NameView
// Hello.HelloView.tpl - template for HelloView
<p>Hello world!</p>
// Hello.NameView.tpl - template for NameView
<p>Hello ${Model['name']}</p>

Gamba's IController has many views whereas an ASP.NET page has one view, the .aspx page itself. Gamba may choose not to call any views of a controller if, for example, the user has not logged in. To view the views, compile the class above and drop the assembly into Gamba's bin folder, then browse:

Let's do something different for any user of the boo sub application. A sub application is accessed via a different URL prefix - http:://localhost:8080/gamba/boo. Gamba will automatically reuse the controller above if we don't do any modifications. Let's create a new layout and also create a controller for the boo sub application. The layout is basic, we'll just change the background to green. I don't use a template here just to show you don't have to use templates at all:

class BooLayout(DefaultLayout):
    override def Execute() as List:
        htmlTop = "<html><head></head><body bgcolor='green'>"
        htmlBottom = "</body></html>"

        return [htmlTop, _child.ViewMethod(), htmlBottom]

Let's assume Foo has a lot of functionality in it and we want to modify one view. Using OOP derive a controller from Foo.

[MixinTemplates]
class Bar(Foo):
    override def Name():
        Model['names'] = ['mgutz', 'bamboo', 'ckknight']
        ViewMethod = NamesView
# Bar.NamesView.tpl - template for NamesView
<% for name in Model['names']: %>
<p>Hello ${name}</p>

<% end %>

Again, compile the classes and drop into Gamba's bin folder. Browse the two pages noting the use of 'boo/bar':

Conclusion

This article described how Gamba solves one of the challenges facing web developers, branding. In particular, Gamba uses object-oriented programming to alter behaviour and appearance unlike some other frameworks which resort to using a template hierarchy.

BTW, as someone commented, I use FireFox. I used IE so I could capture the address bar in a small window.

/// Friday, November 25

Delaying Code Execution with Reflex

I've already found a nice benefit of using my Reflex library. I can delay the execution of an entire block of code. Here's the scenario. A user browses this address:

http://www.mgutz.org/gamba/person/show?id=1

To Gamba (Boo web framework) it means invoke Person.Show(). When the URL above is received, Gamba forwards the URL to ReflexDispatcher for dispatching. ReflexDispatcher returns a ReflexActionProxy object which may be executed at any time. Delaying execution of the action makes Gamba flexible. The action could be chained with pre and post interceptors, it could be queued, or completely discarded by a filter in the pipeline. Here' the pseudo-logic in ReflexDispatcher.

    def Execute():
        klass = ParseClass(_context)
        method = ParseMethod(_context)

        script = """
            ${klass}.new() {
                Context = $0
                ${method}()
                return this
            }
        """

        return ReflexActionProxy(script, _context)

How does it work? First remember Gamba has to parse the class and method from URL. Gamba creates a Reflex script to avoid doing any reflection manually. The script is then packaged into an action proxy along with argument $0 (_context object). The script above, after string interpolation looks like this:

    Person.new() {
        Context = $0
        Show()
        return this
    }
  1. Create Person() controller
  2. Assign the HttpContext
  3. Invoke Show()
  4. return the instanstiated Controller

To do all of this, ReflexActionProxy simply evaluates the script.

    return Reflex.Eval(script, context)

You may be wondering is Reflex using CodeDom? DEFINITELY NOT! Reflex is optimized for small, reflection-based scripts. Reflex caches types, methods and properties to minimize performance loss.

/// Monday, November 21

VS2005 Tip - How to Rename a Class (not so obvious)

If you rename a class by refactoring, VS2005 does not prompt to rename the file to match the new class name. It's good practice for the file to be named after the class.

Here's the trick. Rename the file in Solution Explorer first and VS2005 will ask to rename the class and update all references.

/// Sunday, November 20

XmlDocument for AST?

I played with Coco/R (lexer/parser generator) last week and was a little disappointed Coco doesn't have a quick way to build an AST like ANTLR. For small utilities, the built-in AST library of ANTLR is sufficient. ANTLR's parser grammar uses a '^' character to indicate the root of the AST node for the production rule. Very simple. To do the same with Coco, you add attributes to each element that becomes part of an AST in your production rules. The grammar file quickly becomes a hybrid of EBNF fules interlaced with C# code. To be fair, ANTLR requires interlacing of C# when creating a grammar of any complexity, but the amount of C# code is much less.

On the other hand, Coco is lightweight and doesn't require a run-time library. The reason for trying it in the first place. Coco is supposedly faster. Parsing does seem faster but I haven't built a complex parser with it.

I went back to ANTLR. Then, this little programmer demon in my head would not succumb to such an easy defeat. I'm now back on Coco for this project. Yep, i'm a flip-flopper. Since .NET doesn't provide a tree collection, I decided to use the XmlDocument for my AST instead of spending a few minutes to create an AST Node. This has many advantages:

I'm not a compiler guy but I see this approach having many advantages (and disadvantages such as performance). Imagine being able to convert from one language to another using XSLT. Or, to simplify the use of AstMacros or NRefactory refactoring? Most of us know how to manipulate XmlDocument and even write XSLT. There is huge potential here.

The idea came from an article I read on the web. Something to do with running multiple languages using common XML ASTs on an XML virtual machine. Interesting read, I wish I saved the link.

/// Friday, November 18

Why not MonoRail?

I almost have the core libraries to start a Boo web framework. Someone in the IRC asked why not build on MonoRail or use Brail? The main reason is the use of NVelocity as the official view engine. Brail emulates NVelocity. They completely missed the point of Ruby on Rails.

NVelocity is a full blown language. Why learn another language? NVelocity isn't object-oriented so you have to do something like this:

<head>
$AjaxHelper2.GetJavascriptFunctions()
$Effects2Helper.GetJavascriptFunctions()
</head>

<body>
Auto completion:<br>
$AjaxHelper2.InputTextWithAutoCompletion( "name", "nameautocompletion.rails", null, null )
</body>

MonoRail pushes two objects into the NVelocity context. These objects provide functions to aid in the creation of AJAX controls. I would prefer to have a controller derived from AjaxController that adds the javascript functions to the header as well as provide additional functions.

NVelocity's purpose is to be a template engine. It interprets a template at runtime to render a view. It might be worth incurring a performance hit if NVelocity provided some advantage. I've shown in a previous article how Boo can be used as a template compiler. In short, the article shows how Boo's compiler attributes let you transform a template into managed code.

How would I approach all of this using Boo? First, using OOP I would derive AjaxController from DefaultController:

class AjaxController(DefaultController):
    // add javascript functions to html
    private def AddJavascriptFunctions():
        pass

    // create an AJAX input text control with auto copmletion
    def InputTextWithAutoCompletion():
        pass

To create an actual controller:

[Template('person.tpl', 'RenderFind')]
class Person(AjaxController):
    def Find():
        RenderFind()

    def Auto():
        // logic to return matches for autocompletion
<!-- person.tpl, compiled as RenderPerson() -->
Auto completion:<br>
$InputTextWithAutoCompletion("name", "person/auto", $Model.Name)

The Template compiler attribute transforms person.tpl into a RenderFind() method. You can add multiple Template attributes. To render a person, call Person.Find(). There is no need to send variables to a template. The template is part of the class and has access to any property, field or method of the class. We're not setting up some context and passing it to an engine like NVelocity does. Person is derived from AjaxController giving this class AJAX functionality by inheritance. As an added bonus, you can debug through the generated code of the template. (Boo does not yet have a #line directive. When Boo does, you'll be able to debug the template directly.)

Are templates the way to go? It's an outdated idea for web frameworks. I may go down the road similiar to XAML, instead I'll use Glade markup. Imagine being able to draw your GUI in Glade and having it run as a web app?

If you want to create an innovative web framework, you have to start with an innovative language and take advantage of its features. That's what ROR did. I think the same can be done using Boo.

/// Thursday, November 17

Mint-Green Tea Theme for Blogger

I became bored of not doing any web stuff so I created a mint-green tea theme for Blogger. Actually, I modified the old orange theme I was using. I'm not that good of a designer but I have a utility called ColorImpact (Windows). It lets you pick a main color then based on different formulas (complements, triad, monochrome, etc) ColorImpact presents a palette of harmonius or complementary colors. I simply used the colors it gave me. I'm a little concerned about not using web safe colors so please let me know if the site looks funky!

I cleaned up the blogger template to maximize use of space, to focus on content. Nobody likes looking at dead space or scrolling just to read an entry. Most developer's use a theme good for general thoughts but not very good for describing and presenting source examples. I even indented the post bodies ala Boo. I don't know if that's functional but it says this site is about coding. I also de-empasized dates and static links on the sidebar. I removed extraneous information like "posted by Mario Gutierrez". Yeah, it's my blog who else posted it? The only thing left to do is to float the links sidebar and have a minimize button to shrink it. I could not make it work in both IE and FireFox. I used to be a die-hard IE fan, not anymore. Get Firefox!

It wasn't just a color update! If the layout is distracting, seems crammed or if you have any other suggestions, please leave a comment. I keep these things in mind when building web pages.

/// Wednesday, November 16

SharpDevelop, NRefactory Doing it Right

If you're like me you probably didn't like SharpDevelop 1.1 too much. It was prone to crash. Since I've been using Boo lately, I gave SharpDevelop 2.0 a look. I build it manually from SVN. What have these guys been drinking? I want some. It's a drastic improvement over the previous version. You still encounter errors here and there but mostly for unimplemented features. It's not ready for prime time as it's a work-in-progress but it's very usable.

Daniel Grunwald is a machine. In addition to being a mad coder on SharpDevelop team, he's created tutorial videos on how to create add-ins and NRefactory. Check out this video on NRefactory. Wow! It's only a matter of time before SharpDevelop 2.0 rivals Eclipse in refactoring. Personally, I'd rather have Eclipse-like refactoring and integrated unit-testing than the new stuff in VS2005. I've also heard good comments from the Boo masters the add-in architecture is well thought out. Maybe SharpDevelop will start a trend like Eclipse did for Java. BIG KUDOS to the SharpDevelop team.

/// Tuesday, November 15

Reflex Syntax Change

After some tinkering with Reflex's grammar, I decided on this (very Groovy-like):

01: #ref "../../MGutz.Test.dll"
02:
03: new Person() {
04:     Name = 'mario'
05:     Weight = 160
06:     Hobbies = new string['scuba', 'biking']
07:     ID = GetUserID(Name)
08:     Vehicle = new Car('Infiniti', 'G35') {
09:         Color = 'Silver'
10:         Horses = 260
11:     }
12:     return this
13: }

In English, just in case it doesn't make sense:

Again, Reflex is not meant to be a full blown scripting language. It's a quick, easy way to create objects and invoke methods and properties using reflection with simple syntax. I plan to use it to map database tables and stored procedure results to objects and to map URLs to classes and methods. And, anywhere else I need to do reflection. (A Boo Web 2.0 Framework is coming).

// example in Boo

name = "mario"
weight = "160"
hobbies = "'scuba', 'biking', 'rugby'"

snippet = """
    #ref '../../MGutz.Test.dll'
    new Person() {
        Name = '${name}'
        Weight = ${weight}
        Hobbies = new string[${hobbies}]
    } 
"""

p = Reflex.Eval(snippet) as Person
// example in C#

string name = "mario";
string weight = "160";
string hobbies = "'scuba', 'biking', 'rugby'";

string template = @"
    #ref '../../MGutz.Test.dll'
    new Person() {
        Name = '{0}'
        Weight = {1}
        Hobbies = new string[{2}]
    }
";

string snippet = string.Format(template, name, weight, hobbies);
Person p = Reflex.Eval(snippet) as Person;

Reflex is white space agnostic (except for #ref pragma). This is equivalent:

#ref '../../MGutz.Test.dll'
new Person() { Name = '{0}' Weight = '{1}' Hobbies = new string[{2}] }

VS 2005 Annoyance

Here's an annoying bug. Look where I'm setting the breakpoint with the mouse. VS2005 isn't even close. This is an intermittent issue. I've always been fond of MS products but i'm not at all impressed with VS2005. VS2003 + Resharper was a much better product. Maybe Resharper spoiled me.

/// Monday, November 14

Reflex Reflection Library

Introduction

Reflex, short for reflection expressions, is a library in the works to simplify the use of the .NET reflection library. Simply put a consutrctor, method and property class in a string and let Reflex do the work:

// instantiate Math using default constructor, then invoke Add(1,2)
string expression = "Math()::Add(1,2)";
int result = (int) Reflex.Eval(expression);

Without Reflex you would have to get the type, find the constructor, invoke the constructor, cast the the returned type, get the method, build the parameters, then invoke it. What if you already have an instance?

Math m = new Math();
rx = new Reflex(m);
expression = "Add(1,2)";
int result = (int) rx.Invoke(expression);

Expressions

[../../some.dll] SomeClass()::Method()

Loads some.dll, instantiate SomeClass using default constructor, invoke Method()

SomeClass::Method()

Invoke static method Method()

Languages["Boo"].Url

Get the "Boo" entry from dictionary Languages, then get the Urlproperty from the object.

Languages.Boo.Url

Same as above, dictionary entries are treated as properties.

Languages[0].Url

Get the first entry from dictionary Languages, then get the Urlproperty

Require Attribute

In some scenarios, Reflex should be prohibited from blindly invoking methods. For example, mapping a URL to classes and methods is a huge security hole. Create an attribute to mark the methods you wish to expose, then use the overloaded Eval() or Invoke():

// throws an exception, Fibonacci is not marked with [Expose]
Reflex.Eval("Math::Fibonacci(3)", typeof(Expose));
class Math {
 [Expose]
 public int Add(int a, int b) {
  return a + b;
 }

 public int Fibonacci(int n) {
 ...
 }
}

Usage

Map a data row to a class.

string template = @"
My.Person()::
    Name = ""{0}""
    Age = {1}
    Height = {2}
";
expression = string.Format(template, row.Name, row.Age, row.Height);
Person p = (Person) Reflex.Eval(expression);

Map url to execute method.

string queryString = "Math/Add?a=1&b=2";
expression = ToExpression(queryString); // convert to "Math()::Add(1, 2)"
int result = (int) rx.Eval(expression);

Instantiate an object from Web.Config setting.

// retrieve from config
string value = "MGutz.Person, Person";
expression = ToExpression(value); // convert to "[MGutz.Person] Person()"
IPerson person = (IPerson) Reflex.Eval(expression);
/// Saturday, November 12

ne paroli Esperanto

i don't speak Esperanto...

I remember chatting with ckknight in the IRC and he brought up the Esperanto language. He typed a few phrases and translated. Wow! I didn't really say that but it was interesting. The moral is I didn't know anything about the language, and with the help of a translator I was able to comprehend the sentence.

Boo has a fantastic extensible compiler. The InjectCode attribute I've been writing about has many possibilities. Yet, I doubt it will get much use outside my projects. If the C# compiler allowed something like InjectCode, there would be some interesting projects around it. C# has partial classes which functionally is what InjectCode does. It builds a partial class in memory at compile time. The important difference is by using InjectCode a source file becomes multi-lingual in the computer language sense. C# only understands C#. InjectCode has the potential to understand any language whether it be in XML, XMI (UML), SQL, etc with the appropriate translator (ICodeBuilder).

No programming language or approach has yet solved the dilemma of how to get every member of a team---DBAs, visual designers, coders, architects, analysts ...---to seamlessly work together. Some, like MDA zealots, believe that creating one visual language is the solution. A picture may be worth a thousand words but it takes a thousand pictures to explain the low-level details of an application. I may be reaching here but I think a multi-lingual language that allows each specialist to speak in their domain language may be a big stepping stone in harmonious development. For example, imagine injecting an analyst's use-case constraints directly into a class or method? Yes, some UML tools do that but it's an external, not integrated process. Good IDEs come close to being a multi-lingual language as they can draw forms, create HTML, etc but they're still well-coordinated individual pieces.

Hmm...maybe it's time to go back to college and do some graduate work.

/// Thursday, November 10

Use XSLT Transformations to Ease Glade#

Summary

This article describes how to use the XslCodeBuilder to simplify cross-platform Gtk# application development.

Introduction

In a previous article, I described the implementation of XslCodeBuilder to inject code generated by an XSLT transformation. In this article we'll apply XslCodeBuilder to ease Gtk# development.

There are at least two ways to build a Gtk# application. The first is to use the Gtk# library as is, manually creating widgets and compose them in a hierarchy to build your GUI. The easier way is to use the Glade# library. Draw your GUI using the Glade 2 utility then save the project creating XML metadata in a .glade file. The .glade file is loaded by the glade-sharp library to render your GUI.

Glade development is summed up in these steps:

  1. Draw your GUI - use the glade utility to generate a .glade file
  2. Declare variables for widgets - each widget in a .glade file has a unique id
  3. Set event handlers - each widget has a set of 'signals' or events
  4. Load the .glade file and run

The rest of the article describes the use of XslCodeBuilder to semi-automate these steps.

XslCodeBuilder and the Glade stylesheet

A .glade file is simply an XML file. It's simple enough to create a stylesheet to generate Boo code to perform the steps described above:

01 <xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
02  <xsl:output method="text" />
03  <xsl:template match="/">
04      <xsl:apply-templates  select="glade-interface/widget[1]"/>
05      def RunGlade():
06              RunGlade(null)
07      end
08
09      def RunGlade(init as callable):
10          Application.Init()
11          gxml = Glade.XML("view.glade", "window1", null)
12          gxml.Autoconnect(self)
13
14          if init != null:
15              init()
16          end
17
18          Application.Run()
19      end
20  </xsl:template>
21
22  <!-- build variables -->
23  <xsl:template match="widget">
24      [Widget]
25      <xsl:value-of select="@id"/> as <xsl:value-of select="substring(@class, 4)"/>
26      <xsl:apply-templates select="child/widget" />
27  </xsl:template>
28 </xsl:stylesheet>

Line 03 is the entry point into the stylesheet. The apply-templates statement starts a recursive descent into the XML file invoking any template that matches the current XML node. In this example, there is a template defined for match="widget". The template emits a declaration like this:

[Widget]
id as widgetClass

After all the widgets are found, processing continues at line 05 which emits overloaded methods of RunGlade(). The XSL-savvy will realize this stylesheet only acts on the first widget from the root and all its children. This is just a sample stylesheet. In all honesty I'm a Glade# NoOB. I need a better understanding of Glade to create a robust stylesheet.

Anyway, here is the source of a basic Glade# application using XslCodeBuilder and the stylesheet:

import Gtk from "gtk-sharp"
import Glade from "glade-sharp"
import MGutz.CodeInjection from "MGutz.CodeInjection"

[InjectCode(XslCodeBuilder, "view.glade;glade.xsl")]
public class GladeView:
    def Run():
        RunGlade()

InjectCode generates this code and stores it in a file so you can step through it in the debugger if needed:

    [Widget]
    window1 as Window
    [Widget]
    vbox1 as VBox
    [Widget]
    toolbar1 as Toolbar
    [Widget]
    openButton as ToolButton
    [Widget]
    quitButton as ToolButton
    [Widget]
    scrolledwindow1 as ScrolledWindow
    [Widget]
    textview1 as TextView

    def RunGlade():
            RunGlade(null)
    end

    def RunGlade(init as callable):
        Application.Init()
        gxml = Glade.XML("view.glade", "window1", null)
        gxml.Autoconnect(self)

        if init != null:
            init()
        end

        Application.Run()
    end

Create and compile a simple controller then run it:

o = GladeView()
o.Run()

If you click on either button nothing will happen. We need to hook events and display a FileChooserDialog when the Open button is clicked.

[InjectCode(XslCodeBuilder, "view.glade;glade.xsl")]
public class GladeView:
    def Run():
        RunGlade(Init)

    def Init():
        quitButton.Clicked += OnQuitButtonClicked
        openButton.Clicked += OnOpenButtonClicked

    def OnQuitButtonClicked():
        Application.Quit()

    def OnOpenButtonClicked():
        fcd = FileChooserDialog(Title: "Select File")
        fcd.AddButton("OK", ResponseType.Accept)
        fcd.AddButton("Cancel", ResponseType.Cancel)
        try:
            response = cast(ResponseType, fcd.Run())
            if (response == ResponseType.Accept):
                file = fcd.Filename
                using reader = StreamReader(file):
                    textview1.Buffer.InsertAtCursor(reader.ReadToEnd())
        ensure:
            fcd.Destroy()

The overloaded RunGlade(callable) is used to set event handlers. The handlers are self-explanatory. Run the controller, then click open

File loaded

Conclusion

I've shown how to use XslCodeBuilder to transform Glade's XML metadata to inject Boo code into a class.

Glade masters are probably thinking this doesn't save much time. I agree somewhat. The point is it does save time. When partial classes are merged into Boo event handlers can be autogenerated into a partial file to survive round-trips between regeneration. I didn't attempt to do that in this example as it's bad kharma to modify the source of the decorated class.

How to Inject Code from XSLT Transformations

Summary

This tutorial describes how to implement an ICodeBuilder class to inject code from the result of an XSLT transformation.

Introduction

Application frameworks and GUI builders create and store metadata in XML format. We can take advantage of this metadata to autogenerate code. We can not only generate code, we can attach metadata to a class by utilizing the MGutz.CodeInjection library. When the metadata changes the booc compiler will update the class during the compilation phase.

The compiler cannot perform this magic on its own. We must create a class derived from AbstractAstAttribute to manipulate the AST nodes the compiler works with. The MGutz.CodeInjection.InjectCodeAttribute does exactly that, it uses 'lower-level' operations to inject code. InjectCodeAttribute is flexible in the sense that it can use any ICodeBuilder object to build a string containing Boo code.

The rest of the tutorial describes the steps to implement XslCodeBuilder, an ICodeBuilder derived object that uses the result of an XSLT transformation to inject code.

Implementing XslCodeBuilder

We have two options to process XML metadata: 1) create a dedicated ICodeBuilder to process specific XML metadata with XmlTextReader and Boo code, or 2) use XSL transformations. The first option is the most flexible as you have the entire framework class library at your disposal. The second option is an all-purpose solution using XSL stylesheets. I'll take the second approach for this article. The end goal is to decorate a class with an attribute that says take this xml file, transform it to Boo code then inject the code into the class during compile time. In other words,

[InjectCode(XslCodeBuilder, "metadata.xml;test.xsl")]
class XslTest:
    pass

To create a code builder implement the MGutz.CodeInjection.ICodeBuilder interface. The source for XslCodeBuilder is fairly simple.

01 public class XslCodeBuilder(ICodeBuilder):
02  TargetParser as BooParserType:
03      get:
04          return BooParserType.WSABooParser
05
06  def BuildCode([required] arg as string, [required] klass as ClassDefinition) as string:
07      // parse xml and xsl file name
08      args = arg.Split(char(';'))
09      if args.Length < 2:
10          raise ArgumentOutOfRangeException("arg", "Expected '<xml-file>;<xsl-file>', received: " + arg)
11      xml = args[0].Trim()
12      xsl = args[1].Trim()
13
14      // files are located in same folder as class
15      xml = Path.Combine(Path.GetDirectoryName(Path.GetFullPath(klass.LexicalInfo.FileName)), xml)
16      xsl = Path.Combine(Path.GetDirectoryName(Path.GetFullPath(klass.LexicalInfo.FileName)), xsl)
17
18      // transform to WSABoo code
19      using reader = StreamReader(xml):
20          xmlDoc = XPathDocument(reader)
21          stylesheet = XslTransform()
22          stylesheet.Load(xsl)
23          writer = StringWriter()
24          stylesheet.Transform(xmlDoc, null, writer, null)
25          return writer.ToString()

At line 02, TargetParser informs InjectCode that XslCodeBuilder is building WSABoo (white space agnostic) code. It's very difficult to control indentation in an XSL file. Don't waste any time making it pretty. This is a shortcoming of languages like Boo that rely on indentation. Fortunately, the Boo masters realized this and provide a WSABooParser class to handle the situation.

At line 16, InjectCode calls BuildCode() to build code. It has two arguments. arg is the argument from the attribute declaration, e.g. "metadata.xml;test.xsl". klass is the AST node of the decorated class.

The rest of the code converts the arguments into absolute paths then performs the XSLT transformation. The transformation result is returned to InjectCode as a string.

Using XslCodeBuilder

This example is found in the example/xsl directory.

First, decorate the class with InjectCode attribute. The example specifies test.xml as the metadata and test.xsl as the stylesheet.

# XslView.boo
import MGutz.CodeInjection from MGutz.CodeInjection

[InjectCode(XslCodeBuilder, "test.xml;test.xsl")]
class XslTest:
    pass
# test.xml - metadata
<def name="KR" comment="white book">
    print "Hello world!"
</def>
# test.xsl - stylesheet to transform metadata into WSABoo code
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output method="text"/>
    <xsl:template match="/">
        <xsl:apply-templates />
    </xsl:template>

    <xsl:template match="def">
        # <xsl:value-of select="@comment"/>
        def <xsl:value-of select="@name"/>():
            <xsl:value-of select="."/>
        end
    </xsl:template>

</xsl:stylesheet>

Create a simple controller to use the XslTest.

o = XslTest()
o.KR()

Compile it all together, then run it. The build file for the example creates an executable xsl.exe:

> xsl
Helo world!

Conclusion

This article described the implementation of XslCodeBuilder, an ICodeBuilder derived class to perform XSLT transformations on XML metadata generating Boo code. The Boo code is injected into a decorated class each time the class is compiled syncing the class with the metadata.

A future article applies XslCodeBuilder to Glade#/Gtk# application development.

/// Wednesday, November 9

Dependency Injection and Scripted Configuration for Enterprise Development - Part 2

Nano and a Configuration Script - Thinking Outside the Box

In this part of the article series, I discuss the external configuration file NanoContainer compiles and invokes to create a managed container of objects. An application retrieves objects from this managed container to perform business logic. These objects are not necessarily instantiated. They may be lazy-loaded for performance and resource considerations.

To get a better idea how Nano integrates with an application, consider this sequence diagram:

  1. Administrator starts an application.
  2. Application asks Nano to create a container.
  3. Nano compiles the configuration script.
  4. Nano sets IScript.Parent to a default container.
  5. Nano asks the script to Compose() the container.
  6. Script returns a container.
  7. Nano returns the container to the application.
  8. Application requests objects from the container and uses them to perform busines logic.

Pretty simple, eh? It's worth reiterating that Nano can use a script written in practically any .NET supported language---Boo, C#, VB, JS, JavaScript, VBScript or even an XML configuration file. Pick your poison. One benefit of using Boo (other than it's more elegant than the other .NET languages) is that a Boo install has an interpreter which may be used to run the script independent of the application! Nano gives a team the full power of any .NET language to configure an application. At first this may not seem to be much of an advantage. Think outside of the box! What if a a configuration file could test itself? What if a configuration file could perform a check on its environment before running an application? What if a configuration file could test for missing property values? What if a configuration file could swap in mock objects? A scripted configuration file's behaviour is only limited by your coding skills. Before we go into the actual configuration file, consider the class diagram for this configuration block.

Class

Purpose

ConfigBase

Is the base class for all configuration files. The class contains methods to check for missing keys, test settings any other future features. It also contains a Settings hash to hold objects derived from SettingBase.

SettingBase

Is the base class for all settings. This class defines an abstract test method that must be implemented.

RequiredSetting

Defines a setting that is required. When Test() is executed it always throws an exception. Must be replaced by another setting.

StringSetting

Defines a string setting. Test() simply returns with no side effect

SafeCreateDirSetting

Safely creates a directory. Test() creates the directory if it does not exist. It's assumed Test() will be called once.

SqlConnectionStringSetting

Defines a SqlConnection string. Test() will attempt to open a connection to the database.

BooConfig

Is the external script file. It does not reside in the ConfigBlock assembly.

The complete source of the configuration file follows:

# BooConfig.boo - compiled by NanoContainer when an application starts.
01 class BooConfig(ConfigBase):
02     # Gets or sets check mode.
03     [property(IsCheck)]
04     _isCheck = false
05
06     # Populates the master settings hash.
07     override def PopulateSettings():
08         Settings = {
09             "name" : StringSetting("mgutz"),
10             "connectionString" : RequiredSetting(),
11             "logDirectory" : SafeCreateDirSetting("/temp/log")
12         }
13
14     # Set the environment for DEV.
15     private def DEV():
16         Settings["connectionString"] = SqlConnectionStringSetting("Server=(local);Database=devapp;uid=boodev;pwd=boodev")
17
18     # Set the envrionment for TEST.
19     private def TEST():
20         pass
21
22     # Set the environment for PROD.
23     private def PROD():
24         Settings["connectionString"] = SqlConnectionStringSetting("Server=(local);Database=prodapp;uid=booprod;pwd=booprod")
25
26     # Composes the object container used by an application.
27     # This method is invoked by an application.
28     override def Compose() as IMutablePicoContainer:
29         _container = DefaultPicoContainer(Parent)
30
31         # change to match environment, could be automated by pinging a unique serverr
32         EnvironmentMethod = DEV
33         ConfigureEnvironment(_isCheck)
34
35         # register an instance of an object
36         _container.RegisterComponentInstance("name", Settings["name"].ToString())
37
38         # lazy-load a connection object by registering its implementation not instance
39         _container.RegisterComponentInstance("dbConnectionString", Settings["connectionString"].ToString())
40         params = array (IParameter, (ComponentParameter("dbConnectionString"),))
41         _container.RegisterComponentImplementation("dbConnection", SqlConnection, params)
42
43         # return the container
44         return _container
45
46 # A check on the environment is performed when this script is called from
47 # booi interpreter. When this script compiled by Nano, this method is not invoked.
48 # These are the same steps taken by the NanoBoo contaienr except the .IsCheck
49 # attribute is not set.
50 def main():
51     config = BooConfig()
52     config.Parent = DefaultPicoContainer()
53     config.IsCheck = true
54     config.Compose()
55
56 main()

The Boo file contains a single class BooConfig derived from ConfigBase. Line 32 must be changed based on which environment the application is deployed. This could be automated by pinging a unique server in each environment. If the server pings back, use the appropriate environment method. Additionally, the file contains a main() method that is executed when the file is run from the command line:

> booi BooConfig.boo

So, what is happening?

First, excuse the sequence diagram. Apparently my UML CASE tool can't handle wierd sequences. The lifeline of the methods should be continous as all of this is actioned in one call to Compose().

  1. Nano, if application is running, or booi, if run from command line, invokes Compose() to create a container.
  2. Script file creates a container to hold objects it builds.
  3. Script file sets the environment method callback.
  4. Script file asks ConfigBase to configure the environment.
  5. ConfigBase in turn asks the script file to populate the Settings hash.
  6. ConfigBase then invokes the environment method, say DEV() to use settings specific to the DEV environment.
  7. If checkEnvironment was true, ConfigBase checks for missing keys by iterating over the hash looking for null or RequiredSetting() objects.
  8. If checkEnvironment was true, ConfigBase iterates over the Settings hash invoking SettingBase.Test() on each object. This tests database connections, missing directories, web service connections, etc.
  9. Flow returns to the script file, where it nows registers implementations and instance of objects storing them into the container previously built. See lines 35-41 where objects are inserted into the container.
  10. The container is returned ending the sequence diagram.

As you can see, a script configuration file can be much more than static key and value pairs. With a little bit of thought, it's very easy to create a configuration block of your own that goes far beyond the Microsoft Enterprise Configuration Block.This article described a simple, yet powerful configuration block that:

The block could easily be extended by creating additional SettingBase-derived classes. Some examples include:

To be continued...

Indeed this is just the tip of the iceberg. I intend to create a Utility class to faciliate use of PicoContainer. It's just too much typing and looks more complicated than it really is. In the next part, I'll cover a few scenarios describing the tangible benefits of NanoContainer in testing and refactoring.

/// Tuesday, November 8

C#, VB to Boo Converter in SharpDevelop

I just did an svn update on SharpDevelop. Daniel Grunwald imported a standalone C#/VB to Boo converter. Awesome! I was about to do this myself. Anyways, it's as simple as this:

    StandaloneConverter -o boofiles d:\temp\someCsProject

This recursively converts any C# or VB source file found under d:\temp\someCsProject into a Boo file into boofiles. The directory structure is maintained. A huge time saver! Wish it had a simpler name though like csvb2boo.

This has been a good week for updates: TestDriven.NET, GhostDoc, StandAloneConverter

Dependency Injection and Scripted Configuration for Enterprise Development - Part 1

Summary

This article describes how script-like configuration files and the Dependency Injection pattern (DI) prevent many of the issues encountered when building an enterprise-level application. Two relatively unknown and powerful open source technologies are used: NanoContainer.NET IoC framework and the agile .NET language Boo.

Application Configuration

An application typically configures itself by retrieving settings from a static configuration file in the form of XML, property files, registry values or some other text format. The setting values are used to instantiate objects, then objects are wired together to perform the functions of the application. If implemented poorly, this infrastructure logic is the source of many bugs and wasted man hours. Even when implemented properly the infrastructure logic is intertwined with business logic deviating from the Separation of Concerns principle.

The dependency injection pattern (DI), in practical terms, is the opposite. DI injects dependencies into an application instead of the application pulling and building its dependencies in code. DI injects pre-wired, ready-to-use objects into an application. The application simply asks for a database connection from a managed container instead of retrieving a connection string then instantiating a database connection object. An application simply uses a pipeline object containing wired pipes and filters without having any knowledge of sequence or how to instantiate pipes and filters. The developer focuses on business logic. Factories, singletons become redundant. Unit testing is easier as mock objects are easily injected without any change to the application.

I use the term DI very loosely. DI is not a design pattern in the sense that it can described within a UML class diagram. DI is a methodology. DI is the fundamental principle behind Inversion of Control (IoC) frameworks. These frameworks not only build and instantiate objects, they provide additional benefits such as interceptors, object pooling, objects lifecycles, etc.

Application Deployment

Most medium to large teams deploy aplications to different server environments as an application matures in the development cycle. Environments have a similar network infrastructure and run the same version of software as the actual production environment. Each environment serves a specific purpose whether it be for integration, testing or staging for actual deployment. Here are some common environments utilized within a development cycle:

PERSONAL

Individual developer box with isolated database, web server, etc. Virtual machine software is an indispensible tool for emulating the servers found in production.

DEV

An application is deployed to DEV when a developer has met all requirement for his task including successful unit tests. Developer's are usually given admin access to servers in this environment to facilitate diagnosis of issues that surface. Preliminary black-box testing is performed against this environment in preparation for formal testing. (Try to run the same tests as QA). Developers must resist the urge to deploy a single web page or assembly. Fully automated installations are a must.

TEST

The quality assurance (QA) team's environment. Developers do not have server access to this environment. Installation, database script issues, trivial bugs should have been caught in the DEV environment. The QA team tests the application at a much higher level and more than likely different than developers ensuring the application meets business requirements. It behooves the development team to build or reuse a logging framework to not only log diagnostic messages but also have a website/tool to view and query the logs.

PROD

The exact same application bits passed in TEST are deployed to PROD. The logging threshold is increased to log info and errors.

What can go wrong? Many things, and it will! Some of the culprits are access permissions, invalid or missing configuration settings, DLL hell (Java and .NET too), different software on the servers, bad network connection, missing keys in the database, human error, etc---so much can go wrong.

To be continued...

In the next article I'll detail how configuration scripts, written in any .NET supported language (Boo, C#, VB, J#, JavaScript and VBScript), and the NanoContainer.NET IoC framework alleviate configuration and deployment issues. I'll cover Boo specifically as it provides additional benefits over C# and VB.NET.

/// Wednesday, November 2

When is a PDB not a PDB?

I have been trying all week to debug the CodeInjection library using the .NET SDK's CLR debugger clrdebug.exe. I was able to step through several examples but not the library itself. I checked for the .PDB file and it was there! WTF is going on? I searched the web for a PDB dumper/lister/viewer, nothing. I had no choice but to break out a hex editor. Voila. I don't know what the heck I was looking at but I noticed the executables I could debug had the source filenames embedded into them, my library did not.

I was building them differently. It turns out booc, the command line version, has Parameters.Debug set to true by default, and booc, the NANT task has it false by default even though it generates a .PDB file. Make sure to set debug to true if you want to use clrdbg.

    <booc output="${build.dir}/MGutz.CodeInjection.dll" target="library" debug="${debug}">
        ...
    </booc>

Here's another tip. If you want the debugger to come up automatically, put this in your code.

    // Windows will prompt to start the debugger (if you have SDK installed)
    System.Diagnostics.Debugger.Break() 
/// Tuesday, November 1

CodeInjection uploaded to boo-contrib

I've uploaded the CodeInjection library to boo-contrib. The SVN adress:

svn+ssh://anonymous@forgesvn1.novell.com/svn/boo-contrib/trunk/mgutz

password: anonymous

You'll have to enter the password several times if you're using TortoiseSVN.

You'll need the SVN version of Boo. I'm using a helper method in CompilerErrorFactory not found in the current binary release.

This page is powered by Blogger. Isn't yours?