Silly GORM tricks, part III: SQL keywords as attributes

I was writing a very simple Grails application and ran into a problem when I accidentally used a SQL keyword as a property name. This post documents what happened, and how I (pretty easily) fixed it.

To illustrate the issue, consider a trivial Grails application called “messaging” with a single class called Message.

class Message {
  String from
  String to
  String text
}

This is supposed to represent a simple email message, with fields to represent the sender, the receiver, and the text of the message itself. It seemed quite logical at the time to use the words from, to, and text for the fields, but that leads to problems.

I added a MessageController with dynamic scaffolding (i.e., def scaffold = Message) and started the server. When I accessed the MessageController link, however, I got

org.hibernate.exception.SQLGrammarException: could not execute query

In order to see the actual problem, I modified my DataSource.groovy file to add “loggingSql = true” in the dataSource section. After restarting the server, in the console window I saw that the exception was caused by a SQL command generated by the scaffolded list method in MessageController:

Hibernate:
    select
        this_.id as id0_0_,
        this_.version as version0_0_,
        this_.from as from0_0_,
        this_.text as text0_0_,
        this_.to as to0_0_
    from
        message this_ limit ?

This statement looks fine, and in fact there’s nothing wrong with it. I couldn’t see the problem until I switched to MySQL so that I could browse the database independently.

(Switching to another database is covered many places in the Grails literature. In short, it means adding the JDBC driver to the messaging/lib directory, creating the messaging database in MySQL, and changing the driverClassName, url, username, and password settings in DataSource.groovy.)

When I did that and checked the database with the MySQL client, I found the problem (or at least a symptom of it):

mysql> show tables;
Empty set (0.00 sec)

In other words, the problem was that the message table didn’t exist. Somehow the SQL used to generate the table in the first place didn’t work.

Logging the SQL as I did wasn’t sufficient to show me the CREATE TABLE statement. If, however, I go into Config.groovy and change the value of log4j.logger.org.hibernate to debug, I see in the resulting console:

[1125] hbm2ddl.SchemaExport
    create table message (
        id bigint not null auto_increment,
        version bigint not null,
        from varchar(255) not null,
        text varchar(255) not null,
        to varchar(255) not null,
        primary key (id)
    )

followed immediately by

[1125] hbm2ddl.SchemaExport Unsuccessful: create table message …
[1125] hbm2ddl.SchemaExport You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘from varchar(255) not null, text varchar(255) not null, to varchar(255) not null’ at line 1


The problem is that the word “from” is a SQL keyword. MySQL got upset when I tried to use it in the create table statement, as shown above.

How can I fix this? I could change the name of the from property, to, say, sender. Since Grails 1.0, however, GORM now has the ability to do custom ORM mappings, which feels like a cleaner way to solve the problem. Therefore, I added the following closure to my class:

static mapping = {
  from column: 'sender'
}

Now the generated create statement is:

[1157] hbm2ddl.SchemaExport
    create table message (
        id bigint not null auto_increment,
        version bigint not null,
        sender varchar(255) not null,
        text varchar(255) not null,
        to varchar(255) not null,
        primary key (id)
    )

and the new error is

[1157] hbm2ddl.SchemaExport You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘to varchar(255) not null, primary key (id))’ at line 1

So apparently the word “to” is also a problem. I therefore modified the mapping closure to include it as well:

static mapping = {
  from column: 'sender'
  to column: 'receiver'
}

Now it all works as it should. The lesson appears to be either that I should keep a list of SQL keywords handy, or simply that the custom ORM DSL is a Good Thing(TM), and so are the logging settings.

Silly GORM tricks, part II: dependent variables

This post discusses a relatively simple topic in GORM: how to use dependent variables in a domain class. It’s simple in the sense that it’s been discussed on the mailing list, but I haven’t seen it documented anywhere so I thought I’d do so here.

I started with a simple two-class domain model that I discussed in my last GORM post.

class Quest {
    String name
    static hasMany = [tasks:Task]
    String toString() { name }
}

class Task {
    String name
    static belongsTo = [quest:Quest]
    String toString() { name }
}

As before, there is a one-to-many relationship between quests and tasks. A quest has many tasks, and the belongsTo setting implies a cascade-all relationship, so inserting, updating, or deleting a quest does the same for all of its associated tasks.

In Bootstrap.groovy, I also have:

def init = { servletContext ->
         new Quest(name:'Seek the grail')
            .addToTasks(name:'Join King Arthur')
            .addToTasks(name:'Defeat Knights Who Say Ni')
            .addToTasks(name:'Fight Killer Rabbit')
            .save()
}

which shows how the classes are intended to work together.

The first change I want to make is to give tasks a start date and end date. My first attempt is to just add properties with those names, of type java.util.Date.

class Task {
  String name
  Date start
  Date end
  // ... rest as before ...
}

This leads to a minor problem. If I start up the server, I don’t see any quests or tasks. The reason is that my bootstrap code tries to create tasks without start and end dates, which violates the database schema restriction. My generated schema marks both start and end columns as “not null”.

There are many ways to fix that. I can either assign both start and end properties for each task in my bootstrap code, or add a constraint in Task that both can be nullable, or do what I did here, which is to give them default values.

class Task {
  String name
  Date start = new Date()
  Date end = new Date() + 1
  // ... rest as before ...
}

I do have a constraint in mind, actually. I’d like to ensure that the end date is after the start date. That requires a custom validator, which is also pretty easy to implement:

class Task {
  // ...
  static constraints = {
    name(blank:false)
    start()
    end(validator: {  value, task ->
       value >= task.start
    })
  }
}

That works fine.

Now for the dependent variable. My tasks all have a start and an end, so implicitly they have a duration. I could add the duration variable to my Task class, but I don’t want to save it in the database. It’s dependent on the values of start and end. I also don’t want to be able to set it from the gui.

Here’s the result:

class Task {
  String name
  Date start
  Date end

  int getDuration() { (start..end).size() }
  void setDuration(int value) {}

  static transients = ['duration']

  // … rest as before …
}

This computes the duration from the start and end dates by returning the number of days between them. It relies on the fact that Groovy modifies java.util.Date to have the methods next() and previous(), and since Date implements Comparable, it can then be used in a range, as shown.

(As an aside, this implementation is probably pretty inefficient. If the number of days between start and end was substantial, I think this implementation executes the next() method over and over until it reaches the end. I thought about trying to subtract the two dates, but interestingly enough the Date class only has plus() and minus() methods that take int values, not other Dates. I considered adding a category that implemented those methods, but haven’t tried it yet. I’d like to look in the Groovy source code for the plus() and minus() implementations, but I couldn’t find it. I did find something similar in org.codehaus.groovy.runtime.DefaultGroovyMethods, but I’m not sure that’s the same thing. Sigh. Still a lot to learn…)

By putting 'duration' in the transients closure, I ensure that it isn’t saved in the database.

The getDuration method is pretty intuitive, but adding set method as a no-op is somewhat annoying. If I leave it out, then Groovy will generate a setter that can modify the duration. As an alternative, according to GinA I can also supply my own backing field and mark it as final:

class Task {
  // ...
  final int duration

  int getDuration() { (start..end).size() }
  // ...
}

Just to be sure, I added the following test to my TaskTests:

void testSetDuration() {
    Task t = new Task(name:'Join King Arthur')
    shouldFail(ReadOnlyPropertyException) {
        t.duration = 10
    }
   q.addToTasks(t).save()
}

That passed without a problem.

Interestingly, the dynamic scaffold still generates a modifiable input text field for duration, both in the create and edit views. I can put my own value in it and submit the form without a problem. The result does not get saved, which is correct, but I don’t see an exception thrown anywhere in the console. If I generate the static scaffolding, I know that in Task.save there is a line like

t.properties = params

which is how the form parameters are transfered to the object. Presumably the internal logic knows enough to avoid trying to invoke a setter on a final field. Of course, as soon as I generate the static scaffolding, I usually just delete that row in the GSP form.

There’s one final (no pun intended) issue with the dynamic scaffolding. The generated list view puts its properties in <g:sortableColumn> tags. This holds true for the duration, as well. Normally, when I click on the column header, the result is sorted, ascending or descending, by that property. If I click on the duration column header, however, I get an “org.hibernate.QueryException: could not resolve property: duration of: Task“.

It turns out that the User Guide has a “Show Source” link for every tag. When I clicked on that link for the sortableColumn tag, I saw near the top:

if(!attrs.property)
  throwTagError("Tag [sortableColumn] is missing required attribute [property]“)

The error I got in the console is “could not resolve property”, but it’s possible this is the source of that issue. I’m not sure. The only other source (again, no pun intended) of the problem I could see was the execution of the list action at the bottom. That would imply that Grails is generating the Hibernate query and we’re failing at that point, which would be consistent with the error reported above.

At any rate, the duration property now works in the domain class. I can always modify the views to ensure I don’t try to set it.

Silly GORM tricks, part I: Lists

In GORM, when one class has a hasMany relationship with another, a java.util.Set is injected into the class. Sometimes, though, I want to use a List instead in order to maintain ordering. The Grails reference documents (see section 5.2.4 specifically) discuss how to do that, but there are other issues that I needed to solve in order to make this work.

Consider an application that demonstrates the issue involved. It has only two domain classes, Quest and Task. A Quest consists of many Tasks.

class Quest {
    String name

    static hasMany = [tasks:Task]

    String toString() { name }
}

class Task {
    String name

    static belongsTo = [quest:Quest]

    String toString() { name }
}

I’m using a bi-directional one-to-many association here, mostly because the dynamic scaffolding works well with it (not the best reason, of course, but it makes it easy to illustrate the point). The hasMany assignment means that the Quest class will have a Set injected into it called tasks, and the belongsTo relationship means that all the cascade relationships (save, update, and delete) will work, too.

Before I take advantage of that in my boostrap code, though, I used the dynamic scaffolding just to make sure I could add quests and tasks through the normal views.

class QuestController { def scaffold = Quest }

class TaskController { def scaffold = Task }

As it happens, everything does work as advertised. A simple integration test that demonstrates it is shown below, which works.

void testAddTasks() {
    Quest q = new Quest(name:'Seek the grail')
    q.addToTasks(name:'Join King Arthur')
        .addToTasks(name:'Defeat Knights Who Say Ni')
        .addToTasks(name:'Fight Killer Rabbit')
        .save()
    assertEquals 3, Task.count()
}

Everything so far is standard stuff. One of the defining characteristics of a Set, however, is that it does not support ordering. If I want ordering, there’s a chrysalis stage I can go through on the way to a List, which is to use a SortedSet (assuming Task implements the Comparable interface).

class Quest {
    String name
    SortedSet tasks

    static hasMany = [tasks:Task]

    String toString() { name }
}

class Task implements Comparable {
    String name

    static belongsTo = [quest:Quest]

    String toString() { name }

    int compareTo(Object o) {
        return name.compareTo(o.name)
    }
}

The dynamic scaffolding still works, too. I can add a task, as long as there is a quest available to add it to. The tasks are sorted by name, as they should be. I added the above quest and tasks to my bootstrap code, too, so they were available as soon as my server started.

Incidentally, there’s a down side to using a SortedSet that I hadn’t realized right away. When I first wrote my application, I added a degree of difficulty to my tasks and tried sorting by them.

class Task implements Comparable {
    String name
    Integer difficulty

    // ...

    int compareTo(Object o) {
        return difficulty - o.difficulty
    }
}

That sorts tasks by difficulty all right, but there’s another consequence. I can only add a single task of a given difficulty to a particular quest! I can’t have two tasks both with the same difficulty. A SortedSet may be sorted, but it’s still a set.

So now I move on to using a List. As the reference documentation says, to do that, just declare tasks to be a reference of type List.

class Quest {
    String name
    List tasks

    // ...

    static hasMany = [tasks:Task]
}

Now there’s trouble. The server starts, and the bootstrap code works, because it adds tasks to an existing quest before saving them. The dynamic scaffolding has a serious problem, though. When I go to the tasks list and try to add a new task, everything is fine until I try to save the new task.

If I try to add a new task through the “Create Task” page, I get an exception: “org.hibernate.PropertyValueException: not-null property references a null or transient value“.

The reason is addressed in the reference documentation. First, changing to a list means that the database table for tasks now has an index column. Second, as the documentation says, you can’t save a task by itself any more — you have to add it to a quest first. It’s okay to say:

def t = new Task(’Answer the bridgekeeper’)

but I can’t save it by itself, or that index column will be a problem. I have to add the task to a quest first before saving.

Quest.get(1).addToTasks(t).save()

That works. Otherwise I get a null in that index column, which throws an exception and down goes the server.

So, knowing that, how do I fix the system?

Well, I definitely have to abandon the dynamic scaffolding. The built-in save method isn’t going to work, because it saves the task independently of the quest. So, it’s time to generate the real controllers.

After generating the task controller and views, the save method looks like:

def save = {
    def task = new Task(params)
    if(!task.hasErrors() && task.save()) {
        flash.message = "Task ${task.id} created"
        redirect(action:show,id:task.id)
    }
    else {
        render(view:'create',model:[task:task])
    }
}

I need to add the task to a quest and then save the quest. Fortunately, one of the parameters in the request is the id of the quest, under params.quest.id. That means my first try is to change the above code to this:

def save = {
    def task = new Task(params)
    def q = Quest.get(params.quest.id)
    q.addToTasks(task)
    if (!task.hasErrors() && q.save()) {
        // ... etc ...
}

Unfortunately, this doesn’t work either. When I fill in the page to make a new task and try to save it, I get a NullPointerException due to the fact that the task still has a null id.

This, I believe, turns out to be a Hibernate problem. Hibernate doesn’t do the save when I ask it to, but rather is waiting until the “right moment” to do the commit. Unfortunately, I need that commit right away.

Fortunately, there’s an answer to that, too. The save method takes a flush parameter which can be set to true.

Therefore, I changed the above code to:

def save = {
    def task = new Task(params)
    def q = Quest.get(params.quest.id)
    q.addToTasks(task)
    if (!task.hasErrors() && q.save(flush:true)) {
        // ... etc ...
}

Now, at long last, it all works. The key was to add the task to the quest and save the quest with flush set to true.

It’s possible that there are alternative solutions, but this one worked for me. If you know of better alternatives, please let me know.

assert != assertEquals (duh)

It’s probably not great for my reputation to show how I made a very silly error, but since I did it so consistently I thought showing it might help somebody avoid it.

My Groovy course materials consist of far more scripts than classes. That’s probably not surprising, given that teaching Groovy involves writing lots and lots of quick-and-dirty examples. The problem with scripts, though, is that they’re not as easy to test as classes. With classes, I can create a class that extends GroovyTestCase, put in my tests and go. With scripts, though, the same process would require either executing the script from a Groovy class and checking the binding properties, or simply using assert statements.

In general, I chose the latter. In my scripts, I tried to complete each with at least one assert call that I could use later to validate the script.

Sounds like a reasonable approach, right? Sure, until you start going too quickly. What’s wrong with the following code?

// inject demo
def strings = ["Hello","World"]
int totalLength = strings.inject(0) { len, s ->
    len += s.size()
}
assert 10, totalLength

The code is intended to be a trivial illustration of the inject method for lists. The result is supposed to be the sum total of the lengths of all the strings in the list. Ignoring that there are many other ways to solve that problem, the difficulty here isn’t the inefficiency of the algorithm. No, it’s a trap that as a long-time Java developer I found very easy to fall into.

The problem is with my assert statement. The intention is to assert that the total length of the strings in the list is 10. Sure enough, executing this code has no errors. That’s not a big surprise (for me), because at first I started with a println statement to see what the value should be, and then I replaced println with an assertion.

Unfortunately, though, my assertion didn’t prove anything about the script. That becomes obvious if I add another line to the program:

assert 50000, totalLength

which passes just as easily.

What’s wrong? I’m using assert as though it was a two-parameter method, like assertEquals in GroovyTestCase.

What I’m trying to do is to specify the right answer followed by the actual test. What I’m actually doing is asserting that the first argument (a literal number) is true, and supplying an error message to print if not. By the Groovy truth, any non-zero number is always true, so my totalLength never needs to be converted to a string and printed as the the error message.

What I really want to use, of course, is

assert 10 == total, “total should be 10″

I think this is an easy trap for Java developers to fall into, because they’re not accustomed to the Groovy truth. In Java, only a real boolean expression can be true or false, not just a number.

What’s truly embarrassing is how many of my scripts were just asserting that a non-zero value was true, which always works.

Once I realized my mistake (because somebody pointed out one of my bad examples), I had to go back and fix all of my tests. They’re better now.

Some notes about the Windows installer for Grails

This isn’t really a problem, but I don’t think it’s documented anywhere, so I thought I’d record it here.

(And by the way, if your reaction to my Windows-based comments is going to be “why not use something other than Windows,” my answer is (1) at least one of my machines is always running Windows, but more importantly (2) my clients overwhelmingly use Windows in their training rooms, so I can’t get away from it even if I wanted to.)

Like Groovy, the Grails downloads page includes a special installer for Windows. You don’t have to use it — it’s still fine to just download the zip file, unpack it, set the GRAILS_HOME variable and go on, but if you do decide to use the Windows installer you should know that a few details have changed.

First of all, when the installer runs, it creates a different directory structure than that found in the regular zip file. The installer creates a structure like:

c:\grails-1.0.2\
bin\
gbin\
grails\
… other files ..

which means the GRAILS_HOME variable needs to point to “c:\grails-1.0.2\grails” rather than “c:\grails-1.0.2″ as the User Guide says. That also means that if you want to put Grails in your path for all command prompts, you need the “bin” directory under “grails”, not the one under “grails-1.0.2″. Personally, I assigned GRAILS_HOME to “c:\grails-1.0.2\grails” and put “%GRAILS_HOME%\bin” in my path, just be sure.

This also comes up if you’re using the JetGroovy plug-in for IntelliJ IDEA. The plug-in requires you to specify where the root of the Grails distribution is, which again is the “grails” subdirectory.

In principle, a lot of this isn’t even necessary. The installer creates a desktop shortcut called “Grails Environment” which is a configured command prompt. When I fire it up on my machine and check the path, I see that the directories

c:\jdk1.6.0\bin
C:\GRAILS~1.2\grails\ant\bin
C:\GRAILS~1.2\grails\bin
C:\GRAILS~1.2\GRAILS\..\gbin

have all been prepended (i.e., they appear at the beginning, rather than the end) to my path. The first one is very likely just my “%JAVA_HOME%\bin”, which was already in there, but again that’s not really a problem.

The other interesting characteristic about the “Grails Environment” window is that inside it, you don’t need to type the word “grails” in front of each command. You can just type “create-domain-class myclass” or whatever, and it works automatically. The Grails command line interface is already running.

For the create-* commands to work correctly, of course, you need to be in the root of your Grails application. The environment prompt looks like this:

[C:\grails-1.0.2]
grails:\>

at it starts, which is in the root of the Grails distribution. I needed to change directories to my own application, so after a couple of “cd”s I wound up with

[C:\grails_apps\myapp]
grails:\>

The environment supports all the normal DOS commands, like “dir” or “cd”.

Incidentally, there’s no problem firing up more than one instance of the “Grails Environment”. It creates a separate command window for each case.

There has also been a change to the documentation. In earlier versions, the User Guide (an excellent resource, getting better all the time) was stored in HTML form, and was basically a copy of the documentation found here. Now, using the installer, the documentation has been bundled inside a Windows help file, which has the file extension *.chm.

Finally, the installer adds entries under “Start->All Programs->Grails 1.0.2″. Those entries are:

API Documentation (a link to the included JavaDocs for Grails)
Grails Environment (discussed above)
Grails References (a link to the Windows help file)
Grails Web site (a link to http:\\grails.org)
Uninstall Grails 1.0.2 (which does what it says)

As with most things in Grails, it’s very forgiving. You can use it, or not. If you prefer the older (or should I say, “classic”?) style, just download the zipped distribution and go from there. If you want the installer to do that extra work for you, that’s fine too. I think, though, that including a README file of some kind containing all this information might be helpful.

Still, I tend to like these sorts of installers. I spend more time running training classes than I do on my own systems doing development, so set-up is an ongoing challenge for me. I’m tempted to tell the person setting up a training room to just download and run the Windows installer and they’re done. I may do that for my upcoming Grails class, but I haven’t quite decided yet.

By the way, Groovy also comes with a Windows installer. I really like that one. It creates the same directory structure that the zip file contains, but it also offers to create a GROOVY_HOME variable and add its “bin” directory to either your path or the system path, installs a native environment (whatever that is, but it sounded good), and downloads and installs optional components like Scriptom and the Graphics environment. Using that one is a no-brainer for me, though I’ve found it to be a bit slow.

I hope this helps someone. The creator of the installer is Chanwit Kaewkasi, who is active on the Grails users list. He was kind enough to answer my questions about it there.

Integration tests of controllers in Grails

The documentation on doing integration tests of controllers is a bit thin. I had to ask on the mailing list about how to do some of the basics, so I thought I’d make a record of the results here.

First of all, Grails distinguishes between unit tests and integration tests, in that integration tests involve functionality from the rest of the Grails environment. Running an integration test is rather like using the Grails console. All the dynamic finders are available, as well as any other properties injected by Grails. In a unit test, all of those items would have to be mocked somehow in order to isolate the class under test.

Chapter 6 of DGG discusses testing. The unit testing section is still pretty good, and there’s a section on using Mocks and Stubs which is interesting but feels a bit dated. I’m really looking forward to seeing the next edition of the book. Rumor has it that that’s already in the works.

The cool part is that according to the wiki, when you run “grails test-app”, Grails automatically injects a MockHttpServletRequest, MockHttpServletResponse, and a MockHttpSession from Spring into each integration test. The details of those classes are found in the API documentation for Spring in the org.springframework.mock.web package.

The example given in the wiki page referenced above shows what to do if controller methods end with either a render or a redirect. It turns out that the test case can use the mocked response property to check the resulting value. The sample there looks like

class FooController {
    def text = { render "bar" }

    def someRedirect = { redirect(action:"bar") }
}

so that the test case is

class FooControllerTests extends GroovyTestCase {
    void testText() {
        def fc = new FooController()
        fc.text()
        assertEquals "bar", fc.response.contentAsString
    }

    void testSomeRedirect() {
        def fc = new FooController()
        fc.someRedirect
        assertEquals "/foo/bar", fc.response.redirectedUrl
    }
}

Those are both interesting, of course, and it’s good that they point to the getContentAsString() and getRedirectedUrl() methods in the MockHttpServletResponse classes. My problem is that since both controller methods are invoked without input parameters, neither example shows how to set them.

Here’s the situation I was trying to test, reduced to trivial form. I have a Message class:

class Message {
    String sender
    String msg
}

The generated controller has (among other things) a create action that looks like

class MessageController {
    // ... lots of other actions ...
    def create = {
        def message = new Message()
        message.properties = params
        return ['message':message]
    }
}

So to test the create action, I need to send in some values in the params map.

To make a long story short (too late, I know, but so be it), there are two ways to do it. Here’s the first one:

class MessageControllerTests extends GroovyTestCase {
    void setUp() {
        new Message(sender:'me',msg:'test').save()
    }

    void testCreate() {
        def m = Message.findBySender('me')
        def mc = new MessageController()
        mc.request.parameters = [sender:'me',msg:'test']
        def message = mc.create().message
        assertToString message, “$m”
    }

    void tearDown() {
        Message.findBySender(’me’).delete()
    }
}

I needed to create a new message in setUp and save it, so that it would be accessible in my test. At the end I disposed of the message in the tearDown method.

In testCreate, I set the request parameters by invoking the setParameters(Map) method of MockHttpServletRequest. That took me a while to realize — I kept trying to set a “params” variable, since that’s what exists in the controller itself. Of course, from the request point of view, the map is a collection of parameters, not params.

The other approach is to deal with the params map directly by replacing the mc.request.parameters line above with

mc.params.sender = 'me'
mc.params.msg = 'test'

That works too and does essentially the same thing. I think I prefer the setParameters version, because I can see exactly what method is being invoked that way. Still, the difference is probably just a question of style.

(My thanks to Chris Chen, ckchris@idream.net, and Burt Beckwith, burt@burtbeckwith.com, for answering my question on the Grails user’s list.)

Finally, back in March of 2007, Glen Smith (of course) wrote up a nice blog post on unit testing controllers rather than doing integration testing. That relies on mocking the request, response, session, etc. with closures. While it demonstrates an interesting usage of metaprogramming to handle all the mocks, it feels like a lot of overhead to put in for each controller. Graeme himself commented on that and suggested developing a withMockController method for GroovyTestCase that did all the overhead for you. I don’t know if that ever actually happened or not, but it looks promising.

Since the whole Grails environment doesn’t have to be bootstrapped, unit testing with those mocks is inevitably going to be faster.

At the moment, I’m quite content to do integration tests, because I’m partly doing the tests to see how the environment works.

I sometimes feel that writing tests is a lot like brushing your teeth. Dentists will tell you that there are better ways and worse ways to brush, but in all honesty, if you just do it at all you’re way ahead of the game. I’m going to try and follow that philosophy and not worry about doing tests perfectly as long as I’m doing them at all.

Using Groovy to determine Unicode characters

(Technically speaking, this post doesn’t require Groovy. You could do the same thing in Java. Still, as usual, Groovy is easier.)

I’m teaching a Groovy course this week and having a great time doing it. One of the exercises I put together is to create a concordance, which is a map relating individual words to the lines in a file on which they appear. The program is a variation on a similar one shown in Barclay and Savage’s Groovy Programming book, and is a good illustration of how easy it is to work with maps in Groovy.

A concordance needs to be based on some text somewhere, so I decided to use section 8.00 of the Official Rules of Baseball, which deals with the pitcher.

(And even after reading it again, I couldn’t really explain what a balk really is and what it isn’t, but so be it.)

I copied the text from the web page and pasted it into a text file. Then the exercise code reads the file line by line, breaks each line into words, and then adds them as keys to a map where the values are lists of line numbers. It’s a good example of using eachWithIndex, and map.get(word,[]) + 1, and so on. Once we’ve made each line lower case (so that ‘Pitcher’ and ‘pitcher’ are the same) and coerced the list values to Sets in order to eliminate duplicates, we’re pretty close to a reasonable solution.

The passage is filled with punctuation, however. Fortunately, the tokenize() method in String is overloaded to take a String argument representing the delimiters. Most of the delimiters are obvious and no problem at all (i.e., " .,;:()\'\").

It turns out, however, that the passage also includes sections like:

Pitchers are constantly attempting to “beat the rule” in their efforts to hold runners on bases and in cases where the pitcher fails to make a complete “stop” called for in the rules, the umpire should immediately call a “Balk.”

which are using so-called “smart” quotes. They don’t match the double-quotes in my delimiter string. In other places, there are also possessives which use “smart” apostrophes. How can I add those to my delimiters?

What I needed was the Unicode equivalents for the punctuation. If I know the Unicode values, I can add them as hex values to my delimiters string, like \uXXXX.

After some discussions, I decided to parse the entire passage character by character, and add all non-word characters to a map with their Unicode values. The code looks like this:

def delimiters = [:]
def data = new File(’pitcherrules.txt’).text
data.each { c ->
    def str = Integer.toHexString(c as int)
    if (!(c =~ /\w/)) {
        delimiters[c] = str
    }
}
println delimiters

It’s pretty straightforward once you know what to look for. Java supplies the Integer.toHexString() method, which takes an int. I read the entire passage into the data variable, then iterated over it, passing each character to the toHexString method. The key was to coerce the character to an int, otherwise I get a MissingMethodException.

I originally had a different expression in the if statement. I was using (c < 'A' || c > 'z') instead. The result included the numbers 0 to 9. By matching against a regular expression consisting of \w, though, I check for all word characters, which is equivalent to [A-Za-z0-9].

The output of the code is

[" ":"20", ":":"3a", ".":"2e", "\r":"d", "\n":"a", ",":"2c",
"(":"28", ")":"29", "’":"2019", "“":"201c", "”":"201d",
"-":"2d", "—":"2014", ";":"3b"]
which tells me that the Unicode values I need are \u2019, \u201c, \u201d, and \u2014.

It’s only a small part of a larger problem, but it’s an easy, useful, interesting script that was probably as much of a learning experience as the original lab. It’s all good.

Now the real question is how much of this will actually render properly in this blog post.

Turning Java enums into Groovy ranges

It turns out that it’s easy to turn a Java enum into a type that can be used in a Groovy range.

Consider a simple enum representing the seasons:

public enum Season {
    WINTER, SPRING, SUMMER FALL
}

Since enums implement the Comparable interface, they have a compareTo() method. Despite that, however, you can’t use an enum with < or > in Java.

// Won't compile in Java
// if (Season.WINTER > Season.FALL) { ... }

In Groovy, though, any class that implements Comparable can be used with < and >:

// Groovy version
assert Season.FALL > Season.WINTER

That works just fine.

Since the enum already implements Comparable, turning it into a Groovy range simply requires adding in a next() and a previous() method. Here’s one way to do that, relying on the ordinal() method in enum, which returns the index of the enum in the overall collection.

public enum Season {
    WINTER, SPRING, SUMMER, FALL;

    Season next() {
        Season[] vals = Season.values();
        return vals[(this.ordinal() + 1) % vals.length];
    }

    Season previous() {
        Season[] vals = Season.values();
        return vals[(this.ordinal() - 1 + vals.length) % vals.length];
    }
}

Now in Groovy you can write:

(Season.WINTER..Season.FALL).each {
    println it
}

or even

for (s in Season.values()) {
    println (s++)
}

for (s in Season.values()) {
    println (s--)
}

To be honest, I’m not sure where I’m ever going to use that capability, but it’s interesting.

(In Java, the EnumSet.range() method returns a collection that can be used in the Java 5 for-each loop:

for (Season s : EnumSet.range(Season.WINTER, Season.FALL) { … }

but you still can’t use the < or > operators. And the Groovy approach is still simpler.)

Nothing makes you want Groovy more than XML

I’m in Delaware this week teaching a course in Java Web Services using RAD7. The materials include a chapter on basic XML parsing using Java. An exercise at the end of the chapter presented the students with a trivial XML file, similar to:

<library>
  <book isbn="1932394842">
    <title>Groovy in Action</title>
    <author>Dierk Koenig</author>
  </book>
  <book isbn="1590597583">
    <title>Definitive Guide to Grails</title>
    <author>Graeme Rocher</author>
  </book>
  <book isbn="0978739299">
    <title>Groovy Recipes</title>
    <author>Scott Davis</author>
  </book>
</library>

(with different books, of course) and asked the students to find a book with a particular isbn number and print it’s title and author values.

I sighed and went to work, producing a solution roughly like this:

import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;

import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;

public class ParseLibrary {
    public static void main(String[] args) {
        DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
        Document doc = null;
        try {
            DocumentBuilder builder = factory.newDocumentBuilder();
            doc = builder.parse(”books.xml”);
        } catch (Exception e) {
            e.printStackTrace();
            return;
        }
        NodeList books = doc.getElementsByTagName(”book”);
        for (int i = 0; i < books.getLength(); i++) {
            Element book = (Element) books.item(i);
            if (book.getAttribute(”isbn”).equals(”1932394842″)) {
                NodeList children = book.getChildNodes();
                for (int j = 0; j < children.getLength(); j++) {
                    Node child = children.item(j);
                    if (child.getNodeType() == Node.ELEMENT_NODE) {
                        if (child.getNodeName().equals(”title”)) {
                            System.out.println(”Title: ”
                                + child.getFirstChild().getNodeValue());
                        } else if (child.getNodeName().equals(”author”)) {
                            System.out.println(”Author: ”
                                + child.getFirstChild().getNodeValue());
                        }
                    }
                }
            }
        }
    }
}

The materials didn’t supply a DTD, so I didn’t have any ID attributes to make it easier to get to the book I wanted. That meant I was reduced to continually using getElementsByTagName(String). I certainly didn’t want to traverse the tree, what with all those whitespace nodes containing the carriage-return/line-feed characters. So I found the book nodes, cast them to Element (because only Elements have attributes), found the book I wanted, got all of its children, found the title and author child elements, then grabbed their text values, remembering to go to the element’s first child before doing so.

What an unsightly mess. The only way to simplify it significantly would be to use a 3rd partly library, which the students didn’t have, and it would still be pretty ugly.

One of the students said, “I kept waiting for you to say, ‘this is the hard way, now for the easy way,’ but you never did.”

I couldn’t resist replying, “well, if I had Groovy available, the whole program reduces to:

def library = new XmlSlurper().parse('books.xml')
def book = library.books.find { it.@isbn == '1932394842' }
println "Title: ${book.title}\nAuthor: ${book.author}"

“and I could probably shorted that if I thought about it. How’s that for easy?”

On the bright side, as a result I may have sold another Groovy course. For all of Groovy’s advantages over raw Java (and I keep finding more all the time), nothing sells it to Java developers like dealing with XML.

A few 2GX notes

Late last night I returned home from the Groovy/Grails Experience (2GX) in Reston, VA.  I met many wonderful people and learned tons of new things, which I’m sure will spawn blog posts over the next few weeks.

Just to get started, though, I thought I’d mention a few random observations from the conference.

1. Buy Scott Davis’s Groovy Recipes book!
2. All of the major players I met from the Groovy and Grails projects (Dierk Koenig, Graeme Rocher, Jeff Brown, Jason Rudoph, and many others) were uniformly friendly and encouraging.  Everyone is so easy to talk to and so welcoming of others.  I’m more convinced than ever that Groovy and Grails are going to be huge in the marketplace.
3. Glen Smith, on the other hand, is certifiably insane.  And I mean that in the nicest possible way. :)  My biggest disappointment at the conference (other than bizarrely forgetting to bring my copy of DGG to get autographed) was that Glenn’s “UI Extreme Makeover” talk was so full I was unable to find a seat.  I had to settle for yet another talk by Scott Davis instead (the horror, the horror).
4. Apparently I’m not the only person having a long-term love affair with GinA.
5. Oh, and buy Groovy Recipes!  It’s now available!
6. Dierk Koenig gave a talk entitled “7 Groovy Usage Patterns for Java Projects.”  In retrospect, that may have been the most important talk I attended.  He showed all sorts of ways to apply Groovy to your projects, and gave each category a clever name.  My favorite was “house elf” scripts, defined as programs that “delegate the housework,” i.e., do the everyday background work for you.  I’m going to start collecting my own Groovy programs into his categories.  I really hope he finds a place to publish that presentation, or some article based on it.
7. Jason Rudolph’s Refactotum presentation (basically a how-to on ways to participate in open source projects) got off to a slow start, but finished very strong.  I’ve never actually contributed to an open source project, but now that I know how, I’m sure I’ll be doing so in the future.  I can write test cases at least, even when I’m otherwise busy.  I’m equally sure I’ll mention something about that here.
8. Don’t forget to buy Groovy Recipes!  Don’t let the fact that Scott included a quote from me in it dissuade you.

I see that I’ve left out almost everything.  I wish I could have attended twice as many talks.  I guess I’ll just have to go back next year, too.