Using a codec in a Grails unit test

This is a small issue, but I encountered it and found a solution on the mailing lists, so I thought I’d document it here.

I was demonstrating a trivial Grails application in class today and decided to unit test it. The app has a single controller, called WelcomeController:

class WelcomeController {
  def index = {
    def name = params.name ?: "Grails"
    render "Hello, $name"
  }
}

When I deploy the application and access the Welcome controller (via http://localhost:8080/hellograils/welcome/index), it displays “Hello, Grails!”. If I append “?name=Dolly” to the URL, the result is “Hello, Dolly!”. All nice and simple.

I decided I wanted to write a test case for this, and lately I’ve been learning how to favor unit tests over integration tests as much as possible, mostly for speed. I therefore wrote the following tests:

import grails.test.*

class WelcomeControllerTests extends ControllerUnitTestCase {
  void testWelcomeWithoutParameter() {
    def wc = new WelcomeController()
      wc.index()
      assertEquals "Hello, Grails!", wc.response.contentAsString
    }

  void testWelcomeWithParameter() {
    def wc = new WelcomeController()
    wc.params.name = "Dolly"
    wc.index()
    assertEquals "Hello, Dolly!", wc.response.contentAsString
  }
}

When I run the unit tests (i.e., grails test-app unit:), everything runs correctly.

One of the students pointed out that though this is a trivial example, it’s open to XSS (cross-site scripting) attacks. In the URL, replace “name=Dolly” with “name=alert('dude, you've been hacked')” and the embedded JavaScript code executes and pops up an alert box.

I knew that an easy solution to this would be to modify the index action in the controller to look like:

class WelcomeController {
  def index = {
    def name = params.name ?: "Grails"
    render "Hello, $name".encodeAsHTML()
  }
}

The “encodeAsHTML” method escapes all the HTML, so the output of the hack is just “Hello, alert(…” (i.e., the script is shown as a string, rather than executed) and the problem goes away.

The issue I encountered, though, is that my unit tests started failing, with a missing method exception that claimed that the String class doesn’t have a method called encodeAsHTML. That’s correct, of course, because that method is dynamically injected by Grails based on the org.codehaus.groovy.grails.plugin.codecs.HTMLCodec class. In a unit test, though, the injection doesn’t happen, and I get the exception.

One solution to this, as pointed out on the very valuable grails-users email list, is to add the method to the String class via its metaclass. In other words, in my test, I can add

  void setUp() {
    super.setUp()
    String.metaclass.encodeAsHTML = {
      org.codehaus.groovy.grails.plugins.codecs.HTMLCodec.encode(delegate)
    }
  }

Now the String class has the encodeAsHTML method, and everything works again.

Then I started browsing the Grails API, and found that in ControllerUnitTestCase there’s a method called loadCodec. The GroovyDocs weren’t very informative, but I found in the jira for Grails that issue GRAILS-3816 recommends the addition of the loadCodec method for just this sort of purpose.

That means that I can actually write

  void setUp() {
    super.setUp()
    loadCode(org.codehaus.groovy.grails.plugins.codecs.HTMLCodec)
  }

and everything works as it should. Since this isn’t terribly well documented, I thought I’d say something here. Hopefully this will save somebody some looking around.

Minor bug in Grails selenium plugin

There is a minor bug in the selenium plugin for Grails. It has been discussed on the mailing list, but I thought I would also document it here. I’m using Grails 1.0.3 with version 0.4 of the selenium plugin, which wraps selenium core version 0.8.3.

If you install the plugin and try to run the new Grails tasks (create-selenium-test, create-selenium-domain-test, or run-selenium), the system fails with the error

Could not execute method appContext
No such property: appContext for class: ...

It turns out that the way the appContext variable is handled in Grails has changed, and the plugin hasn’t yet updated to accommodate it.

The recommended fix is to go into the “plugins/selenium-0.4/scripts” directory in your Grails project, and replace ${appContext} with ${grailsAppName} in all the Groovy scripts.

If you use the Selenium IDE in Firefox to write your tests, then the fix doesn’t matter until you try to use the run-selenium target. Still, might as well fix it everywhere. This came up on the mailing list back in July, so I assume an updated plugin will be available as soon as the author gets around to it.

I also find that adding the path to firefox.exe to my system path (default is “c:\Program Files\Mozilla Firefox” — don’t you hate Windows paths with spaces??) makes it easier to run the test suite.

Incidentally, unlike Canoo Web Test, you have to make sure your web app is running before executing the test suite. Selenium fires up the browser and puts it through the proper sequence, so the app has to be running first.

I’ve decided that rather than choosing between Selenium and WebTest, I actually like using both. The Selenium IDE in Firefox is really slick and easy to use, while the output reports in WebTest are gorgeous.

Getting a list of Grails plugins programmatically

In September, I’m very happy to be giving a couple of presentations at the No Fluff, Just Stuff conference in the Boston area.  One of my presentations is a review of the various available Grails plugins.  To prepare for that, I thought I’d create a Grails application that acted as a survey, so people could rate the plugins they like.

One task is to get a list of available Grails plugins.  I wanted to do that programmatically, too, because I’d like to update the list automatically using the Quartz plugin (of course).

How do you get a list of available plugins?  My first thought was to do the HTML equivalent of screen scraping at the main plugin site, http://grails.org/Plugins .  At that site everything is nicely divided into categories, along with links to descriptions and more.

Screen scraping HTML is not fun, though.  I’ve done it before, when necessary, but it’s not very robust and tends to run into problems.  Many of those problems have to do with the fact that HTML is a mess.  Most web sites are filled with HTML that isn’t even well-formed, making processing it programmatically a real pain.

GinA, however, mentioned HTTPUnit as an easy way to access a web page.  Since it’s a regular old Java library, that meant I could use it with Groovy.  Therefore, my first attempt was:

import com.meterware.httpunit.WebConversation

def baseUrl = 'http://grails.org/Plugins'

def wc = new WebConversation()
def resp = wc.getResponse(baseUrl)

Unfortunately, I’m already in trouble even at that point.  If I run that, I get a massive exception stack trace with shows that the included Neko DOM parser choked on the embedded prototype JavaScript library.

While I was debating what to do about that (I really didn’t want to just open the URL, get the text, and start having Fun With Regular Expressions), I noticed a blog posting here, from someone named Isak Rickyanto, from Jakarta, Indonesia.

(A Java developer from Java.  How cool is that?  Or should I say, “how Groovy?” )

Isak points out that there is a list of Grails plugins at http://svn.codehaus.org/grails-plugins/ .  As a Subversion repository listing, it’s not full of JavaScript.  Even better, every plugin is listed as a simple link in an unordered list.

I therefore modified my script to look like this:

def baseUrl = 'http://svn.codehaus.org/grails-plugins/'

def wc = new WebConversation()
def resp = wc.getResponse(baseUrl)
def pluginNames = []
resp.links.each { link ->
    if (link.text =~ /^grails/) {
        def name = link.text - 'grails-' - '/'
        pluginNames << name
    }
}
println pluginNames

Here I’m taking advantage of the fact that the WebResponse class (returned from getResponse(url)) has a method called getLinks().  Since there was one link that had the name “.plugin-meta“, I decided to use a trivial regular expression to filter down to the links definitely associated with plugins.  The WebLink.getText() method then returned the text of the link, with gave values of the form

grails-XXX/

for each plugin.  One of the things I love about Groovy is that I can then just subtract out the characters I don’t want, which is how I added the actual plugin names to an array.

Unfortunately, while that’s part of what I want, that isn’t everything I want.  I’d like the version numbers and the descriptions, too, if possible.  I could go digging into the various directories and look for patterns, but a different idea occurred to me.

I finally remembered that the way I normally find out what plugins are available is to run the

grails list-plugins

command and look at the output.  You’ve probably seen it.  It gives an output like

Welcome to Grails 1.0.3 - http://grails.org/
Licensed under Apache Standard License 2.0
Grails home is set to: c:\grails-1.0.3

Base Directory: c:\
Note: No plugin scripts found
Running script c:\grails-1.0.3\scripts\ListPlugins.groovy
Environment set to development

Plug-ins available in the Grails repository are listed below:
-------------------------------------------------------------

acegi               <0.3>            --  Grails Spring Security 2.0 Plugin
aop                 <no releases>    --  No description available
audit-logging       <0.4>            --  adds hibernate audit logging and onChange event handlers ...
authentication      <1.0>            --  Simple, extensible authentication services with signup ....
autorest            <no releases>    --  No description available

etc.  So if I could get this output, I could break each line into the pieces I want with simple String processing.

How can I do that?  In the spirit of reducing it to a problem already solved, I realized I just wanted to execute that command programmatically and capture the output.  One way to do that is to take advantage of Groovy’s ability to run command line scripts (GinA covers this, of course, but so does Scott Davis’s most excellent Groovy Recipes book).  Here’s the result:

def names = []
def out = "cmd /c grails list-plugins".execute().text
out.split("\n").each { line ->
    if (line =~ /<.*>/) {
        def spaceSplit = line.split()
        def tokenSplit = line.split('--')
        def name = spaceSplit[0]
        def version = spaceSplit[1] - '<' - '>'
        def description = tokenSplit[-1].trim()
        names << name
    }
}

Basically I’m executing the list-plugins command at a command prompt under Windows (sorry, but that’s still my life), splitting the output at the carriage returns (for some odd reason, using eachLine directly kept giving me errors), and processing each line individually.  The lines listing plugins are the ones with version numbers in angle brackets (like <0.3>), and the descriptions came after two dashes.  It seemed easiest to just split the lines both ways in order to get the data I wanted.

I ran this script and the other script together to see if I got the same output.  Here’s the result:

println "From 'grails list-plugins': " + names
println "From svn repo: " + pluginNames
println "Difference: " + (pluginNames - names)

From 'grails list-plugins': ["acegi", "aop", "audit-logging", ..., "yui"]
From svn repo: ["acegi", "aop", "audit-logging", ..., "yui"]
Difference: ["extended-data-binding"]

Why the difference? From the list-plugins output, here’s the line for “extended-data-binding“:

ext-ui              <no releases>    --  No description available
extended-data-binding<0.2>            --  This plugin extends Grails' data binding ...

Yup, the name ran into the version number format.  Sigh. Of course, the other problem with this is that at the moment it’s dependent on my own system configuration (Windows, with the grails command in the path), which can’t be a good thing.

Finally, after all this work, I suddenly realized that I already have the script used to list the plugins.  As with all the other Grails commands, it’s a Gant script in the <GRAILS_HOME>\scripts directory called, obviously enough, ListPlugins.groovy.  According to the documentation at the top, it was written by Sergey Nebolsin for version 0.5.5.

What Sergey does is to go to a slightly different URL and then parse the results as XML.  His script accesses

DEFAULT_PLUGIN_DIST = "http://plugins.grails.org"

instead of the SVN repo location listed above, but if you go there, they look remarkably alike.  I wouldn’t be surprised if http://plugins.grails.org is simply an alias for the SVN repository.

Note that the script also creates a cached version of the plugin list, called plugins-list.xml, which is kept in the

"${userHome}/.grails/${grailsVersion}/plugins/"

directory.  That’s completely understandable, but a lousy location on a Windows box.  I never go to my so-called “user home” directory, so I would never occur to me to look there for information.

His script checks to see if that file is missing or out of date.  If it’s necessary to update it, he opens a URL and starts processing:

def remoteRevision = 0
new URL(DEFAULT_PLUGIN_DIST).withReader { Reader reader ->
    def line = reader.readLine()

...

    // for each plugin directory under Grails Plugins SVN in form of 'grails-*'
    while(line=reader.readLine()) {
        line.eachMatch(/<li><a href="grails-(.+?)">/) {
            // extract plugin name
           def pluginName = it[1][0..-2]

           // collect information about plugin
           buildPluginInfo(pluginsList, pluginName)
        }

etc.

So, in effect, he’s screen scraping the SVN page; he’s just doing a better job of it than I was.

Incidentally, the line in his script that lead to my parsing problems is on line 86:

plugins << "${pluginLine.padRight(20, " ")}${versionLine.padRight(16, " ")} --  ${title}"

I could bump up the padding by one, or learn to parse the output better. I expect the “right” answer, though, is to do what Sergey did, pretty much. Still, if all I have to do is add a little padding, it’s awfully tempting to just “reuse” Sergey’s existing script.

In an upcoming post, I’ll talk about how I used the RichUI plugin to apply a “star rating” to each entry so that people could vote. I don’t have the site ready yet, though. I’ll be sure to mention it when I do.

Silly GORM tricks, part III: SQL keywords as attributes

I was writing a very simple Grails application and ran into a problem when I accidentally used a SQL keyword as a property name. This post documents what happened, and how I (pretty easily) fixed it.

To illustrate the issue, consider a trivial Grails application called “messaging” with a single class called Message.

class Message {
  String from
  String to
  String text
}

This is supposed to represent a simple email message, with fields to represent the sender, the receiver, and the text of the message itself. It seemed quite logical at the time to use the words from, to, and text for the fields, but that leads to problems.

I added a MessageController with dynamic scaffolding (i.e., def scaffold = Message) and started the server. When I accessed the MessageController link, however, I got

org.hibernate.exception.SQLGrammarException: could not execute query

In order to see the actual problem, I modified my DataSource.groovy file to add “loggingSql = true” in the dataSource section. After restarting the server, in the console window I saw that the exception was caused by a SQL command generated by the scaffolded list method in MessageController:

Hibernate: 
    select
        this_.id as id0_0_,
        this_.version as version0_0_,
        this_.from as from0_0_,
        this_.text as text0_0_,
        this_.to as to0_0_ 
    from
        message this_ limit ?

This statement looks fine, and in fact there’s nothing wrong with it. I couldn’t see the problem until I switched to MySQL so that I could browse the database independently.

(Switching to another database is covered many places in the Grails literature. In short, it means adding the JDBC driver to the messaging/lib directory, creating the messaging database in MySQL, and changing the driverClassName, url, username, and password settings in DataSource.groovy.)

When I did that and checked the database with the MySQL client, I found the problem (or at least a symptom of it):

mysql> show tables;
Empty set (0.00 sec)

In other words, the problem was that the message table didn’t exist. Somehow the SQL used to generate the table in the first place didn’t work.

Logging the SQL as I did wasn’t sufficient to show me the CREATE TABLE statement. If, however, I go into Config.groovy and change the value of log4j.logger.org.hibernate to debug, I see in the resulting console:

[1125] hbm2ddl.SchemaExport 
    create table message (
        id bigint not null auto_increment,
        version bigint not null,
        from varchar(255) not null,
        text varchar(255) not null,
        to varchar(255) not null,
        primary key (id)
    )

followed immediately by

[1125] hbm2ddl.SchemaExport Unsuccessful: create table message ...
[1125] hbm2ddl.SchemaExport You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'from varchar(255) not null, text varchar(255) not null, to varchar(255) not null' at line 1


The problem is that the word “from” is a SQL keyword. MySQL got upset when I tried to use it in the create table statement, as shown above.

How can I fix this? I could change the name of the from property, to, say, sender. Since Grails 1.0, however, GORM now has the ability to do custom ORM mappings, which feels like a cleaner way to solve the problem. Therefore, I added the following closure to my class:

static mapping = {
  from column: 'sender'
}

Now the generated create statement is:

[1157] hbm2ddl.SchemaExport 
    create table message (
        id bigint not null auto_increment,
        version bigint not null,
        sender varchar(255) not null,
        text varchar(255) not null,
        to varchar(255) not null,
        primary key (id)
    )

and the new error is

[1157] hbm2ddl.SchemaExport You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'to varchar(255) not null, primary key (id))' at line 1

So apparently the word “to” is also a problem. I therefore modified the mapping closure to include it as well:

static mapping = {
  from column: 'sender'
  to column: 'receiver'
}

Now it all works as it should. The lesson appears to be either that I should keep a list of SQL keywords handy, or simply that the custom ORM DSL is a Good Thing(TM), and so are the logging settings.

Silly GORM tricks, part II: dependent variables

This post discusses a relatively simple topic in GORM: how to use dependent variables in a domain class. It’s simple in the sense that it’s been discussed on the mailing list, but I haven’t seen it documented anywhere so I thought I’d do so here.

I started with a simple two-class domain model that I discussed in my last GORM post.

class Quest {
    String name
    static hasMany = [tasks:Task]
    String toString() { name }
}

class Task {
    String name
    static belongsTo = [quest:Quest]
    String toString() { name }
}

As before, there is a one-to-many relationship between quests and tasks. A quest has many tasks, and the belongsTo setting implies a cascade-all relationship, so inserting, updating, or deleting a quest does the same for all of its associated tasks.

In Bootstrap.groovy, I also have:

def init = { servletContext ->
         new Quest(name:'Seek the grail')
            .addToTasks(name:'Join King Arthur')
            .addToTasks(name:'Defeat Knights Who Say Ni')
            .addToTasks(name:'Fight Killer Rabbit')
            .save()
}

which shows how the classes are intended to work together.

The first change I want to make is to give tasks a start date and end date. My first attempt is to just add properties with those names, of type java.util.Date.

class Task {
  String name
  Date start
  Date end
  // ... rest as before ...
}

This leads to a minor problem. If I start up the server, I don’t see any quests or tasks. The reason is that my bootstrap code tries to create tasks without start and end dates, which violates the database schema restriction. My generated schema marks both start and end columns as “not null”.

There are many ways to fix that. I can either assign both start and end properties for each task in my bootstrap code, or add a constraint in Task that both can be nullable, or do what I did here, which is to give them default values.

class Task {
  String name
  Date start = new Date()
  Date end = new Date() + 1
  // ... rest as before ...
}

I do have a constraint in mind, actually. I’d like to ensure that the end date is after the start date. That requires a custom validator, which is also pretty easy to implement:

class Task {
  // ...
  static constraints = {
    name(blank:false)
    start()
    end(validator: {  value, task ->
       value >= task.start
    })
  }
}

That works fine.

Now for the dependent variable. My tasks all have a start and an end, so implicitly they have a duration. I could add the duration variable to my Task class, but I don’t want to save it in the database. It’s dependent on the values of start and end. I also don’t want to be able to set it from the gui.

Here’s the result:

class Task {
  String name
  Date start
  Date end
  
  int getDuration() { (start..end).size() }
  void setDuration(int value) {}

  static transients = ['duration']

  // ... rest as before ...
}

This computes the duration from the start and end dates by returning the number of days between them. It relies on the fact that Groovy modifies java.util.Date to have the methods next() and previous(), and since Date implements Comparable, it can then be used in a range, as shown.

(As an aside, this implementation is probably pretty inefficient. If the number of days between start and end was substantial, I think this implementation executes the next() method over and over until it reaches the end. I thought about trying to subtract the two dates, but interestingly enough the Date class only has plus() and minus() methods that take int values, not other Dates. I considered adding a category that implemented those methods, but haven’t tried it yet. I’d like to look in the Groovy source code for the plus() and minus() implementations, but I couldn’t find it. I did find something similar in org.codehaus.groovy.runtime.DefaultGroovyMethods, but I’m not sure that’s the same thing. Sigh. Still a lot to learn…)

By putting 'duration' in the transients closure, I ensure that it isn’t saved in the database.

The getDuration method is pretty intuitive, but adding set method as a no-op is somewhat annoying. If I leave it out, then Groovy will generate a setter that can modify the duration. As an alternative, according to GinA I can also supply my own backing field and mark it as final:

class Task {
  // ...
  final int duration

  int getDuration() { (start..end).size() }
  // ...
}

Just to be sure, I added the following test to my TaskTests:

void testSetDuration() {
    Task t = new Task(name:'Join King Arthur')
    shouldFail(ReadOnlyPropertyException) {
        t.duration = 10
    }
   q.addToTasks(t).save()
}

That passed without a problem.

Interestingly, the dynamic scaffold still generates a modifiable input text field for duration, both in the create and edit views. I can put my own value in it and submit the form without a problem. The result does not get saved, which is correct, but I don’t see an exception thrown anywhere in the console. If I generate the static scaffolding, I know that in Task.save there is a line like

t.properties = params

which is how the form parameters are transfered to the object. Presumably the internal logic knows enough to avoid trying to invoke a setter on a final field. Of course, as soon as I generate the static scaffolding, I usually just delete that row in the GSP form.

There’s one final (no pun intended) issue with the dynamic scaffolding. The generated list view puts its properties in <g:sortableColumn> tags. This holds true for the duration, as well. Normally, when I click on the column header, the result is sorted, ascending or descending, by that property. If I click on the duration column header, however, I get an “org.hibernate.QueryException: could not resolve property: duration of: Task“.

It turns out that the User Guide has a “Show Source” link for every tag. When I clicked on that link for the sortableColumn tag, I saw near the top:

if(!attrs.property)
  throwTagError("Tag [sortableColumn] is missing required attribute [property]")

The error I got in the console is “could not resolve property”, but it’s possible this is the source of that issue. I’m not sure. The only other source (again, no pun intended) of the problem I could see was the execution of the list action at the bottom. That would imply that Grails is generating the Hibernate query and we’re failing at that point, which would be consistent with the error reported above.

At any rate, the duration property now works in the domain class. I can always modify the views to ensure I don’t try to set it.

Silly GORM tricks, part I: Lists

In GORM, when one class has a hasMany relationship with another, a java.util.Set is injected into the class. Sometimes, though, I want to use a List instead in order to maintain ordering. The Grails reference documents (see section 5.2.4 specifically) discuss how to do that, but there are other issues that I needed to solve in order to make this work.

Consider an application that demonstrates the issue involved. It has only two domain classes, Quest and Task. A Quest consists of many Tasks.

class Quest {
    String name

    static hasMany = [tasks:Task]

    String toString() { name }
}

class Task {
    String name

    static belongsTo = [quest:Quest]

    String toString() { name }
}

I’m using a bi-directional one-to-many association here, mostly because the dynamic scaffolding works well with it (not the best reason, of course, but it makes it easy to illustrate the point). The hasMany assignment means that the Quest class will have a Set injected into it called tasks, and the belongsTo relationship means that all the cascade relationships (save, update, and delete) will work, too.

Before I take advantage of that in my boostrap code, though, I used the dynamic scaffolding just to make sure I could add quests and tasks through the normal views.

class QuestController { def scaffold = Quest }

class TaskController { def scaffold = Task }

As it happens, everything does work as advertised. A simple integration test that demonstrates it is shown below, which works.

void testAddTasks() {
    Quest q = new Quest(name:'Seek the grail')
    q.addToTasks(name:'Join King Arthur')
        .addToTasks(name:'Defeat Knights Who Say Ni')
        .addToTasks(name:'Fight Killer Rabbit')
        .save()
    assertEquals 3, Task.count()
}

Everything so far is standard stuff. One of the defining characteristics of a Set, however, is that it does not support ordering. If I want ordering, there’s a chrysalis stage I can go through on the way to a List, which is to use a SortedSet (assuming Task implements the Comparable interface).

class Quest {
    String name
    SortedSet tasks

    static hasMany = [tasks:Task]

    String toString() { name }
}

class Task implements Comparable {
    String name

    static belongsTo = [quest:Quest]

    String toString() { name }

    int compareTo(Object o) { 
        return name.compareTo(o.name) 
    }
}

The dynamic scaffolding still works, too. I can add a task, as long as there is a quest available to add it to. The tasks are sorted by name, as they should be. I added the above quest and tasks to my bootstrap code, too, so they were available as soon as my server started.

Incidentally, there’s a down side to using a SortedSet that I hadn’t realized right away. When I first wrote my application, I added a degree of difficulty to my tasks and tried sorting by them.

class Task implements Comparable {
    String name
    Integer difficulty

    // ...

    int compareTo(Object o) { 
        return difficulty - o.difficulty 
    }
}

That sorts tasks by difficulty all right, but there’s another consequence. I can only add a single task of a given difficulty to a particular quest! I can’t have two tasks both with the same difficulty. A SortedSet may be sorted, but it’s still a set.

So now I move on to using a List. As the reference documentation says, to do that, just declare tasks to be a reference of type List.

class Quest {
    String name
    List tasks

    // ...

    static hasMany = [tasks:Task]
}

Now there’s trouble. The server starts, and the bootstrap code works, because it adds tasks to an existing quest before saving them. The dynamic scaffolding has a serious problem, though. When I go to the tasks list and try to add a new task, everything is fine until I try to save the new task.

If I try to add a new task through the “Create Task” page, I get an exception: “org.hibernate.PropertyValueException: not-null property references a null or transient value“.

The reason is addressed in the reference documentation. First, changing to a list means that the database table for tasks now has an index column. Second, as the documentation says, you can’t save a task by itself any more — you have to add it to a quest first. It’s okay to say:

def t = new Task('Answer the bridgekeeper')

but I can’t save it by itself, or that index column will be a problem. I have to add the task to a quest first before saving.

Quest.get(1).addToTasks(t).save()

That works. Otherwise I get a null in that index column, which throws an exception and down goes the server.

So, knowing that, how do I fix the system?

Well, I definitely have to abandon the dynamic scaffolding. The built-in save method isn’t going to work, because it saves the task independently of the quest. So, it’s time to generate the real controllers.

After generating the task controller and views, the save method looks like:

def save = {
    def task = new Task(params)
    if(!task.hasErrors() && task.save()) {
        flash.message = "Task ${task.id} created"
        redirect(action:show,id:task.id)
    }
    else {
        render(view:'create',model:[task:task])
    }
}

I need to add the task to a quest and then save the quest. Fortunately, one of the parameters in the request is the id of the quest, under params.quest.id. That means my first try is to change the above code to this:

def save = {
    def task = new Task(params)
    def q = Quest.get(params.quest.id)
    q.addToTasks(task)
    if (!task.hasErrors() && q.save()) {
        // ... etc ...
}

Unfortunately, this doesn’t work either. When I fill in the page to make a new task and try to save it, I get a NullPointerException due to the fact that the task still has a null id.

This, I believe, turns out to be a Hibernate problem. Hibernate doesn’t do the save when I ask it to, but rather is waiting until the “right moment” to do the commit. Unfortunately, I need that commit right away.

Fortunately, there’s an answer to that, too. The save method takes a flush parameter which can be set to true.

Therefore, I changed the above code to:

def save = {
    def task = new Task(params)
    def q = Quest.get(params.quest.id)
    q.addToTasks(task)
    if (!task.hasErrors() && q.save(flush:true)) {
        // ... etc ...
}

Now, at long last, it all works. The key was to add the task to the quest and save the quest with flush set to true.

It’s possible that there are alternative solutions, but this one worked for me. If you know of better alternatives, please let me know.

Some notes about the Windows installer for Grails

This isn’t really a problem, but I don’t think it’s documented anywhere, so I thought I’d record it here.

(And by the way, if your reaction to my Windows-based comments is going to be “why not use something other than Windows,” my answer is (1) at least one of my machines is always running Windows, but more importantly (2) my clients overwhelmingly use Windows in their training rooms, so I can’t get away from it even if I wanted to.)

Like Groovy, the Grails downloads page includes a special installer for Windows. You don’t have to use it — it’s still fine to just download the zip file, unpack it, set the GRAILS_HOME variable and go on, but if you do decide to use the Windows installer you should know that a few details have changed.

First of all, when the installer runs, it creates a different directory structure than that found in the regular zip file. The installer creates a structure like:

c:\grails-1.0.2\
bin\
gbin\
grails\
… other files ..

which means the GRAILS_HOME variable needs to point to “c:\grails-1.0.2\grails” rather than “c:\grails-1.0.2″ as the User Guide says. That also means that if you want to put Grails in your path for all command prompts, you need the “bin” directory under “grails”, not the one under “grails-1.0.2″. Personally, I assigned GRAILS_HOME to “c:\grails-1.0.2\grails” and put “%GRAILS_HOME%\bin” in my path, just be sure.

This also comes up if you’re using the JetGroovy plug-in for IntelliJ IDEA. The plug-in requires you to specify where the root of the Grails distribution is, which again is the “grails” subdirectory.

In principle, a lot of this isn’t even necessary. The installer creates a desktop shortcut called “Grails Environment” which is a configured command prompt. When I fire it up on my machine and check the path, I see that the directories

c:\jdk1.6.0\bin
C:\GRAILS~1.2\grails\ant\bin
C:\GRAILS~1.2\grails\bin
C:\GRAILS~1.2\GRAILS\..\gbin

have all been prepended (i.e., they appear at the beginning, rather than the end) to my path. The first one is very likely just my “%JAVA_HOME%\bin”, which was already in there, but again that’s not really a problem.

The other interesting characteristic about the “Grails Environment” window is that inside it, you don’t need to type the word “grails” in front of each command. You can just type “create-domain-class myclass” or whatever, and it works automatically. The Grails command line interface is already running.

For the create-* commands to work correctly, of course, you need to be in the root of your Grails application. The environment prompt looks like this:

[C:\grails-1.0.2]
grails:\>

at it starts, which is in the root of the Grails distribution. I needed to change directories to my own application, so after a couple of “cd”s I wound up with

[C:\grails_apps\myapp]
grails:\>

The environment supports all the normal DOS commands, like “dir” or “cd”.

Incidentally, there’s no problem firing up more than one instance of the “Grails Environment”. It creates a separate command window for each case.

There has also been a change to the documentation. In earlier versions, the User Guide (an excellent resource, getting better all the time) was stored in HTML form, and was basically a copy of the documentation found here. Now, using the installer, the documentation has been bundled inside a Windows help file, which has the file extension *.chm.

Finally, the installer adds entries under “Start->All Programs->Grails 1.0.2″. Those entries are:

API Documentation (a link to the included JavaDocs for Grails)
Grails Environment (discussed above)
Grails References (a link to the Windows help file)
Grails Web site (a link to http:\\grails.org)
Uninstall Grails 1.0.2 (which does what it says)

As with most things in Grails, it’s very forgiving. You can use it, or not. If you prefer the older (or should I say, “classic”?) style, just download the zipped distribution and go from there. If you want the installer to do that extra work for you, that’s fine too. I think, though, that including a README file of some kind containing all this information might be helpful.

Still, I tend to like these sorts of installers. I spend more time running training classes than I do on my own systems doing development, so set-up is an ongoing challenge for me. I’m tempted to tell the person setting up a training room to just download and run the Windows installer and they’re done. I may do that for my upcoming Grails class, but I haven’t quite decided yet.

By the way, Groovy also comes with a Windows installer. I really like that one. It creates the same directory structure that the zip file contains, but it also offers to create a GROOVY_HOME variable and add its “bin” directory to either your path or the system path, installs a native environment (whatever that is, but it sounded good), and downloads and installs optional components like Scriptom and the Graphics environment. Using that one is a no-brainer for me, though I’ve found it to be a bit slow.

I hope this helps someone. The creator of the installer is Chanwit Kaewkasi, who is active on the Grails users list. He was kind enough to answer my questions about it there.

Integration tests of controllers in Grails

The documentation on doing integration tests of controllers is a bit thin. I had to ask on the mailing list about how to do some of the basics, so I thought I’d make a record of the results here.

First of all, Grails distinguishes between unit tests and integration tests, in that integration tests involve functionality from the rest of the Grails environment. Running an integration test is rather like using the Grails console. All the dynamic finders are available, as well as any other properties injected by Grails. In a unit test, all of those items would have to be mocked somehow in order to isolate the class under test.

Chapter 6 of DGG discusses testing. The unit testing section is still pretty good, and there’s a section on using Mocks and Stubs which is interesting but feels a bit dated. I’m really looking forward to seeing the next edition of the book. Rumor has it that that’s already in the works.

The cool part is that according to the wiki, when you run “grails test-app”, Grails automatically injects a MockHttpServletRequest, MockHttpServletResponse, and a MockHttpSession from Spring into each integration test. The details of those classes are found in the API documentation for Spring in the org.springframework.mock.web package.

The example given in the wiki page referenced above shows what to do if controller methods end with either a render or a redirect. It turns out that the test case can use the mocked response property to check the resulting value. The sample there looks like

class FooController {
    def text = { render "bar" }

    def someRedirect = { redirect(action:"bar") }
}

so that the test case is

class FooControllerTests extends GroovyTestCase {
    void testText() {
        def fc = new FooController()
        fc.text()
        assertEquals "bar", fc.response.contentAsString
    }

    void testSomeRedirect() {
        def fc = new FooController()
        fc.someRedirect
        assertEquals "/foo/bar", fc.response.redirectedUrl
    }
}

Those are both interesting, of course, and it’s good that they point to the getContentAsString() and getRedirectedUrl() methods in the MockHttpServletResponse classes. My problem is that since both controller methods are invoked without input parameters, neither example shows how to set them.

Here’s the situation I was trying to test, reduced to trivial form. I have a Message class:

class Message {
    String sender
    String msg
}

The generated controller has (among other things) a create action that looks like

class MessageController {
    // ... lots of other actions ...
    def create = {
        def message = new Message()
        message.properties = params
        return ['message':message]
    }
}

So to test the create action, I need to send in some values in the params map.

To make a long story short (too late, I know, but so be it), there are two ways to do it. Here’s the first one:

class MessageControllerTests extends GroovyTestCase {
    void setUp() {
        new Message(sender:'me',msg:'test').save()
    }

    void testCreate() {
        def m = Message.findBySender('me')
        def mc = new MessageController()
        mc.request.parameters = [sender:'me',msg:'test']
        def message = mc.create().message
        assertToString message, "$m"
    }

    void tearDown() {
        Message.findBySender('me').delete()
    }
}

I needed to create a new message in setUp and save it, so that it would be accessible in my test. At the end I disposed of the message in the tearDown method.

In testCreate, I set the request parameters by invoking the setParameters(Map) method of MockHttpServletRequest. That took me a while to realize — I kept trying to set a “params” variable, since that’s what exists in the controller itself. Of course, from the request point of view, the map is a collection of parameters, not params.

The other approach is to deal with the params map directly by replacing the mc.request.parameters line above with

mc.params.sender = 'me'
mc.params.msg = 'test'

That works too and does essentially the same thing. I think I prefer the setParameters version, because I can see exactly what method is being invoked that way. Still, the difference is probably just a question of style.

(My thanks to Chris Chen, ckchris@idream.net, and Burt Beckwith, burt@burtbeckwith.com, for answering my question on the Grails user’s list.)

Finally, back in March of 2007, Glen Smith (of course) wrote up a nice blog post on unit testing controllers rather than doing integration testing. That relies on mocking the request, response, session, etc. with closures. While it demonstrates an interesting usage of metaprogramming to handle all the mocks, it feels like a lot of overhead to put in for each controller. Graeme himself commented on that and suggested developing a withMockController method for GroovyTestCase that did all the overhead for you. I don’t know if that ever actually happened or not, but it looks promising.

Since the whole Grails environment doesn’t have to be bootstrapped, unit testing with those mocks is inevitably going to be faster.

At the moment, I’m quite content to do integration tests, because I’m partly doing the tests to see how the environment works.

I sometimes feel that writing tests is a lot like brushing your teeth. Dentists will tell you that there are better ways and worse ways to brush, but in all honesty, if you just do it at all you’re way ahead of the game. I’m going to try and follow that philosophy and not worry about doing tests perfectly as long as I’m doing them at all.

A few 2GX notes

Late last night I returned home from the Groovy/Grails Experience (2GX) in Reston, VA.  I met many wonderful people and learned tons of new things, which I’m sure will spawn blog posts over the next few weeks.

Just to get started, though, I thought I’d mention a few random observations from the conference.

1. Buy Scott Davis’s Groovy Recipes book!
2. All of the major players I met from the Groovy and Grails projects (Dierk Koenig, Graeme Rocher, Jeff Brown, Jason Rudoph, and many others) were uniformly friendly and encouraging.  Everyone is so easy to talk to and so welcoming of others.  I’m more convinced than ever that Groovy and Grails are going to be huge in the marketplace.
3. Glen Smith, on the other hand, is certifiably insane.  And I mean that in the nicest possible way.   My biggest disappointment at the conference (other than bizarrely forgetting to bring my copy of DGG to get autographed) was that Glenn’s “UI Extreme Makeover” talk was so full I was unable to find a seat.  I had to settle for yet another talk by Scott Davis instead (the horror, the horror).
4. Apparently I’m not the only person having a long-term love affair with GinA.
5. Oh, and buy Groovy Recipes!  It’s now available!
6. Dierk Koenig gave a talk entitled “7 Groovy Usage Patterns for Java Projects.”  In retrospect, that may have been the most important talk I attended.  He showed all sorts of ways to apply Groovy to your projects, and gave each category a clever name.  My favorite was “house elf” scripts, defined as programs that “delegate the housework,” i.e., do the everyday background work for you.  I’m going to start collecting my own Groovy programs into his categories.  I really hope he finds a place to publish that presentation, or some article based on it.
7. Jason Rudolph’s Refactotum presentation (basically a how-to on ways to participate in open source projects) got off to a slow start, but finished very strong.  I’ve never actually contributed to an open source project, but now that I know how, I’m sure I’ll be doing so in the future.  I can write test cases at least, even when I’m otherwise busy.  I’m equally sure I’ll mention something about that here.
8. Don’t forget to buy Groovy Recipes!  Don’t let the fact that Scott included a quote from me in it dissuade you.

I see that I’ve left out almost everything.  I wish I could have attended twice as many talks.  I guess I’ll just have to go back next year, too.

Additional comment about GRAILS_HOME

Yesterday I commented on how I needed to change my GRAILS_HOME variable to point to the grails subdirectory of the grails-1.0 distribution. I did a bit more checking and discovered something interesting.

I’ve been using the Windows installer for Grails. I like how it also installs links for all the JavaDoc documentation, how it includes optional components, etc. It turns out, though, that it’s the Windows installer that’s changed the directory structure.

I just download the zipped binary distribution and expanded it, and it has the original directory structure. With that, I can just set the GRAILS_HOME variable to c:\grails-1.0.1 as usual and everything should be fine.

I’ll be watching to see if the Windows installer changes to go back to the original structure or not. If so, I’ll be sure to mention it here. I probably should say something on the mailing list, too, which I might if I get a chance.