Integration tests of controllers in Grails

March 24, 2008

The documentation on doing integration tests of controllers is a bit thin. I had to ask on the mailing list about how to do some of the basics, so I thought I’d make a record of the results here.

First of all, Grails distinguishes between unit tests and integration tests, in that integration tests involve functionality from the rest of the Grails environment. Running an integration test is rather like using the Grails console. All the dynamic finders are available, as well as any other properties injected by Grails. In a unit test, all of those items would have to be mocked somehow in order to isolate the class under test.

Chapter 6 of DGG discusses testing. The unit testing section is still pretty good, and there’s a section on using Mocks and Stubs which is interesting but feels a bit dated. I’m really looking forward to seeing the next edition of the book. Rumor has it that that’s already in the works.

The cool part is that according to the wiki, when you run “grails test-app”, Grails automatically injects a MockHttpServletRequest, MockHttpServletResponse, and a MockHttpSession from Spring into each integration test. The details of those classes are found in the API documentation for Spring in the org.springframework.mock.web package.

The example given in the wiki page referenced above shows what to do if controller methods end with either a render or a redirect. It turns out that the test case can use the mocked response property to check the resulting value. The sample there looks like


class FooController {
    def text = { render "bar" }

    def someRedirect = { redirect(action:"bar") }
}

so that the test case is


class FooControllerTests extends GroovyTestCase {
    void testText() {
        def fc = new FooController()
        fc.text()
        assertEquals "bar", fc.response.contentAsString
    }

    void testSomeRedirect() {
        def fc = new FooController()
        fc.someRedirect
        assertEquals "/foo/bar", fc.response.redirectedUrl
    }
}

Those are both interesting, of course, and it’s good that they point to the getContentAsString() and getRedirectedUrl() methods in the MockHttpServletResponse classes. My problem is that since both controller methods are invoked without input parameters, neither example shows how to set them.

Here’s the situation I was trying to test, reduced to trivial form. I have a Message class:


class Message {
    String sender
    String msg
}

The generated controller has (among other things) a create action that looks like


class MessageController {
    // ... lots of other actions ...
    def create = {
        def message = new Message()
        message.properties = params
        return ['message':message]
    }
}

So to test the create action, I need to send in some values in the params map.

To make a long story short (too late, I know, but so be it), there are two ways to do it. Here’s the first one:


class MessageControllerTests extends GroovyTestCase {
    void setUp() {
        new Message(sender:'me',msg:'test').save()
    }

    void testCreate() {
        def m = Message.findBySender('me')
        def mc = new MessageController()
        mc.request.parameters = [sender:'me',msg:'test']
        def message = mc.create().message
        assertToString message, "$m"
    }

    void tearDown() {
        Message.findBySender('me').delete()
    }
}

I needed to create a new message in setUp and save it, so that it would be accessible in my test. At the end I disposed of the message in the tearDown method.

In testCreate, I set the request parameters by invoking the setParameters(Map) method of MockHttpServletRequest. That took me a while to realize — I kept trying to set a “params” variable, since that’s what exists in the controller itself. Of course, from the request point of view, the map is a collection of parameters, not params.

The other approach is to deal with the params map directly by replacing the mc.request.parameters line above with


mc.params.sender = 'me'
mc.params.msg = 'test'

That works too and does essentially the same thing. I think I prefer the setParameters version, because I can see exactly what method is being invoked that way. Still, the difference is probably just a question of style.

(My thanks to Chris Chen, ckchris@idream.net, and Burt Beckwith, burt@burtbeckwith.com, for answering my question on the Grails user’s list.)

Finally, back in March of 2007, Glen Smith (of course) wrote up a nice blog post on unit testing controllers rather than doing integration testing. That relies on mocking the request, response, session, etc. with closures. While it demonstrates an interesting usage of metaprogramming to handle all the mocks, it feels like a lot of overhead to put in for each controller. Graeme himself commented on that and suggested developing a withMockController method for GroovyTestCase that did all the overhead for you. I don’t know if that ever actually happened or not, but it looks promising.

Since the whole Grails environment doesn’t have to be bootstrapped, unit testing with those mocks is inevitably going to be faster.

At the moment, I’m quite content to do integration tests, because I’m partly doing the tests to see how the environment works.

I sometimes feel that writing tests is a lot like brushing your teeth. Dentists will tell you that there are better ways and worse ways to brush, but in all honesty, if you just do it at all you’re way ahead of the game. I’m going to try and follow that philosophy and not worry about doing tests perfectly as long as I’m doing them at all.


Using Groovy to determine Unicode characters

March 20, 2008

(Technically speaking, this post doesn’t require Groovy. You could do the same thing in Java. Still, as usual, Groovy is easier.)

I’m teaching a Groovy course this week and having a great time doing it. One of the exercises I put together is to create a concordance, which is a map relating individual words to the lines in a file on which they appear. The program is a variation on a similar one shown in Barclay and Savage’s Groovy Programming book, and is a good illustration of how easy it is to work with maps in Groovy.

A concordance needs to be based on some text somewhere, so I decided to use section 8.00 of the Official Rules of Baseball, which deals with the pitcher.

(And even after reading it again, I couldn’t really explain what a balk really is and what it isn’t, but so be it.)

I copied the text from the web page and pasted it into a text file. Then the exercise code reads the file line by line, breaks each line into words, and then adds them as keys to a map where the values are lists of line numbers. It’s a good example of using eachWithIndex, and map.get(word,[]) + 1, and so on. Once we’ve made each line lower case (so that ‘Pitcher’ and ‘pitcher’ are the same) and coerced the list values to Sets in order to eliminate duplicates, we’re pretty close to a reasonable solution.

The passage is filled with punctuation, however. Fortunately, the tokenize() method in String is overloaded to take a String argument representing the delimiters. Most of the delimiters are obvious and no problem at all (i.e., " .,;:()\'\").

It turns out, however, that the passage also includes sections like:

Pitchers are constantly attempting to “beat the rule” in their efforts to hold runners on bases and in cases where the pitcher fails to make a complete “stop” called for in the rules, the umpire should immediately call a “Balk.”

which are using so-called “smart” quotes. They don’t match the double-quotes in my delimiter string. In other places, there are also possessives which use “smart” apostrophes. How can I add those to my delimiters?

What I needed was the Unicode equivalents for the punctuation. If I know the Unicode values, I can add them as hex values to my delimiters string, like \uXXXX.

After some discussions, I decided to parse the entire passage character by character, and add all non-word characters to a map with their Unicode values. The code looks like this:


def delimiters = [:]
def data = new File('pitcherrules.txt').text
data.each { c ->
    def str = Integer.toHexString(c as int)
    if (!(c =~ /\w/)) {
        delimiters[c] = str
    }
}
println delimiters

It’s pretty straightforward once you know what to look for. Java supplies the Integer.toHexString() method, which takes an int. I read the entire passage into the data variable, then iterated over it, passing each character to the toHexString method. The key was to coerce the character to an int, otherwise I get a MissingMethodException.

I originally had a different expression in the if statement. I was using (c < 'A' || c > 'z') instead. The result included the numbers 0 to 9. By matching against a regular expression consisting of \w, though, I check for all word characters, which is equivalent to [A-Za-z0-9].

The output of the code is

[" ":"20", ":":"3a", ".":"2e", "\r":"d", "\n":"a", ",":"2c",
"(":"28", ")":"29", "’":"2019", "“":"201c", "”":"201d",
"-":"2d", "—":"2014", ";":"3b"]

which tells me that the Unicode values I need are \u2019, \u201c, \u201d, and \u2014.

It’s only a small part of a larger problem, but it’s an easy, useful, interesting script that was probably as much of a learning experience as the original lab. It’s all good. :)

Now the real question is how much of this will actually render properly in this blog post.


Turning Java enums into Groovy ranges

March 19, 2008

It turns out that it’s easy to turn a Java enum into a type that can be used in a Groovy range.

Consider a simple enum representing the seasons:


public enum Season {
    WINTER, SPRING, SUMMER FALL
}

Since enums implement the Comparable interface, they have a compareTo() method. Despite that, however, you can’t use an enum with < or > in Java.


// Won't compile in Java
// if (Season.WINTER > Season.FALL) { ... }

In Groovy, though, any class that implements Comparable can be used with < and >:


// Groovy version
assert Season.FALL > Season.WINTER

That works just fine.

Since the enum already implements Comparable, turning it into a Groovy range simply requires adding in a next() and a previous() method. Here’s one way to do that, relying on the ordinal() method in enum, which returns the index of the enum in the overall collection.


public enum Season {
    WINTER, SPRING, SUMMER, FALL;

    Season next() {
        Season[] vals = Season.values();
        return vals[(this.ordinal() + 1) % vals.length];
    }

    Season previous() {
        Season[] vals = Season.values();
        return vals[(this.ordinal() - 1 + vals.length) % vals.length];
    }
}

Now in Groovy you can write:


(Season.WINTER..Season.FALL).each {
    println it
}

or even


for (s in Season.values()) {
    println (s++)
}

for (s in Season.values()) {
    println (s--)
}

To be honest, I’m not sure where I’m ever going to use that capability, but it’s interesting.

(In Java, the EnumSet.range() method returns a collection that can be used in the Java 5 for-each loop:

for (Season s : EnumSet.range(Season.WINTER, Season.FALL) { … }

but you still can’t use the < or > operators. And the Groovy approach is still simpler.)


Nothing makes you want Groovy more than XML

March 12, 2008

I’m in Delaware this week teaching a course in Java Web Services using RAD7. The materials include a chapter on basic XML parsing using Java. An exercise at the end of the chapter presented the students with a trivial XML file, similar to:


<library>
  <book isbn="1932394842">
    <title>Groovy in Action</title>
    <author>Dierk Koenig</author>
  </book>
  <book isbn="1590597583">
    <title>Definitive Guide to Grails</title>
    <author>Graeme Rocher</author>
  </book>
  <book isbn="0978739299">
    <title>Groovy Recipes</title>
    <author>Scott Davis</author>
  </book>
</library>

(with different books, of course) and asked the students to find a book with a particular isbn number and print it’s title and author values.

I sighed and went to work, producing a solution roughly like this:


import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;

import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;

public class ParseLibrary {
    public static void main(String[] args) {
        DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
        Document doc = null;
        try {
            DocumentBuilder builder = factory.newDocumentBuilder();
            doc = builder.parse("books.xml");
        } catch (Exception e) {
            e.printStackTrace();
            return;
        }
        NodeList books = doc.getElementsByTagName("book");
        for (int i = 0; i < books.getLength(); i++) {
            Element book = (Element) books.item(i);
            if (book.getAttribute("isbn").equals("1932394842")) {
                NodeList children = book.getChildNodes();
                for (int j = 0; j < children.getLength(); j++) {
                    Node child = children.item(j);
                    if (child.getNodeType() == Node.ELEMENT_NODE) {
                        if (child.getNodeName().equals("title")) {
                            System.out.println("Title: "
                                + child.getFirstChild().getNodeValue());
                        } else if (child.getNodeName().equals("author")) {
                            System.out.println("Author: "
                                + child.getFirstChild().getNodeValue());
                        }
                    }
                }
            }
        }
    }
}

The materials didn’t supply a DTD, so I didn’t have any ID attributes to make it easier to get to the book I wanted. That meant I was reduced to continually using getElementsByTagName(String). I certainly didn’t want to traverse the tree, what with all those whitespace nodes containing the carriage-return/line-feed characters. So I found the book nodes, cast them to Element (because only Elements have attributes), found the book I wanted, got all of its children, found the title and author child elements, then grabbed their text values, remembering to go to the element’s first child before doing so.

What an unsightly mess. The only way to simplify it significantly would be to use a 3rd partly library, which the students didn’t have, and it would still be pretty ugly.

One of the students said, “I kept waiting for you to say, ‘this is the hard way, now for the easy way,’ but you never did.”

I couldn’t resist replying, “well, if I had Groovy available, the whole program reduces to:


def library = new XmlSlurper().parse('books.xml')
def book = library.books.find { it.@isbn == '1932394842' }
println "Title: ${book.title}\nAuthor: ${book.author}"

“and I could probably shorted that if I thought about it. How’s that for easy?”

On the bright side, as a result I may have sold another Groovy course. :) For all of Groovy’s advantages over raw Java (and I keep finding more all the time), nothing sells it to Java developers like dealing with XML.


A few 2GX notes

February 24, 2008

Late last night I returned home from the Groovy/Grails Experience (2GX) in Reston, VA.  I met many wonderful people and learned tons of new things, which I’m sure will spawn blog posts over the next few weeks.

Just to get started, though, I thought I’d mention a few random observations from the conference.

  1. Buy Scott Davis’s Groovy Recipes book!
  2. All of the major players I met from the Groovy and Grails projects (Dierk Koenig, Graeme Rocher, Jeff Brown, Jason Rudoph, and many others) were uniformly friendly and encouraging.  Everyone is so easy to talk to and so welcoming of others.  I’m more convinced than ever that Groovy and Grails are going to be huge in the marketplace.
  3. Glen Smith, on the other hand, is certifiably insane.  And I mean that in the nicest possible way. :)  My biggest disappointment at the conference (other than bizarrely forgetting to bring my copy of DGG to get autographed) was that Glenn’s “UI Extreme Makeover” talk was so full I was unable to find a seat.  I had to settle for yet another talk by Scott Davis instead (the horror, the horror).
  4. Apparently I’m not the only person having a long-term love affair with GinA.
  5. Oh, and buy Groovy Recipes!  It’s now available!
  6. Dierk Koenig gave a talk entitled “7 Groovy Usage Patterns for Java Projects.”  In retrospect, that may have been the most important talk I attended.  He showed all sorts of ways to apply Groovy to your projects, and gave each category a clever name.  My favorite was “house elf” scripts, defined as programs that “delegate the housework,” i.e., do the everyday background work for you.  I’m going to start collecting my own Groovy programs into his categories.  I really hope he finds a place to publish that presentation, or some article based on it.
  7. Jason Rudolph’s Refactotum presentation (basically a how-to on ways to participate in open source projects) got off to a slow start, but finished very strong.  I’ve never actually contributed to an open source project, but now that I know how, I’m sure I’ll be doing so in the future.  I can write test cases at least, even when I’m otherwise busy.  I’m equally sure I’ll mention something about that here. :)
  8. Don’t forget to buy Groovy Recipes!  Don’t let the fact that Scott included a quote from me in it dissuade you.

I see that I’ve left out almost everything.  I wish I could have attended twice as many talks.  I guess I’ll just have to go back next year, too.


Additional comment about GRAILS_HOME

February 22, 2008

Yesterday I commented on how I needed to change my GRAILS_HOME variable to point to the grails subdirectory of the grails-1.0 distribution. I did a bit more checking and discovered something interesting.

I’ve been using the Windows installer for Grails. I like how it also installs links for all the JavaDoc documentation, how it includes optional components, etc. It turns out, though, that it’s the Windows installer that’s changed the directory structure.

I just download the zipped binary distribution and expanded it, and it has the original directory structure. With that, I can just set the GRAILS_HOME variable to c:\grails-1.0.1 as usual and everything should be fine.

I’ll be watching to see if the Windows installer changes to go back to the original structure or not. If so, I’ll be sure to mention it here. I probably should say something on the mailing list, too, which I might if I get a chance.


Grails Home moved in 1.0

February 21, 2008

Like approximately 15,000 others, I downloaded and installed Grails 1.0 final in the couple of days after it was released. To my great surprise, however, I couldn’t get it to run properly any more, especially in either Eclipse or IntelliJ IDEA. I couldn’t get either IDE to understand Grails 1.0. I kept trying to set the Grails configuration in IDEA to c:\grails-1.0, and it kept saying that that was an invalid Grails directory. My Eclipse installation messed up the entire build path.

For about a week now, I’ve been assuming my IDE problems were related to plugins that hadn’t yet been updated. I kept checking to see if they were going to update any time soon.

That, as they say, turned out not to be the issue.

This evening I finally realized what happened. Between the last release candidate and the final release of 1.0, the directory structure of the installer changed. Now, the real location of GRAILS_HOME is the grails directory underneath the grails installer. In other words, my GRAILS_HOME variable is

GRAILS_HOME=c:\grails-1.0\grails

whereas before it would have been the grails-1.0 directory itself. Now that I’ve updated that, I’ve been able to get both IDEA and Eclipse to work.

What surprises me is that I can’t find this change documented anywhere. I don’t see it on the Grails wiki, or anywhere in the mailing list archives. Either this is obvious to everyone, or it just hasn’t been recorded anywhere yet.

Thus, this post. Hopefully it’ll help somebody else who might be having the same problem.

(Incidentally, to be strictly correct, I actually set my location to c:\grails-1.0.1\grails. Tonight I updated to the incremental version 1.0.1. :))


“How Groovy Helps” presentation now available

February 6, 2008

Last night I gave a presentation at my local CT Java User’s Group on Groovy.  Rather than present a laundry list of Groovy features, I took a somewhat different approach.  Instead, I selected three relatively small, self-contained problems that I’d worked on over the past few months and showed how using Groovy simplified my work.

I gave the PowerPoint slides to the group moderator so he could add them to the group’s web site, but I decided to share them as well.  Below is a link to my uploaded presentation at SlideShare.net (don’t go to slideshare.com, that’s something completely different).

View kousen's profile on slideshare

I’d embed the presentation here, but I don’t know how to do that, so I’ll have to settle for that button or this direct link.

I tried to upload my source code here, but it’s in zip form and that’s not an acceptable format.  Maybe later I’ll add them to my own web site, but for now know at least that all the code examples are contained in the slides themselves.

I’m sure experienced Groovy developers will find ways to make the included code groovier.  If so, please let me know — that’s how I learn.  A lot of the code has already been posted here at one time or another and I’ve learned a lot from the comments.  Hopefully I haven’t said anything in the slides that’s outright incorrect, though. :)

Enjoy!


Simple Google Chart with Groovy

January 3, 2008

Google has a nice little chart plotting capability at Google Chart API. It’s a web service that generates images based on supplied parameters in a URL string. I think of it as a RESTful web service, despite the fact that the return values aren’t, strictly speaking, XML.

I thought I’d use it to create a pie chart showing the categories of courses I taught in 2007. That meant I had to grab the name of each course from my database, put it in a category, count how many times each category appeared, and then encode the results in a URL I could send to Google. I believe that the right way to solve that problem in the long run is to add a label or category to my Grails application and then search on that, but I haven’t done that yet.

At this point, I thought I’d do the whole thing as a simple Groovy script. I won’t bother discussing here how I retrieved the course titles from the database — that’s pretty straightforward. Putting the courses into categories consisted of snippets of code like:


courses.each { c ->
  if (c.name =~ (/Spring|Hibernate/) ) {
    addToMap('Open Source')
}

where addToMap() is a private function that checks to see if the label is already in the map and either adds it if it isn’t, or increments its value if it is.

For the record, that looks like:


def addToMap(label) {
  if (!map.containsKey(label)) {
    map.put(label,1)
  } else {
    map.put(label, map.get(label) + 1)
  }
}

I assume there’s a Groovier way to do that, but since I didn’t know what it might be, I fell back on straight Java.

The next part is to encode the results the way the Google Chart API expects. I put my courses into some pretty coarse (no pun intended) categories, so my map looks like:

["Open Source":9, "J2EE":12, "Web Services":9, "Ajax":9, "Other":1]

(Some of my courses — Ajax and Java, for example — fell into more than one category.)

Google Chart wants to see the chart labels as a string of values separated by vertical bars. In Groovy, that’s a one-liner:


map.keySet().collect {
  URLEncoder.encode(it,"UTF-8")
}.join('|')

which handles the situation where the label has a space in it, like “Web Services”.

The values are a bit more tricky. Google Chart uses a simple encoding scheme where all the numbers from 0 to 61 turn into the characters A through Z, a through z, and 0 through 9, in that order. The developer’s guide shows a JavaScript example for converting numerical values into those characters.

I decided to do it in Groovy, of course. There’s probably a simpler way, but this worked for me:


def encodeMap() {
  List list = []
  for (i in (('A'..'Z') + ('a'..'z') + (0..9)))  {
    list << i
  }
  String s = "&chd=s:"
  map.each { k,v ->
    s += list[v]
  }
  // ...
}

I initialized the string with “&chd=s:” which tells Google Chart I’m supplying chart data using the simple encoding. Starting from the map values shown above (9, 12, 9, 9, 1), my encoded values were “JMJJB”.

That created the string I needed. I was worried that the values would be retrieved in a different order from the keys returned by keySet(), but I think the map.each function retrieves the keySet first and then uses it to get each value, so it wasn’t a problem.

The rest is simple. I decided to use a 3D pie chart, and made it 500×150 pixels in size to make room for the labels. All Google Chart requests go to “http://chart.apis.google.com/chart”
and query parameters specify the chart type, size, data, and labels. In my case, the parameter string is


cht=p3
&chs=500x150
&chd=s:JMJJB
&chl=Open+Source|J2EE|Web+Services|Ajax|Other

Pasting the complete URL into a browser gives me the image, but what I really want is to make that the source of an HTML image tag. The developer’s guide says to do that, I need to put that URL in the src attribute, but I have to replace all the ampersands with the associated entity reference, &.

That too is easy enough in Groovy.

String urle = url.replaceAll(/&/,’&’)

so the resulting image tag is


<img src='http://chart.apis.google.com/chart?cht=p3
&chs=500x150
&chd=s:JMJJB
&chl=Open+Source|J2EE|Web+Services|Ajax|Other'
alt='Course distribution'/>

all on one line, of course.

The final result is shown below.

That sure was a lot easier in Groovy than it would have been in Java. :)


All I know I learned from GinA (and DGG)

December 19, 2007

Okay, maybe that’s an exaggeration, but I’ve been digging into Groovy in Action (GinA) and the Definitive Guide to Grails (DGG) more lately and keep finding nuggets that I apparently missed on my first few readings.

(I’m trying not to be annoyed about that, btw. By this stage in my career, I know all too well that I have to re-read things over and over to really “get” them. I just wish I retained more than I apparently do. I guess I’ll always think that.)

Anyway, recently I wrote a blog post about how I was able to iterate over Dates because Groovy treated them as a Range (yes, capital R). I thought that was really cool and wasn’t sure exactly why it worked.

Lo and behold, on page 96 of GinA, Listing 4.1 includes


def today = new Date()
def yesterday = today - 1
assert (yesterday..today).size() == 2

Page 98 points out that “any datatype can be used with ranges,” provided it implements next() and previous() and implements Comparable. This is equivalent to overriding ++, –, and the good ol’ spaceship operator, <=>.

(As a long-time Java developer, I have to say that any language that has a spaceship operator is inherently cool. Except Perl, of course. :))

I keep finding ways to rewrite my code as more idiomatic Groovy (or maybe I should just say, groovier). I used to calculate the number of days in a training course from the start and end dates using:


int days = 0
for (date in startDate..endDate) { days++ }

Now I realize that since a Range is an actual object, it’s simpler just to write:


int days = (startDate..endDate).size()

At least when I wrote the first version, I had a feeling I was making it too complicated.

Moving on to Grails, I wanted to have a “days” property on my Course class, but it’s a dependent property. Its value is determined by the existing startDate and endDate properties. That meant I didn’t want to add it as a formal property, because that would imply the existence of a setter method as well as a getter. Also, it doesn’t exist in the database and doesn’t need to.

I couldn’t find an appropriate annotation in DGG, but a quick question on the mailing list cleared it up. My class now looks something like


class Course {
  int id
  Date startDate
  Date endDate
  String title

  // associations, toString() override, and constraints as usual

  transients = ['days']
  int getDays() { return (startDate..endDate).size() }
}

By providing the getter, I don’t need the attribute, which is a normal JavaBeans convention. By putting the property in the transients closure, I’m telling Grails that it doesn’t have a corresponding column in the database.

Unfortunately, I found that when I tried using “days” as a property in in a GSP, sorting on it didn’t work. I think that’s because the tag actually generates a database query with an order by clause, and that’s going to be a problem with a property that doesn’t exist in the database.

I did realize I could compute my total number of training days in a one-liner:


Course.list()*.days.sum()

That uses the wicked cool spread operator, too. In my CourseStatistics class I don’t actually make that list query more than once, but it’s nice to know I could.

I’ve been using Groovy and Grails for about a year now, and it’s nice to know they’re starting to sink in. :)