Unit testing Grails controllers, revisited

I’ve been neglecting my blog, which I blame on a combination of using twitter and being on a book project.  More about those later.  In the meantime, I’ve recently been working on some Grails projects and found an issue with unit testing controllers.

I’m now on Grails 1.3.5, and I’m trying hard to do unit tests rather than integration tests.  I rapidly hit a couple of issues, which I want to log here mostly so I don’t forget how I resolved them.

Let’s look at a trivial Hello, World type of example.  Say I have a Grails project called helloworld, with a single controller.

package cc.hello
class WelcomeController {
    def index = { 
        log.info "params: $params"

        String name = params.name ?: 'Grails'
        render "Hello, $name!"	
    }
    
    def redirectMethod = { redirect action:"other" }
    
    def other = { render "Made it here" }
}

This is (deliberately) very similar to the example in the Grails reference guide. If I specify a parameter called name, the index action returns “Hello, $name”, otherwise it returns “Hello, Grails!”. I also added a method to do a redirect, because the generated Grails controllers do that a lot and I want to be able to test them.

To keep the story short, here’s my unit test, which I’ll explain afterwards.

package cc.hello

import grails.test.*

class WelcomeControllerTests extends ControllerUnitTestCase {
    WelcomeController controller

    protected void setUp() {
        super.setUp()
        controller = new WelcomeController()
        mockController(WelcomeController)
    }

    void testIndexNoParameters() {
        controller.index()
        assertEquals "Hello, Grails!", controller.response.contentAsString
    }

    void testIndexWithName() {
        controller.params.name = "Dolly"
        //mockParams.name = "Dolly"
        controller.index()
        assertEquals "Hello, Dolly!", controller.response.contentAsString
    }

    void testRedirect() {
        controller.redirectMethod()
        assertEquals "other", controller.redirectArgs['action']
    }
}

The key features of this test are:

• I have to call super.setUp() in my setUp() method, or calling mockController throws a NullPointerException. The actual error is “Cannot invoke containsKey() on a null object”. In other words, the maps for the mock objects aren’t being set up without calling setUp() in the superclass. This cost me a lot of searching to figure out.
• Unlike mocking the domain objects, you can instantiate the controller and then call MockController afterwards. For domain classes you either have to mock the domain first, or call the version of mockDomain or mockForConstraintsTests that takes a list of instances as its second argument.
• The reference documentation tests the redirected method by comparing the generated URL to controller.response.redirectedUrl. That expression always returned null for me, however, and a search of the grails-users list showed it returns null for lots of people. Eventually I found in the excellent Grails in Action book by Peter Ledbrook and Glen Smith that the redirectArgs map has the name of the redirected action under the "action" key, so I can just compare to that. That’s what I’m doing above.

Hopefully these points will help keep someone else from making the same mistakes I did.

Using a codec in a Grails unit test

This is a small issue, but I encountered it and found a solution on the mailing lists, so I thought I’d document it here.

I was demonstrating a trivial Grails application in class today and decided to unit test it. The app has a single controller, called WelcomeController:

class WelcomeController {
  def index = {
    def name = params.name ?: "Grails"
    render "Hello, $name"
  }
}

When I deploy the application and access the Welcome controller (via http://localhost:8080/hellograils/welcome/index), it displays “Hello, Grails!”. If I append “?name=Dolly” to the URL, the result is “Hello, Dolly!”. All nice and simple.

I decided I wanted to write a test case for this, and lately I’ve been learning how to favor unit tests over integration tests as much as possible, mostly for speed. I therefore wrote the following tests:

import grails.test.*

class WelcomeControllerTests extends ControllerUnitTestCase {
  void testWelcomeWithoutParameter() {
    def wc = new WelcomeController()
      wc.index()
      assertEquals "Hello, Grails!", wc.response.contentAsString
    }

  void testWelcomeWithParameter() {
    def wc = new WelcomeController()
    wc.params.name = "Dolly"
    wc.index()
    assertEquals "Hello, Dolly!", wc.response.contentAsString
  }
}

When I run the unit tests (i.e., grails test-app unit:), everything runs correctly.

One of the students pointed out that though this is a trivial example, it’s open to XSS (cross-site scripting) attacks. In the URL, replace “name=Dolly” with “name=alert('dude, you've been hacked')” and the embedded JavaScript code executes and pops up an alert box.

I knew that an easy solution to this would be to modify the index action in the controller to look like:

class WelcomeController {
  def index = {
    def name = params.name ?: "Grails"
    render "Hello, $name".encodeAsHTML()
  }
}

The “encodeAsHTML” method escapes all the HTML, so the output of the hack is just “Hello, alert(…” (i.e., the script is shown as a string, rather than executed) and the problem goes away.

The issue I encountered, though, is that my unit tests started failing, with a missing method exception that claimed that the String class doesn’t have a method called encodeAsHTML. That’s correct, of course, because that method is dynamically injected by Grails based on the org.codehaus.groovy.grails.plugin.codecs.HTMLCodec class. In a unit test, though, the injection doesn’t happen, and I get the exception.

One solution to this, as pointed out on the very valuable grails-users email list, is to add the method to the String class via its metaclass. In other words, in my test, I can add

  void setUp() {
    super.setUp()
    String.metaclass.encodeAsHTML = {
      org.codehaus.groovy.grails.plugins.codecs.HTMLCodec.encode(delegate)
    }
  }

Now the String class has the encodeAsHTML method, and everything works again.

Then I started browsing the Grails API, and found that in ControllerUnitTestCase there’s a method called loadCodec. The GroovyDocs weren’t very informative, but I found in the jira for Grails that issue GRAILS-3816 recommends the addition of the loadCodec method for just this sort of purpose.

That means that I can actually write

  void setUp() {
    super.setUp()
    loadCode(org.codehaus.groovy.grails.plugins.codecs.HTMLCodec)
  }

and everything works as it should. Since this isn’t terribly well documented, I thought I’d say something here. Hopefully this will save somebody some looking around.

SpringOne 2GX final thoughts

I wound up too busy to maintain my daily reports, but here are a couple of items I want to highlight as important take-aways from SpringOne 2GX. Rather than just recap sessions or dole out marketing-type praise, I’m going to focus on some things I learned that I didn’t necessarily expect.

1. A lot of people who advocate Scala or Clojure over Groovy emphasize their scalability and features like immutable objects. Groovy’s @Immutable annotation takes care of the latter, and the gpars project handles the rest. You can use @Immutable right away. The gpars project is still pretty early in its lifecycle, but it’s going to be HUGE.
2. By the way, despite the fact it looks like it’s pronounced “Gee-Pars” (and Paul King kept calling it that), I love the way Scott Davis kept referring to it as “jeepers”
3. Twitter has reached “essential” status at conferences. This is the first conference I attended where I would have missed half of what was going on if I hadn’t been using my Twitter client the whole time (I use twhirl, btw, but I’m open to other possibilities). Most of the presenters (@glaforge, @graemerocher, @paulk_asert, @daveklein, @jeffscottbrown, @scottdavis99, @aalmiray, and several others that would come to mind if I thought harder about it) were continually tweeting good info. As a company, @ManningBooks did an excellent job, especially with their #hideandtweet game.
4. As a totally unexpected (to me) underlying theme, the rise of non-relational databases is striking. Apparently, the major cloud providers (Google AppEngine, Amazon SimpleDB) have decided that relational simply doesn’t scale, so they’re going with “schemaless” solutions. I had no idea how significant that was until I heard enthusiastic support for the idea from the audience of one of the Amazon cloud computing sessions. I know a lot of DBAs who are in for quite a shock. So is Oracle, too, and that’s got to be a Good Thing.
5. Like Grails recently and Ruby on Rails before that, the new Spring Roo project makes existing web development approaches look antiquated. Roo and Grails are siblings that will learn a lot from each other as they continue to grow. For example, Grails has an interactive console, but it isn’t nearly as cool as Roo’s. I’m sure that’ll change soon enough.
6. The extraordinarily humility and friendliness of the Groovy and Grails core teams is charming. Everyone I met seems almost embarrassed to be having so much fun working on something they like so much. There’s none of the arrogance or elitism that characterizes so many other revolutionary groups, and they always go out of their way to help and answer questions. I love talking to them and really hope to be included as one of them some day.
7. Speaking of that, sometimes timing is everything. I told Guillaume Laforge (Groovy team lead, for those who don’t know) that he was one of my personal heroes and nearly made him spit up his drink. Sorry, I didn’t get a picture.
8. Griffon made several fans at the conference, especially among the existing Groovy people. I still think it’s a bit early for mainstream practice, but all the signs are favorable.
9. October is definitely the right time to visit New Orleans.

I had a very good time at the conference and am already looking forward to the next one.

SpringOne 2GX Day 0

I’m at the SpringOne2GX conference (http://www.springone2gx.com) in New Orleans this week. Monday (which they’re calling day 1 but I’m referring to here as day 0) consisted only of registration plus a reception and finally a keynote by Rod Johnson.

As a frequent NFJS attendee, the “reception” was a bit of a culture shock. One of the great appeals of No Fluff, Just Stuff is that there are no vendors present. This reception turned out to be purely a vendor reception, with wine and beer and light munchies. The vendors were all talking about Spring, of course, not Groovy or Grails. I didn’t find any of them particularly overwhelming.

The most amusing part is that one of the Platinum Sponsors was, of all companies, Microsoft. The Microsoft rep was sitting at a table by himself, with no posters or anything. He was sitting in front of a Mac (!).

When I asked him about the Mac, he showed me that it was running VMWare and a Windows 7 pre-release version. Then I asked him about he platinum status, in a rather obnoxious way.

“I can understand why Microsoft might want to be here,” I said, “but Platinum Sponsor? What’s up with that? What’s your goal?”

“We’re talking about our integration story,” he said.

“You mean web services?”

“No,” he said, and then proceeded to describe accessing MS apps via RESTful web services (or maybe just GETful — I didn’t get the details). He also mentioned some MS product I’d never heard of.

When I prodded again about the platinum sponsorship, he confessed that this isn’t really a large conference for MS, so the platinum sponsorship really wasn’t much money to them. Must be nice.

I also asked him why he didn’t have any posters or anything. He said that MS has a major developer conference coming up in a couple of weeks. When he went to get the stuff he needed for this conf, it turned out everything was already packed away.

The other big event of the evening was Rod Johnson’s keynote. That turned out to be a mixed bag, at least from my point of view. He spent the first 20 minutes or so reviewing the glorious history of Spring, focusing on a timeline and all the enthusiasm in the developer community. Now, I’ve been using (and teaching) Spring for years and I really like it, but this felt like a “fire up the troops” talk as though we were marketeers.

He did break for demos of Spring Integration, and tc server with the cool performance monitoring stuff, and a review of what’s new in 3.0. Then he set up the demo by Graeme Rocher, the extremely impressive lead of the Grails project.

To introduce the topic of Grails, Rod made an extremely odd segue. He mentioned how he was able to get “bacon ice cream” in the hotel (a real, of strange product). He somehow related that to a picture of three pigs, and said that pigs build brick houses, and that Grails is built on a brick foundation. That foundation included Spring of course, but he forgot to say anything about Hibernate.

Graeme, wisely IMHO, ignored all that. He showed how the latest version of STS (SpringSource Tool Suite) had very good Grails support. The version he showed will be released Wednesday. Graeme was great, as usual. The only problem I ever have with him is that he tends to make everything look easy, which can be a tad misleading. Still, the support looked solid, and as a community we desperately need that.

The one practical point I took from Rod is that apparently there’s going to be a developer version of tc server. I was really looking forward to checking into tc server, until I found you have to pay for it. I can’t justify that, especially when GlassFish has improved so much over the past few years and JBoss still works, too. Now if there’s a free dev version, maybe I’ll try it.

The biggest message I got from the whole evening, though, is that so far this is a Spring conference, not a Groovy/Grails conference. I hope that doesn’t carry through the whole way.

For me, the best part was finally getting to meet Guillaume Laforge, Paul King, and Robert Fischer in person. I also re-connected with Dave Klein and the indefatigable Andres Almiray. That rocked. The Groovy/Grails/Griffon (!) community is filled with great people.

I’ll try to update this blog periodically as the conference goes on. Any comments, of course, are welcome and appreciated.

Humor lost on Grails Podcast

During the last Grails Podcast, Glen and Sven mentioned how they’d received feedback from me on the previous podcast, part two of their interview with Scott Davis. Instead of reading my feedback verbatim, they made the (possibly wise) decision to just summarize it.

The only problem I have with their decision is that I went to a lot of trouble to make my message as funny as possible, and I even included a comment about Dave Klein in it. All of that was lost. So, as a (possibly unwise) service to my readers, I thought I’d reproduce my message here.

My message is below. I also feel the need to embed some of my own comments, which I’ll include in square brackets [like this].

———— Original message [with annotations] ——————
Hi Glen and Sven,

I just listened to your two-part podcast with Scott Davis, and I feel I have to correct the record on a couple of points. In part two, Glen asked Scott about the O’Reilly book that he and I are working on, entitled “Making Java Groovy“. During that discussion, Scott pronounced my name correctly, which is pretty amazing, but he mistakenly said that I live in the Philadelphia area. Actually, I live in Connecticut, and have for the past 20 years. My sister lives outside of Philly, but I don’t think Scott knows that.

[Missed opportunity for a joke: "Wait a minute... what is Scott Davis doing with my sister?"]

Scott also mentioned that the book has been taking a bit longer than we originally anticipated. Actually, that’s true. One of the reasons is that writing this book has been too much fun. I’m really enjoying working through all the different ways to enhance Java systems with Groovy, ranging from Swing user interfaces (as Scott mentioned) to using Groovy to configure Spring beans to building Groovy handlers for JAX-WS web services. These days I’m hard at work on a chapter called “Groovy in the Cloud”, describing my experiences with Groovy and Java on Google App Engine. I’m also working with a lot of XML-based baseball data, which means that Red Sox games are deductible, right?

[Note the subtle interplay of self-deprecating humor with actual info about the book. Plus, I managed to work in a baseball joke, though it was pretty lame.]

Let’s also get real here. Some of the delays are your fault, Glen. You keep tweeting about all the interesting things you’re learning and doing, and then I feel I have to follow up and discuss them in our book. So quit it.

[I thought the "so quit it" line was maybe the best one of the message, but I probably should have put it in all caps. Also, I meant to say that it's all the news items in the podcast that I find time consuming, since they inevitably cause me to dig into them.]

Finally, you and I both know that producing real quality takes time. It’s hard to write good code, and it’s hard to write good prose, and it’s doubly hard to do both at the same time. So this stuff just takes time. Unless you’re that hoser Dave Klein, who produced his fantastic book “Grails Quickly” so quickly (pun intended) that it’s making us all look bad. He’s a freakin’ loser, and I mean that in the nicest possible way.

[Total back-handed compliment for Dave Klein. Didn't see that one coming, did you? I'm sure he didn't. As it turns out, the actual title of his book is "Grails: A Quick Start Guide" and can be found through the link at PragProg. Why pick on Dave, other than the fact that he did such an awesome job? One good way to sell books is to generate artificial controversy, and who's a better target for that than the Klein-meister? Besides, he's got so many kids at home he'll never have time to read this message. (Oops! Too over the top? YMMV)]

Keep up the good work and see you both at 2GX,

[Ooh, subtle plug for the SpringOne2GX conference, too. Sweet.]

Ken Kousen
————– End of annotated message ————

See? The message had everything, with even a reference to the 2GX conference, so I was quite surprised when Glen and Sven decided to just summarize the newsy parts. Don’t get me wrong — I was half-terrified that they would read it, and if you listen closely you can almost hear both Glen and Sven uncomfortably debating what to do with it. The result, though, probably left most listeners (almost certainly including my poor co-author) wondering what the heck I was on about in the first place.

Well, now you know. While I doubt it would qualify as worthy of a Grails Podcast “Poll of the Week”, I’m curious to see what you think. Did they do the right thing by leaving out all the good stuff (I mean, “humor”)? Am I not nearly as funny as I think I am? (Wait, don’t answer that one.)

Comments are of course welcome. You can also harass (I mean “tweet”) me on Twitter. I’m @kenkousen over there.

Now back to working on the book…

Minor bug in Grails selenium plugin

There is a minor bug in the selenium plugin for Grails. It has been discussed on the mailing list, but I thought I would also document it here. I’m using Grails 1.0.3 with version 0.4 of the selenium plugin, which wraps selenium core version 0.8.3.

If you install the plugin and try to run the new Grails tasks (create-selenium-test, create-selenium-domain-test, or run-selenium), the system fails with the error

Could not execute method appContext
No such property: appContext for class: ...

It turns out that the way the appContext variable is handled in Grails has changed, and the plugin hasn’t yet updated to accommodate it.

The recommended fix is to go into the “plugins/selenium-0.4/scripts” directory in your Grails project, and replace ${appContext} with ${grailsAppName} in all the Groovy scripts.

If you use the Selenium IDE in Firefox to write your tests, then the fix doesn’t matter until you try to use the run-selenium target. Still, might as well fix it everywhere. This came up on the mailing list back in July, so I assume an updated plugin will be available as soon as the author gets around to it.

I also find that adding the path to firefox.exe to my system path (default is “c:\Program Files\Mozilla Firefox” — don’t you hate Windows paths with spaces??) makes it easier to run the test suite.

Incidentally, unlike Canoo Web Test, you have to make sure your web app is running before executing the test suite. Selenium fires up the browser and puts it through the proper sequence, so the app has to be running first.

I’ve decided that rather than choosing between Selenium and WebTest, I actually like using both. The Selenium IDE in Firefox is really slick and easy to use, while the output reports in WebTest are gorgeous.

Star rating in the Grails RichUI plugin

I’ve been working on a Grails application for rating the popularity of Grails plugins.  Rather than just use a simple form with radio buttons or a drop-down list, I thought I’d use the star rating component of the RichUI plugin.

The initial set-up is easy enough.  First, install the RichUI plugin.

grails install-plugin richui

then go into any GSP page where you want to use star rating and add

<resource:rating />

and after that it’s just a question of using the <g:render /> tag appropriately.

Mine looks like this:

<g:each in="${pluginList}" var="plugin">
   ...
   <g:render template="rate"
      model='[plugin: plugin, rating: "${plugin.rating}"]' />

along with some other stuff.

The documentation says to add a RatingController which computes the new rate.  I considered just adding the rating method to my PluginController, but eventually decided to keep it separate.  My RatingController looks like

class RatingController {
    def rate = {
        def rating = params.rating
        def plugin = Plugin.get( params.id )
        def average = (rating.toDouble() + 
            plugin.rating*plugin.totalVotes)/
                (plugin.totalVotes + 1)
        plugin.rating = average
        plugin.totalVotes += 1
        plugin.save()
        session.voted[plugin.name] = true
        render(template: "/plugin/rate", 
            model: [plugin: plugin, rating: average])
    }
}

That’s all pretty much taken from the sample, except for the session.voted[...] business.  I’ll come back to that in a moment.

The template I’m using is called _rate.gsp in the plugin view folder.  It consists of

<div id="plugin${plugin.id}">
    <% def dynamic = !session.voted[plugin.name] %>
    <richui:rating dynamic="${dynamic.toString()}" id="${plugin.id}" units="5"
        rating="${rating}" updateId="plugin${plugin.id}" controller="rating" action="rate"  />
    <p class="static">
        Rating ${java.text.NumberFormat.instance.format(plugin.rating)}
        based on ${plugin.totalVotes} vote<g:if test="${plugin.totalVotes != 1}">s</g:if>
    </p>
    <g:if test="${!dynamic}">
        <div style="color: green;" id="vote${plugin.id}">Thanks for voting!</div>
    </g:if>
</div>

And here we see some good stuff.  First, when you click on the star rating as it comes “out of the box”, it only updates the average value, which it optionally displays.  I wanted to show the number of total votes, too.  As it turns out, there’s an excellent blog post by Jan Sokol that deals with exactly this problem.  See the blog post for details, but essentially it involves changing update ID to that for a div wrapper, which allows you to update a whole section instead.

The <richui:rating> tag has an attribute called dynamic which determines whether you can vote or not.  If dynamic is false, you get a (naturally enough) static view.  If dynamic is true, you can mouseover the stars, highlighting them as you go, and then click to vote.

In most applications I’ve seen that use the star rating, you have to register and login in order to be able to rate anything.  I now believe that’s so they can render the rating tag as dynamic when you enter and then change it to static when you click.  The state is then kept in a user table, which remembers whether you’ve already voted or not.

My problem, though, is that I wanted to let anyone vote without having to register (a decision I’m currently reviewing).  So the question is, how do I keep a person from just voting over and over again?

I tried a couple of ideas, like a toggle or putting something in the page, but if someone browsed to a different page and came back they could vote again.  In the end, I found that I can use the session, even though nobody has logged in!

(That may be obvious to you, but even after teaching server-side Java courses for years now, I guess it never really hit me that a session exists even if you’re not logged in.  Whenever I visited a site with a shopping cart, I always had to log in.  It turns out that was only to buy the products, not to have a session at all. I guess in retrospect it seems obvious, but I never really thought about it until now.)

Anyway, that left me with the question of how to use the session for 90-some different plugins.  I decided to use a boolean array, where the index was the plugin name and the value was true if the person had voted and false otherwise.  As you can see from the RatingController code above, whenever anyone votes, I simply go

session.voted[plugin.name] = true

and I’m all set, because the rating template has

<% def dynamic = !session.voted[plugin.name] %>

and

<richui:rating dynamic="${dynamic.toString()}" ... />

in it.

The only remaining question was how to initialize that array for the session.  I decided that was a classic application of an interceptor, so in my controller for the plugins themselves, I have

def beforeInterceptor = {
    if (!session || !session.voted) {
        def voted = [:]
        def names = Plugin.list().collect { it.name }
        names.each { name ->
            voted[name] = false
        }
        session.voted = voted
   }
}

and there you have it.  Of course, it’s better if you test, and while I still struggle with that, I was able to come up with something effective in this case.  Here’s my PluginControllerTests class, which is, of course, an integration test.  (There’s no doubt a way to make it a unit test instead, but hey, at least it’s tested.)

class PluginControllerTests extends GroovyTestCase {
    def pc

    void setUp() {
        pc = new PluginController()
    }

    void testNoVotedArrayBeforeIntercepting() {
        assertNull pc.session?.voted
    }
   
    void testVotedArrayExistsAfterIntercepting() {
        pc.beforeInterceptor()
        assertNotNull pc.session
        assertNotNull pc.session.voted
    }

    void testVotedArrayHasAcegi() {
        pc.beforeInterceptor()
        assertFalse pc.session.voted['acegi']
   }

   void tearDown() {
        pc = null
    }
}

So it all works, at least so far.  Soon I’ll be able to deploy it, but now I’m still fighting with CSS styles.  Whoever thought doing layout with CSS was a good idea has some explaining to do.

Incidentally, anyone who has used WordPress for blogging knows the frustrations of trying to format code in it. I think there are plugins available to make it easier, but I’m not running WordPress myself — I’m letting them host my blog. The bottom line, therefore, is that sometimes it’s really hard to read code that is posted here. I think, when my app is finished, I’ll follow the recent trend and upload it to github. If I do that, I’ll be sure to mention it here.

Getting a list of Grails plugins programmatically

In September, I’m very happy to be giving a couple of presentations at the No Fluff, Just Stuff conference in the Boston area.  One of my presentations is a review of the various available Grails plugins.  To prepare for that, I thought I’d create a Grails application that acted as a survey, so people could rate the plugins they like.

One task is to get a list of available Grails plugins.  I wanted to do that programmatically, too, because I’d like to update the list automatically using the Quartz plugin (of course).

How do you get a list of available plugins?  My first thought was to do the HTML equivalent of screen scraping at the main plugin site, http://grails.org/Plugins .  At that site everything is nicely divided into categories, along with links to descriptions and more.

Screen scraping HTML is not fun, though.  I’ve done it before, when necessary, but it’s not very robust and tends to run into problems.  Many of those problems have to do with the fact that HTML is a mess.  Most web sites are filled with HTML that isn’t even well-formed, making processing it programmatically a real pain.

GinA, however, mentioned HTTPUnit as an easy way to access a web page.  Since it’s a regular old Java library, that meant I could use it with Groovy.  Therefore, my first attempt was:

import com.meterware.httpunit.WebConversation

def baseUrl = 'http://grails.org/Plugins'

def wc = new WebConversation()
def resp = wc.getResponse(baseUrl)

Unfortunately, I’m already in trouble even at that point.  If I run that, I get a massive exception stack trace with shows that the included Neko DOM parser choked on the embedded prototype JavaScript library.

While I was debating what to do about that (I really didn’t want to just open the URL, get the text, and start having Fun With Regular Expressions), I noticed a blog posting here, from someone named Isak Rickyanto, from Jakarta, Indonesia.

(A Java developer from Java.  How cool is that?  Or should I say, “how Groovy?” )

Isak points out that there is a list of Grails plugins at http://svn.codehaus.org/grails-plugins/ .  As a Subversion repository listing, it’s not full of JavaScript.  Even better, every plugin is listed as a simple link in an unordered list.

I therefore modified my script to look like this:

def baseUrl = 'http://svn.codehaus.org/grails-plugins/'

def wc = new WebConversation()
def resp = wc.getResponse(baseUrl)
def pluginNames = []
resp.links.each { link ->
    if (link.text =~ /^grails/) {
        def name = link.text - 'grails-' - '/'
        pluginNames << name
    }
}
println pluginNames

Here I’m taking advantage of the fact that the WebResponse class (returned from getResponse(url)) has a method called getLinks().  Since there was one link that had the name “.plugin-meta“, I decided to use a trivial regular expression to filter down to the links definitely associated with plugins.  The WebLink.getText() method then returned the text of the link, with gave values of the form

grails-XXX/

for each plugin.  One of the things I love about Groovy is that I can then just subtract out the characters I don’t want, which is how I added the actual plugin names to an array.

Unfortunately, while that’s part of what I want, that isn’t everything I want.  I’d like the version numbers and the descriptions, too, if possible.  I could go digging into the various directories and look for patterns, but a different idea occurred to me.

I finally remembered that the way I normally find out what plugins are available is to run the

grails list-plugins

command and look at the output.  You’ve probably seen it.  It gives an output like

Welcome to Grails 1.0.3 - http://grails.org/
Licensed under Apache Standard License 2.0
Grails home is set to: c:\grails-1.0.3

Base Directory: c:\
Note: No plugin scripts found
Running script c:\grails-1.0.3\scripts\ListPlugins.groovy
Environment set to development

Plug-ins available in the Grails repository are listed below:
-------------------------------------------------------------

acegi               <0.3>            --  Grails Spring Security 2.0 Plugin
aop                 <no releases>    --  No description available
audit-logging       <0.4>            --  adds hibernate audit logging and onChange event handlers ...
authentication      <1.0>            --  Simple, extensible authentication services with signup ....
autorest            <no releases>    --  No description available

etc.  So if I could get this output, I could break each line into the pieces I want with simple String processing.

How can I do that?  In the spirit of reducing it to a problem already solved, I realized I just wanted to execute that command programmatically and capture the output.  One way to do that is to take advantage of Groovy’s ability to run command line scripts (GinA covers this, of course, but so does Scott Davis’s most excellent Groovy Recipes book).  Here’s the result:

def names = []
def out = "cmd /c grails list-plugins".execute().text
out.split("\n").each { line ->
    if (line =~ /<.*>/) {
        def spaceSplit = line.split()
        def tokenSplit = line.split('--')
        def name = spaceSplit[0]
        def version = spaceSplit[1] - '<' - '>'
        def description = tokenSplit[-1].trim()
        names << name
    }
}

Basically I’m executing the list-plugins command at a command prompt under Windows (sorry, but that’s still my life), splitting the output at the carriage returns (for some odd reason, using eachLine directly kept giving me errors), and processing each line individually.  The lines listing plugins are the ones with version numbers in angle brackets (like <0.3>), and the descriptions came after two dashes.  It seemed easiest to just split the lines both ways in order to get the data I wanted.

I ran this script and the other script together to see if I got the same output.  Here’s the result:

println "From 'grails list-plugins': " + names
println "From svn repo: " + pluginNames
println "Difference: " + (pluginNames - names)

From 'grails list-plugins': ["acegi", "aop", "audit-logging", ..., "yui"]
From svn repo: ["acegi", "aop", "audit-logging", ..., "yui"]
Difference: ["extended-data-binding"]

Why the difference? From the list-plugins output, here’s the line for “extended-data-binding“:

ext-ui              <no releases>    --  No description available
extended-data-binding<0.2>            --  This plugin extends Grails' data binding ...

Yup, the name ran into the version number format.  Sigh. Of course, the other problem with this is that at the moment it’s dependent on my own system configuration (Windows, with the grails command in the path), which can’t be a good thing.

Finally, after all this work, I suddenly realized that I already have the script used to list the plugins.  As with all the other Grails commands, it’s a Gant script in the <GRAILS_HOME>\scripts directory called, obviously enough, ListPlugins.groovy.  According to the documentation at the top, it was written by Sergey Nebolsin for version 0.5.5.

What Sergey does is to go to a slightly different URL and then parse the results as XML.  His script accesses

DEFAULT_PLUGIN_DIST = "http://plugins.grails.org"

instead of the SVN repo location listed above, but if you go there, they look remarkably alike.  I wouldn’t be surprised if http://plugins.grails.org is simply an alias for the SVN repository.

Note that the script also creates a cached version of the plugin list, called plugins-list.xml, which is kept in the

"${userHome}/.grails/${grailsVersion}/plugins/"

directory.  That’s completely understandable, but a lousy location on a Windows box.  I never go to my so-called “user home” directory, so I would never occur to me to look there for information.

His script checks to see if that file is missing or out of date.  If it’s necessary to update it, he opens a URL and starts processing:

def remoteRevision = 0
new URL(DEFAULT_PLUGIN_DIST).withReader { Reader reader ->
    def line = reader.readLine()

...

    // for each plugin directory under Grails Plugins SVN in form of 'grails-*'
    while(line=reader.readLine()) {
        line.eachMatch(/<li><a href="grails-(.+?)">/) {
            // extract plugin name
           def pluginName = it[1][0..-2]

           // collect information about plugin
           buildPluginInfo(pluginsList, pluginName)
        }

etc.

So, in effect, he’s screen scraping the SVN page; he’s just doing a better job of it than I was.

Incidentally, the line in his script that lead to my parsing problems is on line 86:

plugins << "${pluginLine.padRight(20, " ")}${versionLine.padRight(16, " ")} --  ${title}"

I could bump up the padding by one, or learn to parse the output better. I expect the “right” answer, though, is to do what Sergey did, pretty much. Still, if all I have to do is add a little padding, it’s awfully tempting to just “reuse” Sergey’s existing script.

In an upcoming post, I’ll talk about how I used the RichUI plugin to apply a “star rating” to each entry so that people could vote. I don’t have the site ready yet, though. I’ll be sure to mention it when I do.

Silly GORM tricks, part III: SQL keywords as attributes

I was writing a very simple Grails application and ran into a problem when I accidentally used a SQL keyword as a property name. This post documents what happened, and how I (pretty easily) fixed it.

To illustrate the issue, consider a trivial Grails application called “messaging” with a single class called Message.

class Message {
  String from
  String to
  String text
}

This is supposed to represent a simple email message, with fields to represent the sender, the receiver, and the text of the message itself. It seemed quite logical at the time to use the words from, to, and text for the fields, but that leads to problems.

I added a MessageController with dynamic scaffolding (i.e., def scaffold = Message) and started the server. When I accessed the MessageController link, however, I got

org.hibernate.exception.SQLGrammarException: could not execute query

In order to see the actual problem, I modified my DataSource.groovy file to add “loggingSql = true” in the dataSource section. After restarting the server, in the console window I saw that the exception was caused by a SQL command generated by the scaffolded list method in MessageController:

Hibernate: 
    select
        this_.id as id0_0_,
        this_.version as version0_0_,
        this_.from as from0_0_,
        this_.text as text0_0_,
        this_.to as to0_0_ 
    from
        message this_ limit ?

This statement looks fine, and in fact there’s nothing wrong with it. I couldn’t see the problem until I switched to MySQL so that I could browse the database independently.

(Switching to another database is covered many places in the Grails literature. In short, it means adding the JDBC driver to the messaging/lib directory, creating the messaging database in MySQL, and changing the driverClassName, url, username, and password settings in DataSource.groovy.)

When I did that and checked the database with the MySQL client, I found the problem (or at least a symptom of it):

mysql> show tables;
Empty set (0.00 sec)

In other words, the problem was that the message table didn’t exist. Somehow the SQL used to generate the table in the first place didn’t work.

Logging the SQL as I did wasn’t sufficient to show me the CREATE TABLE statement. If, however, I go into Config.groovy and change the value of log4j.logger.org.hibernate to debug, I see in the resulting console:

[1125] hbm2ddl.SchemaExport 
    create table message (
        id bigint not null auto_increment,
        version bigint not null,
        from varchar(255) not null,
        text varchar(255) not null,
        to varchar(255) not null,
        primary key (id)
    )

followed immediately by

[1125] hbm2ddl.SchemaExport Unsuccessful: create table message ...
[1125] hbm2ddl.SchemaExport You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'from varchar(255) not null, text varchar(255) not null, to varchar(255) not null' at line 1


The problem is that the word “from” is a SQL keyword. MySQL got upset when I tried to use it in the create table statement, as shown above.

How can I fix this? I could change the name of the from property, to, say, sender. Since Grails 1.0, however, GORM now has the ability to do custom ORM mappings, which feels like a cleaner way to solve the problem. Therefore, I added the following closure to my class:

static mapping = {
  from column: 'sender'
}

Now the generated create statement is:

[1157] hbm2ddl.SchemaExport 
    create table message (
        id bigint not null auto_increment,
        version bigint not null,
        sender varchar(255) not null,
        text varchar(255) not null,
        to varchar(255) not null,
        primary key (id)
    )

and the new error is

[1157] hbm2ddl.SchemaExport You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'to varchar(255) not null, primary key (id))' at line 1

So apparently the word “to” is also a problem. I therefore modified the mapping closure to include it as well:

static mapping = {
  from column: 'sender'
  to column: 'receiver'
}

Now it all works as it should. The lesson appears to be either that I should keep a list of SQL keywords handy, or simply that the custom ORM DSL is a Good Thing(TM), and so are the logging settings.

Silly GORM tricks, part II: dependent variables

This post discusses a relatively simple topic in GORM: how to use dependent variables in a domain class. It’s simple in the sense that it’s been discussed on the mailing list, but I haven’t seen it documented anywhere so I thought I’d do so here.

I started with a simple two-class domain model that I discussed in my last GORM post.

class Quest {
    String name
    static hasMany = [tasks:Task]
    String toString() { name }
}

class Task {
    String name
    static belongsTo = [quest:Quest]
    String toString() { name }
}

As before, there is a one-to-many relationship between quests and tasks. A quest has many tasks, and the belongsTo setting implies a cascade-all relationship, so inserting, updating, or deleting a quest does the same for all of its associated tasks.

In Bootstrap.groovy, I also have:

def init = { servletContext ->
         new Quest(name:'Seek the grail')
            .addToTasks(name:'Join King Arthur')
            .addToTasks(name:'Defeat Knights Who Say Ni')
            .addToTasks(name:'Fight Killer Rabbit')
            .save()
}

which shows how the classes are intended to work together.

The first change I want to make is to give tasks a start date and end date. My first attempt is to just add properties with those names, of type java.util.Date.

class Task {
  String name
  Date start
  Date end
  // ... rest as before ...
}

This leads to a minor problem. If I start up the server, I don’t see any quests or tasks. The reason is that my bootstrap code tries to create tasks without start and end dates, which violates the database schema restriction. My generated schema marks both start and end columns as “not null”.

There are many ways to fix that. I can either assign both start and end properties for each task in my bootstrap code, or add a constraint in Task that both can be nullable, or do what I did here, which is to give them default values.

class Task {
  String name
  Date start = new Date()
  Date end = new Date() + 1
  // ... rest as before ...
}

I do have a constraint in mind, actually. I’d like to ensure that the end date is after the start date. That requires a custom validator, which is also pretty easy to implement:

class Task {
  // ...
  static constraints = {
    name(blank:false)
    start()
    end(validator: {  value, task ->
       value >= task.start
    })
  }
}

That works fine.

Now for the dependent variable. My tasks all have a start and an end, so implicitly they have a duration. I could add the duration variable to my Task class, but I don’t want to save it in the database. It’s dependent on the values of start and end. I also don’t want to be able to set it from the gui.

Here’s the result:

class Task {
  String name
  Date start
  Date end
  
  int getDuration() { (start..end).size() }
  void setDuration(int value) {}

  static transients = ['duration']

  // ... rest as before ...
}

This computes the duration from the start and end dates by returning the number of days between them. It relies on the fact that Groovy modifies java.util.Date to have the methods next() and previous(), and since Date implements Comparable, it can then be used in a range, as shown.

(As an aside, this implementation is probably pretty inefficient. If the number of days between start and end was substantial, I think this implementation executes the next() method over and over until it reaches the end. I thought about trying to subtract the two dates, but interestingly enough the Date class only has plus() and minus() methods that take int values, not other Dates. I considered adding a category that implemented those methods, but haven’t tried it yet. I’d like to look in the Groovy source code for the plus() and minus() implementations, but I couldn’t find it. I did find something similar in org.codehaus.groovy.runtime.DefaultGroovyMethods, but I’m not sure that’s the same thing. Sigh. Still a lot to learn…)

By putting 'duration' in the transients closure, I ensure that it isn’t saved in the database.

The getDuration method is pretty intuitive, but adding set method as a no-op is somewhat annoying. If I leave it out, then Groovy will generate a setter that can modify the duration. As an alternative, according to GinA I can also supply my own backing field and mark it as final:

class Task {
  // ...
  final int duration

  int getDuration() { (start..end).size() }
  // ...
}

Just to be sure, I added the following test to my TaskTests:

void testSetDuration() {
    Task t = new Task(name:'Join King Arthur')
    shouldFail(ReadOnlyPropertyException) {
        t.duration = 10
    }
   q.addToTasks(t).save()
}

That passed without a problem.

Interestingly, the dynamic scaffold still generates a modifiable input text field for duration, both in the create and edit views. I can put my own value in it and submit the form without a problem. The result does not get saved, which is correct, but I don’t see an exception thrown anywhere in the console. If I generate the static scaffolding, I know that in Task.save there is a line like

t.properties = params

which is how the form parameters are transfered to the object. Presumably the internal logic knows enough to avoid trying to invoke a setter on a final field. Of course, as soon as I generate the static scaffolding, I usually just delete that row in the GSP form.

There’s one final (no pun intended) issue with the dynamic scaffolding. The generated list view puts its properties in <g:sortableColumn> tags. This holds true for the duration, as well. Normally, when I click on the column header, the result is sorted, ascending or descending, by that property. If I click on the duration column header, however, I get an “org.hibernate.QueryException: could not resolve property: duration of: Task“.

It turns out that the User Guide has a “Show Source” link for every tag. When I clicked on that link for the sortableColumn tag, I saw near the top:

if(!attrs.property)
  throwTagError("Tag [sortableColumn] is missing required attribute [property]")

The error I got in the console is “could not resolve property”, but it’s possible this is the source of that issue. I’m not sure. The only other source (again, no pun intended) of the problem I could see was the execution of the list action at the bottom. That would imply that Grails is generating the Hibernate query and we’re failing at that point, which would be consistent with the error reported above.

At any rate, the duration property now works in the domain class. I can always modify the views to ensure I don’t try to set it.