Upcoming Events, and “The Streak”

I’m really not a workaholic. I prefer days off as much as anybody. The problem is that there are all these things I want to do, so I volunteer to do them, and suddenly I’m overbooked so much I don’t have time for a break.

I think part of it comes down to an acronym that’s making the rounds these days. It’s called FOMO, and stands for Fear Of Missing Out. I suffer from it as much as anybody, I guess. I’ll need to work on that.

At any rate, I’m beginning a particularly busy part of my calendar, and I thought I’d mention it here in case I have the opportunity to meet any of you along the way.

On Tuesday, May 26, I’m giving my Groovy Vampires talk at the Austin Java Users Group. I really have a lot of fun with that talk. I plan to update it a bit, especially because a recording of the version I gave at last year’s SpringOne 2GX conference is currently available on InfoQ.

To give you a clue what it’s about, one day I was wandering in a Barnes and Noble and noticed that there was only one book shelf labeled “Computer”:

ComputerBooks

while there were three (yes, three) labeled “Teen Paranormal Romance”:

TeenParanormalRomance

Instead of lamenting the decline of western civilization, I decided that what this meant was that I needed to add some Groovy vampires to my book. The talk is based on that, and involves accessing RESTful web services, parsing JSON data, using a MongoDB database, creating a RESTful service using ratpack and Grails, and more.

The next day, May 27, I’ll be speaking at the Austin Groovy and Grails Users Group, this time on Testing Grails. While the bulk of the talk was created using Grails 2.3 and works under 2.4, I do plan to say a few words about Grails 3.0 as well. The testing mechanisms haven’t changed much in going from 2.3 to 2.4 to 3.0, though now they’ve added Geb as an included dependency, so you can make functional tests easily.

If you’re in the Austin, TX area this week, please drop by and say hi.

Also this week, starting tomorrow I’ll be hanging out at JavaRanch, because my Groovy Fundamentals video course will be featured in the Groovy forum this week. I’ve always been a fan of the ranch. I first joined way back in the late 90’s, when I was getting Java certified for the first time.

Speaking of the Groovy Fundamentals video (did I really need to link to that again? Yes. Yes I did), that’s the first of three Groovy video courses I recorded for O’Reilly recently. It covers the basics, from POGOs to collections to closures, in almost exactly four hours.

The second video course in the series is called “Practical Groovy”, and covers topics like integrating Java and Groovy, runtime metaprogramming, using AST transformations, the Groovy SQL class, and more. That one ought to go live within a week.

The third video in the series is called “Mastering Groovy”. It covers Spock testing (though I use Spock tests in the earlier videos), working with Spring, JPA, and NoSQL databases, functional programming, traits, and RESTful web services, among other topics. That one will go live when I finally finish reviewing all the video segments. It’s rather dismaying to discover that reviewing six hours of video takes at least six hours.

Though I must admit I’m tired of watching and listening to myself, I’m very proud of these videos and hope they’ll be helpful. I used to joke about selling the movie rights to Making Java Groovy, and speculate about who would I would recommend to play me.

(Best suggestion: Vincent D’Onofrio from his Men In Black days; most definitely not his Kingpin days.)

In essence, these video courses are that movie. They capture pretty much everything I’ve learned about Groovy over the years, condensed to an easily digestible form. If you’ve ever seen me give Groovy talks at No Fluff, Just Stuff conferences, you’ve seen a lot of what I’ve included, though with fewer jokes.

That brings me to next weekend, which is the Dallas NFJS event. I’ll be giving a Spock talk, and my “Managing Your Manager” talk, and a series on Android development. Again, if you’re in the neighborhood, please drop by and say hi.

Incidentally, some time this week (tentatively Wednesday, 5/27), Peter Ledbrook, Baruch Sadogursky, and I are planning to do another Groovy Podcast. I love doing those, so assuming that happens I’ll definitely tweet about it.

From Dallas I’m off to San Diego, where I’ll be teaching a (private) Groovy and Grails class. Teaching technical training classes is What I Do, practically my raison d’être, so feel free to contact me for details.

After the Grails class I’m heading to Sebastopol, CA, home of O’Reilly, to get back into the recording studio again. This time I’m working on a couple of Android videos, and if I manage to finish those I’ll also try to record something on Gradle for Android. That will all be the same week that culminates in the Gradle Summit in Santa Clara, where I’m doing an updated talk on the Gradle plugin for Android. I’m really looking forward to that conference, though I may miss the first day if we’re still recording.

(Yes, I’m making progress on the Gradle for Android book. Yes, it would be a lot easier if the Android plugin for Gradle, Android Studio, and even Android itself stopped changing so much so frequently, causing me to have to constantly rewrite chapters. Yes, the video course will be related, and will help me write the book. Yes, I’ll probably scowl if you ask me for an actual release date, but don’t let that stop you.)

When the Gradle Summit ends, I finally get to go home again, at least for a few hours, before I’m headed to South Carolina for another training class. I might have another one after that, too, but I haven’t decided.

Eventually I’m going to need a break, so it’s a good thing I scheduled one. Next year (!) my wife and I decided to go on the JoCo Cruise in the Caribbean, which is a sweet nerd cruise featuring Jonathan Coulton, Wil Wheaton, Paul and Storm, and many others. That really ought to be fun.

Finally, I need to say something about “The Streak”. Like many people in the I.T. field, I was a career changer. I came from engineering. More accurately, I should say that I was a research scientist, specializing in the field of turbomachinery aeroacoustics. What that really meant was I did lots and lots of math and wrote lots and lots of Fortran (the nightmares have ended, but it took a while). Ultimately I joined an AI group, went back to school at night, got my MS in CS and decided to leave my job.

My new job as a technical trainer started May 31, 2000. That day I helped teach a training course in Object Oriented Concepts (remember when did stuff like that?). I spent five years teaching classes (including roughly one Intro Java course a month back then) before I went out on my own in March of 2005.

We’re coming up on May 31, 2015, and in all that time, I have never missed a day of training. Not one. I call that The Streak, and since it looks like I’m going to make it to that date I figured it was okay to announce it here.

That journey, and the life changes that accompanied and preceded it, deserve their own blog post. I didn’t want the date to pass, though, without mentioning it. I’m rather inordinately proud of The Streak. Some of it is certainly luck, and it can’t last forever, but it means a lot to me. Right now my job and my career are everything I ever dreamed they could be, and I think The Streak is a side-effect. At the very least, it always gets me up in the morning. :)

Concurrent Kitties Using GPars

On today’s Groovy Podcast, I mentioned that I was teaching a Groovy training class this week at Research Now in Plano, TX. That’s not how I said it, though. I said that I was broadcasting live in front of a studio audience and that they were the most amazingly awesome group I’ve ever encountered.

(Yeah, okay, sometimes I shamelessly pander for good evals. I’ll let you know if it worked after the class ends. Unless it doesn’t, in which case I probably won’t.)

During the podcast, I told my inestimable co-host, Peter Ledbrook, that we got a chance to use GPars in class. The app we used it on was all about the primary goal of the internet, which is to display cat pictures.

Peter then shamed me into writing a blog post about it, which you’re reading now.

I’ve actually written about this app before, for another client. My post there was originally entitled, “The Reason The Internet Was Invented, Or Cat Pictures FTW”, but the host changed it to the far more mundane Calling RESTful Services in Groovy.

The basic idea is that Flickr (remember them? Me neither) has a RESTful API that lets you search for photos. The “flickr.photos.search” request doesn’t require authentication, but does require a whole set of query parameters, including an API key.

Funny story: in order to get a Flickr API key, you actually have to register at Yahoo! Remember them, too? Yeah, neither did I.

At any rate, I registered and got my key, so I can now do the searches. Here’s the start of my Groovy script to do it:

import groovy.json.*

String key = new File('flickr_key.txt').text
String endPoint = 'https://api.flickr.com/services/rest?'
def params = [method        : 'flickr.photos.search',
              api_key       : key,
              format        : 'json',
              tags          : 'kitty',
              nojsoncallback: 1,
              media         : 'photos',
              per_page      : 6]

// Build URL and download JSON data
String qs = params.collect { it }.join('&')
String jsonTxt = "$endPoint$qs".toURL().text

The query string is constructed from the map of params by running a collect on each element (which returns key=value for each Map.Entry) and then joining the resulting list with an ampersand. Notice the tags key was assigned to the word “kitty”.

The next part of my script writes out the results and appends them to a file.

// write formatted JSON data to file
File f = new File('cats.json')
if (f) f.delete()
f << JsonOutput.prettyPrint(jsonTxt)
println JsonOutput.prettyPrint(jsonTxt)

Here’s a sample formatted JSON response:

{
    "photos": {
        "page": 1,
        "pages": 127979,
        "perpage": 6,
        "total": "767873",
        "photo": [
            {
                "id": "17418175405",
                "owner": "31469819@N02",
                "secret": "9055856685",
                "server": "5453",
                "farm": 6,
                "title": "A Ghostly Cat",
                "ispublic": 1,
                "isfriend": 0,
                "isfamily": 0
            },
            {
                "id": "16795470464",
                "owner": "95966544@N07",
                "secret": "cc4af0d44f",
                "server": "8799",
                "farm": 9,
                "title": "Looking for a home",
                "ispublic": 1,
                "isfriend": 0,
                "isfamily": 0
            },
            {
                "id": "17228164988",
                "owner": "92936362@N06",
                "secret": "d42c68bbf3",
                "server": "8734",
                "farm": 9,
                "title": "peaches the cat",
                "ispublic": 1,
                "isfriend": 0,
                "isfamily": 0
            },
            {
                "id": "17208304157",
                "owner": "102705402@N02",
                "secret": "582fff8f44",
                "server": "8688",
                "farm": 9,
                "title": "This is the sweetest cat in the world!",
                "ispublic": 1,
                "isfriend": 0,
                "isfamily": 0
            },
            {
                "id": "17228717179",
                "owner": "37561081@N07",
                "secret": "eb8d0119fe",
                "server": "7722",
                "farm": 8,
                "title": "\u65e9\u5b89",
                "ispublic": 1,
                "isfriend": 0,
                "isfamily": 0
            },
            {
                "id": "17388635206",
                "owner": "127041099@N08",
                "secret": "6310c6012a",
                "server": "7745",
                "farm": 8,
                "title": "Tsim Tung Brother Cream (\u5c16\u6771\u5fcc\u5ec9\u54e5)",
                "ispublic": 1,
                "isfriend": 0,
                "isfamily": 0
            }
        ]
    },
    "stat": "ok"
}

Note that nowhere in the various photo elements do you find a URL for the actual image. It turns out that to assemble the image you have to plug various pieces of the photo elements into a string, which is something Groovy is good at. First, however, I have to parse this and grab the photo elements:

// parse JSON data and build URL for pictures
def json = new JsonSlurper().parseText(jsonTxt)
def photos = json.photos.photo

The photos variable is now a list of maps for each photo, which I can transform into URLs using a collect:

def images = photos.collect { p ->
    String url =
        "http://farm${p.farm}.staticflickr.com/${p.server}/${p.id}_${p.secret}.jpg"
    url.toURL().bytes
}

The Groovy string uses the farm, server, id, and secret elements of the response in each photo and builds a complete URL for the JPG image. Then I convert that to an actual URL and call getBytes() to return byte arrays.

I can then use a SwingBuilder to assemble a trivial GUI showing all the images:

// build UI using Swing
new SwingBuilder().edt {
    frame(title: 'Cat pictures', visible: true, pack: true,
            defaultCloseOperation: WC.EXIT_ON_CLOSE,
            layout: new GridLayout(0, 2, 2, 2)) {
        images.each {
            label(icon: new ImageIcon(it))
        }
    }
}

That requires some additional imports:

import groovy.swing.SwingBuilder

import java.awt.GridLayout
import javax.swing.ImageIcon
import javax.swing.WindowConstants as WC  // Ooh, aliased imports

Here’s where we improved the system using GPars. The download of the images can be done in a multithreaded fashion by adding a GParsPool:

import static groovyx.gpars.GParsPool.*

// ...

def images = []
withPool {
    images = photos.collectParallel { p ->
        String url =
                "http://farm${p.farm}.staticflickr.com/${p.server}/${p.id}_${p.secret}.jpg"
        url.toURL().bytes
    }
}

// ...

That uses the default pool size, which is the number of processors you have plus one. The images are now downloaded concurrently as part of transforming the photo elements into byte arrays using collectParallel.

Here’s the whole script together:

import static groovyx.gpars.GParsPool.*

import groovy.json.*
import groovy.swing.SwingBuilder

import java.awt.GridLayout
import javax.swing.ImageIcon
import javax.swing.WindowConstants as WC

String key = new File('flickr_key.txt').text
String endPoint = 'https://api.flickr.com/services/rest?'
def params = [method        : 'flickr.photos.search',
              api_key       : key,
              format        : 'json',
              tags          : 'kitty',
              nojsoncallback: 1,
              media         : 'photos',
              per_page      : 6]

// Build URL and download JSON data
String qs = params.collect { it }.join('&')
String jsonTxt = "$endPoint$qs".toURL().text

// write formatted JSON data to file
File f = new File('cats.json')
if (f) f.delete()
f << JsonOutput.prettyPrint(jsonTxt)
println JsonOutput.prettyPrint(jsonTxt)

// parse JSON data and build URL for pictures
def json = new JsonSlurper().parseText(jsonTxt)
def photos = json.photos.photo

def images = []
withPool {
    images = photos.collectParallel { p ->
        String url =
                "http://farm${p.farm}.staticflickr.com/${p.server}/${p.id}_${p.secret}.jpg"
        url.toURL().bytes
    }
}

// build UI using Swing
new SwingBuilder().edt {
    frame(title: 'Cat pictures', visible: true, pack: true,
            defaultCloseOperation: WC.EXIT_ON_CLOSE,
            layout: new GridLayout(0, 2, 2, 2)) {
        images.each {
            label(icon: new ImageIcon(it))
        }
    }
}

Here is the result of a sample run:
Cat-pictures

So there you have it, except for the stupid Flickr key, which I decided to let you register for on your own. Hey, I had to go through that pain, so everybody else does, too.

Well, not everybody. As part of my pandering for evals technique, I did give my key to the students in my class, who no doubt will reward me with stellar evals once we’re done. Probably. It could happen. Either way, at least there were cat pictures, and that’s a Good Thing.

Groovy posts in other places

Recently I’ve been writing about Groovy and Grails for my friends at Accelebrate. I do that because:

  1. They’re a great client
  2. They support what I do
  3. They pay me CASH MONEY

The only problem is, whenever I post there, I don’t post here. I thought, therefore, that I’d add some links here to let everyone know what I’ve been writing about on their blog.

  • Calling RESTful services with Groovy is all about using Groovy to invoke Flikr’s API for the reason the Internet was invented: Cat Pictures.
  • That Which We Call A POGO, By Any Other Name, is a historical review of POJOs and POGOs, with Groovy AST transformations, and lots and lots of links to wildly unlikely resources that form the word POGO.
  • Building a Geolocation Web Application with Groovy and Grails was originally entitled, “Where in the World is Steve Heckler?”, but they changed it. In that post, I describe writing a Grails app to display locations on a map. (It helps to know that Steve Heckler is the owner of Accelebrate.) The best part is that I even managed to use pictures of Steve as custom markers on the resulting Google Map. :)
  • Kicking AST and Taking Names is all about using Groovy’s abstract syntax tree (AST) transformations to dramatically simplify coding. That sounds dry, but it does have figures that reference CaddyShack, South Park, and even Gladiator, so it’s got that going for it. Which is nice.

So if you’re enjoying any of my posts and you’re wondering why I haven’t been more active here, there you go. And wherever you go — (waits a beat) — there you are.

If a method arg is a closure, for crying out loud pass it a closure

This is a (mildly) embarrassing post, because it demonstrates how in my transition to functional programming I missed something important. I get it now, though, and I might as well help others avoid the same mistake I made.

I teach a lot of Grails training classes, and one question that always comes up is how to map to existing databases rather than generate the schema from the Grails application. Hibernate (and therefore Grails) would love to generate the database schema for you, for a variety of reasons, but mostly because going the other way isn’t unique. You have to make a lot of decisions about a domain model when you reverse-engineer an existing schema, ranging from multiplicities in relationships to how to map many-to-manys to directionality and more.

In my standard Grails course, rather than create a database we can map to, I go with an existing, freely available sample. For years I’ve chosen the Sakila sample database from MySQL.

(As an aside, it took me years to realize that the word Sakila is actually intended to be a subtle joke. It’s the letters SQL with some vowels thrown in to make it possible to pronounce the word, with the “Q” replaced by a “k”. Go figure. As it turns out, it’s also the name of their dolphin emblem, but that’s neither here nor there.)

Over the years I’ve tried to map the Sakila schema to a Grails domain model, with varying degrees of success. I think I have it all worked out now, but that’s the subject for another blog post. Here, instead, I want to talk about stored procedures.

Many of my clients want to use Grails with a database that has lots of stored procedures in it. Hibernate doesn’t like that. Hibernate wants to map all the domain classes directly to tables, without going through any possible intermediaries. While it is possible to invoke a stored procedure from Hibernate, it’s a bit awkward, mostly because Hibernate doesn’t want to do it.

(Length of Java Persistence Using Hibernate: about 850 pages. Number of pages that discuss stored procedures: about 5.)

That’s where Groovy (as opposed to Grails) comes in. The Groovy API includes the wonderful groovy.sql.Sql class, which is a simple, but effective, façade over JDBC. That class will open and close connections, eliminate all the annoying boilerplate code associated with JDBC, and more. If I ever have to write raw SQL code, I always switch to groovy.sql.Sql rather than use JDBC.

(That’s not quite true. The JdbcTemplate class from Spring is nearly as helpful. If I don’t have Groovy available and I have to do JDBC, I go with that.)

Let me get specific. The Sakila database represents a video store (which shows how old that is). It has tables representing stores, and inventory, and actors, and films, and more. But it also contains a handful of stored procedures, one of which is called film_in_stock. According to the documentation, the film_in_stock procedure takes two input parameters (the id of the film and the id of a store) and one output parameter (the number of copies of that film currently at that store). The example they show is:

mysql> call film_in_stock(1, 1, @count);
+--------------+
| inventory_id |
+--------------+
| 1            |
| 2            |
| 3            |
| 4            |
+--------------+
4 rows in set (0.06 sec)

Query OK, 0 rows affected (0.06 sec)

mysql> select @count;
+--------+
| @count |
+--------+
| 4      |
+--------+
1 row in set (0.00 sec)

There are four copies of film 1 (“Academy Dinosaur”) at store 1 (a store at “47 MySakila Drive” in Alberta). This provides me with a test case:

import spock.lang.*

class StoreProcServiceIntegrationSpec extends Specification {
    def storedProcService  // inject the service

    void "call film_in_stock stored procedure"() {
        expect:
        storedProcService.callFilmInStock(1, 1) == 4
    }
}

This assumes my call to the stored procedure happens in a class called StoredProcService. I have to use an integration test here, because I want to use the real database, but that means I can rely on Spring dependency injection to get the service into the test.

To access the Sakila database from a Grails app, I configured my conf/DataSource.groovy file to include the dataSource properties:

import org.hibernate.dialect.MySQL5InnoDBDialect

dataSource {
    pooled = true
    jmxExport = true
    driverClassName = "com.mysql.jdbc.Driver"
    username = "root"
    password = ""
    dialect = MySQL5InnoDBDialect
}
// ...
environments {
    development {
        dataSource {
            dbCreate = "validate"
            url = "jdbc:mysql://localhost:3306/sakila"
        }
    }
    test {
        dataSource {
            dbCreate = "validate"
            url = "jdbc:mysql://localhost:3306/sakila"
        }
    }
// ...
}

(Yeah, I have super high security on this database. Don’t even think about breaking in. You’d never guess that the root user has no password whatsoever. I mean, who would do such a ridiculous thing?)

So the default data source in my Grails app is the Sakila database, using a MySQL driver that I have listed in the conf/BuildConfig.groovy file:

dependencies {
    runtime 'mysql:mysql-connector-java:5.1.29'
    test "org.grails:grails-datastore-test-support:1.0-grails-2.4"
}

Again, nothing terribly unusual here.

The key observation is that the dataSource reference in the config file refers to a Spring bean, which can be injected in the usual manner. So I created a Grails service called StoredProcService and specified the dataSource bean as a dependency.

@Transactional
class StoredProcService {
    def dataSource
    // ...
}

The GroovyDocs for the Groovy API show that the groovy.sql.Sql class has a constructor that takes a data source, so I’m all set. Then, to call the stored procedure, I need to invoke the call method from that class.

This is where life gets interesting, and ultimately tricky. The procedure I want to call has both input and output parameters, so my call has to take that into account. It turns out that the way to call a stored procedure with both inputs and outputs is to invoke:

void call(String sql, List<Object> params, Closure closure)

The SQL string uses the JDBC escape syntax for stored procedure calls. The params list take the input parameters and placeholders for the output parameters, and the closure is then invoked with the output parameters.

Wait, what? That’s a little fast. Here’s the actual call, or at least this is the version I had before last weekend:

import groovy.sql.Sql

@Transactional
class StoredProcService {
    def dataSource
    
    int callFilmInStock(filmId, storeId) {
        String sqlTxt = '{call film_in_stock(?,?,?)}' // JDBC escape syntax
        Sql db = new Sql(dataSource)  // use the injected datasource
        db.call(sqlTxt, [filmId, storeId, Sql.INTEGER]) { count ->
            // use the count here
        }
    }
}

The Sql class provides constants like “static OutParameter INTEGER” to represent output variable in the stored procedure call. Then the last argument is a closure that has as many arguments as there are output parameters (in this case, one, which I called count).

Now, though, I have a problem. I’d like my callFilmInStock method to return the result of invoking the film_in_stock stored procedure, i.e., the count of films at that store. The problem is, I can’t just say return count inside that closure, because returning from a closure returns only from the closure, not the calling method.

I’ve blogged about that before, and even submitted a variation to Baruch Sadogursky as a Groovy Puzzler, but this is where I first encountered that problem.

My solution was always to create a local variable outside the call, assign the count to that variable, and return the variable:

int callFilmInStock(filmId, storeId) {
    String sqlTxt = '{call film_in_stock(?,?,?)}' // JDBC escape syntax
    Sql db = new Sql(dataSource)  // use the injected datasource
    int result = 0
    db.call(sqlTxt, [filmId, storeId, Sql.INTEGER]) { count ->
        result = count
    }
    return result
}

This works, and (sigh) I’ve been teaching this approach for some time now. I’ve always been uncomfortable with it, though, and when I started digging into Java 8 lambdas I realized why. I’m using a closure here and relying on it having a side effect, which is to modify a variable defined outside the closure. In fact, in Java 8, lambdas are definitely not supposed to do that. The only variables lambdas are supposed to access from outside the lambda are those that are “effectively final”, meaning they’re never modified. Groovy closures don’t have that limitation, but I still had this nagging feeling that I was, as they say, doing it wrong.

Last weekend, the new NFJS season started for me in Minneapolis, and I got a chance to talk about this to the inimitable Venkat Subramaniam, who writes books on this stuff (among other things). After I explained the problem, he thought about it for about five whole entire minutes and said that I should be passing the method a closure, not returning an integer.

The right way to write my service function is to add a closure to the method signature and use it in the sql call:

void callFilmInStock(filmId, storeId, closure) {
    String sqlTxt = '{call film_in_stock(?,?,?)}' // JDBC escape syntax
    Sql db = new Sql(dataSource)  // use the injected datasource
    db.call(sqlTxt, [filmId, storeId, Sql.INTEGER], closure)
}

Wow, that’s so much simpler. The client now passes in a closure that uses the returned value, rather than trying to kludge returning the value itself. That means the test case becomes:

void "call film_in_stock stored procedure"() {
    expect:
    storedProcService.callFilmInStock(1, 1) { count ->
        assert count == 4
    }
}

(This uses the normal Groovy idiom where a method taking a closure as its last argument can put the closure after the parentheses.)

This test passes just as the previous one did, even though the method return type is now void.

That’s the part I never got before. The groovy.sql.Sql.call(String, List, Closure) method takes a closure as its last argument, which means I can supply the closure myself and the method will use it with the returned count.

When I learned about functional programming, I got the idea of transforming collections rather than iterating over them, and using filters to select only the values I want, and reducing the results down to single values. I got that I’m supposed to favor immutability wherever possible. What I apparently hadn’t grokked was sending closures as method arguments and how that simplified everything. I’m still not sure I’m all the way there, but I’m certainly a lot closer now.

Over the weekend I tweeted that Venkat had fundamentally changed my understanding of functional programming, which is both cool and seriously annoying. Frankly, those are also words I would use to describe Venkat. If he wasn’t such a nice guy, he would be insufferable, mostly due to sheer jealousy. But we rapidly learn on the NFJS tour never to compare ourselves to Venkat in any way or you’ll just drive yourself crazy. Just don’t tell him I said that.

Now I have to go update my Grails materials…

Now Co-Hosting the Groovy Podcast…

When I first starting learning Groovy in a serious way, one tool I enjoyed was listening to the Grails Podcast. Glen Smith and Sven Haiges led a discussion about Grails-related issues on a regular basis. Of course, since they talked about new developments in Grails, they also had to cover new developments in Groovy, so the podcast was a valuable resource for both.

At the beginning of each podcast, Glen would say that he was podcasting from Canberra, Australia, and Sven was podcasting from San Jose, California. Of course, Sven might have been in CA, but Sven is actually a European and eventually moved back to Munich. That led me to contact Glen and ask to be the third person in the booth, arguing (a) that they needed an American, too, and (b) together we’d be Glen, Sven, and Ken, and how cool was that?

(To be honest, I didn’t care whether the podcast included an American or not. It just seemed a stronger argument than the fact that our names rhymed.)

The Grails Podcast went on for several years, but underwent significant changes over time and became far less frequent. Eventually Peter Ledbrook (inimitable co-author, with Glen, of Grails in Action) took over, and a year ago he renamed it the Groovy Podcast.

Meanwhile, whenever I saw Peter at a conference, I casually mentioned that I wanted to be involved in the new podcast. Finally, last year at the SpringOne2GX conference, Peter finally broke down and said he’d be willing to let me work with him.

Okay, that’s perhaps a bit too self-deprecating, but when you want something like that you have to keep prodding until they say yes. I often tell my son that in order to be at the right place at the right time, you have to be at the wrong place at the wrong time a lot.

So, in the end, I’m very happy to announce that I’m now the co-host of the Groovy Podcast. Peter uses Google Hangouts to record the podcast, so anyone can join in real time. The results are published in the Groovy Podcast YouTube channel in video form and on PodBean for those who just want the audio feed.

I participated in episode 6 back on Feb 5, and we just recorded and published episode 7 on Feb 20. The goal is to get the podcast back on a regular, two-week schedule, which should make it easier to build a following again. Episode 5, from January, included a discussion with Groovy project lead Guillaume Laforge and Grails project lead Graeme Rocher about the change in Pivotal support for the projects, so that one is worth watching regardless.

Here’s an embedded version of episode 7, assuming that works correctly:

I should also mention that the show notes for the last couple of podcasts are available on GitHub. That’s now a public repo, so anyone can see them and potentially suggest topics for future discussions.

So far I’ve written this post assuming you know me already, but it’s possible that you found this from the podcast itself and are wondering who I am and why I would have anything productive to say about the Groovy ecosystem. Let me present a short bio here.

My name is Ken Kousen. The last name is pronounced as though it were spelled cousin, like the relative, for reasons unknown but probably having something to do with my grandfather’s very heavy eastern European accent. I’ve been working with Groovy since late 2006, but much more as a user than as a contributor. Eventually I started writing about the language, and my book on adding Groovy to Java projects to make development easier, entitled Making Java Groovy, was published by Manning in the Fall of 2013. I’m currently working on another book, this time from O’Reilly, entitled Gradle for Android, covering the new Gradle build system for Android projects. That book will be available in early access mode Real Soon Now(TM).

I run a one-person company called Kousen IT, Inc., where my day job is teaching technical training courses on all areas related to Java, specializing in open source projects like Spring, Hibernate, Android, and naturally Groovy, Grails, Gradle, and more. I do some consulting and mentoring as well, but my schedule tends to fill with training classes so quickly that it’s hard to arrange for other activities.

I’m also a regular speaker on the No Fluff, Just Stuff conference tour, which meets on roughly 20 to 25 weekends a year at various cities throughout the U.S. On the tour I mostly speak about Groovy-related projects, but I also discuss Android, Java 8, and a few other areas like Bayesian statistics and even “Managing Your Manager”.

In fact, my “Managing Your Manager” talk has been sufficiently well received at previous conferences that it will be the opening day keynote at the DevNexus conference in Atlanta the week of March 9, 2015. I’ve also spoken at the U.S. version of the Gr8 conference several times, and (spoiler alert!) will be giving a keynote there in July discussing The Mobile World and Groovy’s role in it.

Please feel free to contact me with questions, comments, or any good IT-related jokes, which I’ll freely reuse and even give you credit/blame if you want. If you and/or your company are interested in training classes, you should definitely contact me sooner rather than later, because as the Groovy Podcast explodes in popularity and my reputation skyrockets as a result, my prices will no doubt also become appropriately astronomical. So get your requests in now, while I’m still a relative nobody, at least compared to Peter.

Years ago I used to joke to my wife about having a 10-year plan to becoming an overnight sensation (btw, I’m currently in year 15…). I thought that winning a JavaOne Rock Star award in 2013 was the amazing accomplishment of a lifelong dream, but being a part of this podcast is so beyond epic —

Okay, I’ll stop now. Let’s just say I’m really happy to be here, and if you’re at all interested in the Groovy world you should enjoy the podcast. We’re certainly having fun making it.

Groovy/Grails – Pivotal == Opportunity

The news broke this morning that Pivotal plans to withdraw its financial support from the Groovy and Grails projects by the end of March, 2015. The heads of both projects, Guillaume Laforge and Graeme Rocher, have each blogged about it, with their typical grace, thanking Pivotal for the opportunity and assuring everybody that both projects have a long, bright future ahead of them.

Since I.T. is a field that frequently has very little memory, let me remind people about a couple of items:

  1. Groovy existed as a successful open source project for at least four years before SpringSource started supporting it. Grails started as an offshoot of Groovy and was just as popular.
  2. Several of the core teams members of both projects formed the G2One company, which was sufficiently successful in its first year that SpringSource acquired it in the first place

Neither Groovy nor Grails are radical departures from their underlying technologies. That makes them quiet projects — they’re popular, but they aren’t showy. They never have been popular among the hipster IT community, but they get the job done. It’s so easy to add Groovy to existing Java projects that most developers who do it don’t feel obligated to crow about it. Groovy is being used throughout the Java industry, and in most places it’s just acknowledged without comment. Grails, for all its power, feels like an easy way to build traditional Spring/Hibernate web applications, which work really well but are hardly sexy.

It’s therefore not surprising that the potential of Groovy and Grails is often underestimated. I have no idea what Pivotal was thinking, and the public statements about them have been uniformly positive (again, not a big surprise given the people involved), but I find it easy to believe Groovy and Grails were underrated yet again.

Many people will now write many posts demonstrating, yet again, how powerful Groovy is and how easy it is for Java developers to learn. The Grails web site already has dozens of success stories. I’m not going to try to add to that, other than to say I love working with both technologies and they’ve completely changed my life.

I want to mention something else. I’ve been in the IT industry for over 20 years. I was in the engineering community for a dozen years before that. I would hold up my academic background against anybody’s. From all those experiences, I’ve known geniuses and incredibly hard workers, and communities who are friendly, hostile, and everything in between.

I’m here to tell you that the Groovy and Grails core teams members are among the brightest people I’ve ever met, but they’re also successful because they’re wonderful individuals to be around. In an industry often marred by what I like to call “boys behaving badly,” I’ve never heard a negative word about anyone involved in Groovy and Grails. That attitude starts at the top, with great people like Graeme and Guillaume, and I feel privileged to know and work with them, however tangentially.

Look, community matters. It makes the difference between struggling to accomplish anything and enjoying your job. I teach technical training courses for a living, and you should see the joy in Java developers eyes when they learn both how much they can now do, and — this is important — how newcomers are treated with respect and how their questions are answered without patronizing or humiliating attitudes.

They say it’s hard for companies to find good developers these days. Well, here’s your opportunity. For the price of supporting technologies that will inevitably help your bottom line, you can acquire a team of coders that are among the most accomplished, most dedicated, and most easy to work with that you’ll ever meet.

Pivotal is now out of the picture. This is a great opportunity for a smart company to step in, acquire two fantastic teams of developers, and be a hero to the open source community. Heck, I’d do it myself if I could afford it. Don’t miss this chance. They don’t come along very often.

Spaceships, Elvis, and Groovy inject

When I first started learning Groovy, I took to collect pretty quickly. The current trend of adopting “functional programming” practices works well in Groovy, though the names of the methods are somewhat surprising. For example, collect is like map, findAll is the equivalent of filter, and inject is the proposed replacement for reduce (or whatever the similar process is for your favorite language).

As a trivial example, in Groovy you can write:

(1..20).collect { it * 2 }      // double them all
       .findAll { it % 3 == 0 } // find the doubles divisible by 3
       .sum()                   // add them up

which is a functional style, even though Groovy is an object-oriented language.

Notice, however, that I didn’t use inject. That’s not an accident, of course. For years, while I “got” collect and findAll, I never found a usage for inject that couldn’t be done in an easier way. The inject version of the above example would be:

(1..20).collect { it * 2 }                   // double them all
       .findAll { it % 3 == 0 }              // find the doubles divisible by 3
       .inject(0) { acc, val -> acc + val }  // add them up

That seems like a lot of work compared to the sum method, especially when I always had trouble remembering exactly what the arguments to the closure meant.

That changed recently. One of the examples I use when teaching Groovy to Java developers is do some basic sorting. I like that example, because it shows not only how easy it is to replace anonymous inner classes with closures, but also because it shows how much the Groovy JDK simplifies coding.

As a preface to my example, consider making an ArrayList of strings:

def strings = 'this is a list of strings'.split()
assert strings.class == java.lang.String[]

The split method splits the string at spaces by default, and returns, sadly, a string array. What I want is a List, and converting an array into a List is a special blend of Java awkwardness and verbosity, though the code isn’t too bad once you’ve seen it.

The conversion is trivial in Groovy, however.

List strings = 'this is a list of strings'.split()
assert strings.class == java.util.ArrayList

Just replace def with the datatype you want, and Groovy will do its best to do the conversion for you. :)

To sort a list, Java has the various static sort methods in the java.util.Collections class. The sort method with no arguments does the natural, alphabetical (more properly, lexicographical, where there capital letters come before the lowercase letters) sort.

List strings = 'this is a list of strings'.split()
Collections.sort()  // natural sort (alphabetical)
assert strings == ['a', 'is', 'list', 'of', 'strings', 'this']

This sorts the strings in place (a destructive sort) and returns void, so to see the actual sort you have to print it.

How do you test this? I wrote an assert that hardwired the results, because I knew what they had to be. That’s hardly generalizable, however, and this is where inject comes in.

Have you ever looked at the definition of inject in the GroovyDocs? Here it is, from the class org.codehaus.groovy.runtime.DefaultGroovyMethods.

public static T inject(E[] self, U initialValue, @ClosureParams(value=FromString.class,options=”U,E”) Closure closure)

Iterates through the given array, passing in the initial value to the closure along with the first item. The result is passed back (injected) into the closure along with the second item. The new result is injected back into the closure along with the third item and so on until all elements of the array have been used. Also known as foldLeft in functional parlance.

Parameters:
self – an Object[]
initialValue – some initial value
closure – a closure

Returns:
the result of the last closure call

You would be forgiven for being seriously confused right now. I’ve been using Groovy since about 2007 and if I didn’t already know what inject did, I’d be seriously confused, too.

The DefaultGroovyMethods class contains lots of methods that are added to the library at runtime via Groovy metaprogramming. In this case, the inject method is added to collection (the first argument above). The second argument to inject is an initial value. An initial value to what, you say? The third argument to inject is a closure, and it takes two arguments, and the second argument to inject is the initial value of the first argument of the closure. The subsequent values to that argument are the result of the closure. The second argument to the closure is each element of the collection, in turn.

I expect that almost nobody actually read that last paragraph. Or, more likely, you started it and abandoned it somewhere in the middle. I can hardly blame you.

As usual, the indefatigable Mr. Haki comes to the rescue. Here is one of his examples:

(1..4).inject(0) { result, i ->
    println "$result + $i = ${result + i}"
    result + i
}

and the output is:

0 + 1 = 1
1 + 2 = 3
3 + 3 = 6
6 + 4 = 10

The value of result starts at the initial value (here, 0), and is assigned the result of each execution of the closure.

That’s the sort (no pun intended) of example I’d seen before, and because I always associated inject with accumulators, I never actually needed it. After all, the Groovy JDK adds a sum method already.

The key is to note that the value of “result” is actually whatever is returned from the closure (Mr. Haki’s second example illustrates this beautifully — seriously, go read his post). This actually makes it easy to use to test the sort.

Try this out for size:

List strings = 'this is a list of strings'.split()
Collections.sort(strings)
strings.inject('') { prev, curr ->
    println "prev: $prev, curr: $curr"
    assert prev <= curr
    curr  // value of 'prev' during next iteration
}

The result is:

prev: , curr: a
prev: a, curr: is
prev: is, curr: list
prev: list, curr: of
prev: of, curr: strings
prev: strings, curr: this

Since the closure returns the current value, that becomes the value of prev during the next iteration.

Actually, this can be simplified too. As of Groovy 1.8, there’s now an inject method that leaves out the initialization value. When you call it, the first two elements of the collection become the first two arguments of inject. In other words, now I can do this:

List strings = 'this is a list of strings'.split()
Collections.sort(strings)
strings.inject { prev, curr ->
    println "prev: $prev, curr: $curr"
    assert prev <= curr
    curr  // value of 'prev' during next iteration
}

The output now is:

prev: a, curr: is
prev: is, curr: list
prev: list, curr: of
prev: of, curr: strings
prev: strings, curr: this

Sweet. Now I have a real, live use case for inject. :) I promised you more, however. The title of this post refers to Elvis and spaceships, too.

Assume you want to sort the strings by length. In Java, when you can’t modify the class to be sorted (String), you use the two-argument sort method in Collections. The second argument is of type java.util.Comparator, which gives rise to the dreaded anonymous inner class monster:

List strings = 'this is a list of strings'.split()
Collections.sort(strings, new Comparator<String>()) { // R: Holy anonymous inner class, Batman!
    int compare(String s1, String s2) {               // B: Yes, Robin, with generics and everything.
        s1.size() <=> s2.size()                       // R: Gosh, gee, and a spaceship!
    }
})
assert strings == ['a', 'is', 'of', 'this', 'list', 'strings']
assert strings*.size() == [1, 2, 2, 4, 4, 7]          // R: Holy spread-dot operator, too!
                                                      // B: Dude, seriously, get a grip.

The spaceship operator returns -1, 0, or 1 when the left side is less than, equal to, or greater than the right side. It’s like a comparator, except the values are fixed to -1, 0, and 1.

The nice thing about the spread-dot operator here is that it doesn’t care whether the resulting strings are also alphabetical or not. The fact that equals length strings were also sorted alphabetically is just a side effect of the algorithm.

One of the common idioms in the Groovy JDK is to take static methods in Java and make them instance methods in Groovy. Here, the sort method is a static method in the Collections class. The Groovy JDK makes it an instance method in Collection (singular).

List strings = 'this is a list of strings'.split()
strings.sort { s1, s2 -> s1.size() <=> s2.size() }  // R: Holy closure coercion, Batman!
                                                    // B: No, the instance method takes a closure.
                                                    //    Seriously, did you remember your ADHD meds today?
assert strings*.size() == [1, 2, 2, 4, 4, 7]

The sort method in Groovy is now an instance method, which takes a one- or two-argument closure. The two-argument variety is implemented like a traditional comparator, meaning you return negative, zero, or positive as usual.

Even better, the one-argument version of sort says, in effect, transform each element into a number and Groovy will sort the numbers and use that as a way to sort the collection.

List strings = 'this is a list of strings'.split()
strings.sort { it.size() }                     // R: Everything is awesome!
assert strings*.size() == [1, 2, 2, 4, 4, 7]   // B: Shut up kid, or you'll find yourself floating home.

Here’s the best part. What if you want to sort by length, and then sort equal lengths reverse alphabetically? (I’ll use reverse alpha because the length sort also did alphabetical by accident).

Now I can use Elvis and spaceships together:

List strings = 'this is a list of strings'.split()
strings.sort { s1, s2 -> 
    s1.size() <=> s2.size() ?: s2 <=> s1
}

The Elvis operator says if the result is true by the Groovy Truth, use it, else use a default. Here it means do the length comparison and if the result is non-zero, we’re good. Otherwise do the (reverse) alphabetical comparison.

R: So that’s Elvis being carried back to his home planet by two tandem spaceships, right? Meaning it’s the fat Elvis from the 70s and not the thin Elvis from the 50s?
B: BAM! POW! OOF!

Here, finally, is a test based on inject for that sort:

strings.inject { prev, curr ->
    assert prev.size() <= curr.size()
    if (prev.size() == curr.size()) {
        assert prev >= curr
    }
    curr
}

There you have it: Elvis, spaceships, and inject all in a dozen lines of code. Now you add that to your Groovy utility belt.

(B: POW! BAM! OOF!)

Follow

Get every new post delivered to your Inbox.

Join 1,634 other followers

%d bloggers like this: