Excluding Gradle Tasks with a Name Pattern

I’ve been spending a lot of time with Gradle build files in Android projects, which probably isn’t a big surprise given that I’m working on a book called Gradle Recipes for Android (coming soon to all your better ebook stores and (who knows?) maybe an actual, physical bookstore somewhere (but probably not), but you can always get it at O’Reilly or Amazon). In one chapter, I talk about excluding certain tasks in Gradle builds, and that led me to write an interesting custom task.

Gradle builds on Android have tons of tasks, and that number goes up and up when you add new build types or flavors. For example, on a trivial Android project, asking for the list of tasks gives:

> ./gradlew tasks
Starting a new Gradle Daemon for this build (subsequent builds will be faster).
Incremental java compilation is an incubating feature.
:tasks

------------------------------------------------------------
All tasks runnable from root project
------------------------------------------------------------

Android tasks
-------------
androidDependencies - Displays the Android dependencies of the project.
signingReport - Displays the signing info for each variant.
sourceSets - Prints out all the source sets defined in this project.

Build tasks
-----------
assemble - Assembles all variants of all applications and secondary packages.
assembleAndroidTest - Assembles all the Test applications.
assembleDebug - Assembles all Debug builds.
assembleRelease - Assembles all Release builds.
build - Assembles and tests this project.
buildDependents - Assembles and tests this project and all projects that depend on it.
buildNeeded - Assembles and tests this project and all projects it depends on.
clean - Deletes the build directory.
compileDebugAndroidTestSources
compileDebugSources
compileDebugUnitTestSources
compileReleaseSources
compileReleaseUnitTestSources
mockableAndroidJar - Creates a version of android.jar that's suitable for unit tests.

Build Setup tasks
-----------------
init - Initializes a new Gradle build. [incubating]
wrapper - Generates Gradle wrapper files. [incubating]

Help tasks
----------
buildEnvironment - Displays all buildscript dependencies declared in root project 'HelloWorldAS'.
components - Displays the components produced by root project 'HelloWorldAS'. [incubating]
dependencies - Displays all dependencies declared in root project 'HelloWorldAS'.
dependencyInsight - Displays the insight into a specific dependency in root project 'HelloWorldAS'.
help - Displays a help message.
model - Displays the configuration model of root project 'HelloWorldAS'. [incubating]
projects - Displays the sub-projects of root project 'HelloWorldAS'.
properties - Displays the properties of root project 'HelloWorldAS'.
tasks - Displays the tasks runnable from root project 'HelloWorldAS' (some of the displayed tasks may belong to subprojects).

Install tasks
-------------
installDebug - Installs the Debug build.
installDebugAndroidTest - Installs the android (on device) tests for the Debug build.
uninstallAll - Uninstall all applications.
uninstallDebug - Uninstalls the Debug build.
uninstallDebugAndroidTest - Uninstalls the android (on device) tests for the Debug build.
uninstallRelease - Uninstalls the Release build.

Verification tasks
------------------
check - Runs all checks.
connectedAndroidTest - Installs and runs instrumentation tests for all flavors on connected devices.
connectedCheck - Runs all device checks on currently connected devices.
connectedDebugAndroidTest - Installs and runs the tests for debug on connected devices.
deviceAndroidTest - Installs and runs instrumentation tests using all Device Providers.
deviceCheck - Runs all device checks using Device Providers and Test Servers.
lint - Runs lint on all variants.
lintDebug - Runs lint on the Debug build.
lintRelease - Runs lint on the Release build.
test - Run unit tests for all variants.
testDebugUnitTest - Run unit tests for the debug build.
testReleaseUnitTest - Run unit tests for the release build.

Other tasks
-----------
jarDebugClasses
jarReleaseClasses
transformResourcesWithMergeJavaResForDebugUnitTest
transformResourcesWithMergeJavaResForReleaseUnitTest

To see all tasks and more detail, run gradlew tasks --all

To see more detail about a task, run gradlew help --task <task>

BUILD SUCCESSFUL

That’s about 50 tasks, and I haven’t added anything yet.

Android projects also include variants, which are combinations of build types and flavors. A build type describes whether you want to use debug or release configuration or define one of your own. Flavors allow you to build multiple similar applications that vary only in look and feel or minor code changes.

For example, in my giant Hello, World example (the same one I used in my O’Reilly video courses Learning Android and Practical Android), I use just the debug and release build types, but I have three flavors: arrogant, friendly, and obsequious.

Obsequious is such a good word. I rarely get to use it, though probably for good reason. If you’re trying to remember what it means, think Dobby from the Harry Potter novels:

Obsequious-Welcome

Under those circumstances, the number of tasks increases considerably:

> ./gradlew tasks
Incremental java compilation is an incubating feature.
:tasks

------------------------------------------------------------
All tasks runnable from root project
------------------------------------------------------------

Android tasks
-------------
androidDependencies - Displays the Android dependencies of the project.
signingReport - Displays the signing info for each variant.
sourceSets - Prints out all the source sets defined in this project.

Build tasks
-----------
assemble - Assembles all variants of all applications and secondary packages.
assembleAndroidTest - Assembles all the Test applications.
assembleArrogant - Assembles all Arrogant builds.
assembleDebug - Assembles all Debug builds.
assembleFriendly - Assembles all Friendly builds.
assembleObsequious - Assembles all Obsequious builds.
assembleRelease - Assembles all Release builds.
build - Assembles and tests this project.
buildDependents - Assembles and tests this project and all projects that depend on it.
buildNeeded - Assembles and tests this project and all projects it depends on.
clean - Deletes the build directory.
compileArrogantDebugAndroidTestSources
compileArrogantDebugSources
compileArrogantDebugUnitTestSources
compileArrogantReleaseSources
compileArrogantReleaseUnitTestSources
compileFriendlyDebugAndroidTestSources
compileFriendlyDebugSources
compileFriendlyDebugUnitTestSources
compileFriendlyReleaseSources
compileFriendlyReleaseUnitTestSources
compileObsequiousDebugAndroidTestSources
compileObsequiousDebugSources
compileObsequiousDebugUnitTestSources
compileObsequiousReleaseSources
compileObsequiousReleaseUnitTestSources
mockableAndroidJar - Creates a version of android.jar that's suitable for unit tests.

Build Setup tasks
-----------------
init - Initializes a new Gradle build. [incubating]
wrapper - Generates Gradle wrapper files. [incubating]

Help tasks
----------
buildEnvironment - Displays all buildscript dependencies declared in root project 'HelloWorldAS'.
components - Displays the components produced by root project 'HelloWorldAS'. [incubating]
dependencies - Displays all dependencies declared in root project 'HelloWorldAS'.
dependencyInsight - Displays the insight into a specific dependency in root project 'HelloWorldAS'.
help - Displays a help message.
model - Displays the configuration model of root project 'HelloWorldAS'. [incubating]
projects - Displays the sub-projects of root project 'HelloWorldAS'.
properties - Displays the properties of root project 'HelloWorldAS'.
tasks - Displays the tasks runnable from root project 'HelloWorldAS' (some of the displayed tasks may belong to subprojects).

Install tasks
-------------
installArrogantDebug - Installs the DebugArrogant build.
installArrogantDebugAndroidTest - Installs the android (on device) tests for the ArrogantDebug build.
installFriendlyDebug - Installs the DebugFriendly build.
installFriendlyDebugAndroidTest - Installs the android (on device) tests for the FriendlyDebug build.
installObsequiousDebug - Installs the DebugObsequious build.
installObsequiousDebugAndroidTest - Installs the android (on device) tests for the ObsequiousDebug build.
uninstallAll - Uninstall all applications.
uninstallArrogantDebug - Uninstalls the DebugArrogant build.
uninstallArrogantDebugAndroidTest - Uninstalls the android (on device) tests for the ArrogantDebug build.
uninstallArrogantRelease - Uninstalls the ReleaseArrogant build.
uninstallFriendlyDebug - Uninstalls the DebugFriendly build.
uninstallFriendlyDebugAndroidTest - Uninstalls the android (on device) tests for the FriendlyDebug build.
uninstallFriendlyRelease - Uninstalls the ReleaseFriendly build.
uninstallObsequiousDebug - Uninstalls the DebugObsequious build.
uninstallObsequiousDebugAndroidTest - Uninstalls the android (on device) tests for the ObsequiousDebug build.
uninstallObsequiousRelease - Uninstalls the ReleaseObsequious build.

Verification tasks
------------------
check - Runs all checks.
connectedAndroidTest - Installs and runs instrumentation tests for all flavors on connected devices.
connectedArrogantDebugAndroidTest - Installs and runs the tests for arrogantDebug on connected devices.
connectedCheck - Runs all device checks on currently connected devices.
connectedFriendlyDebugAndroidTest - Installs and runs the tests for friendlyDebug on connected devices.
connectedObsequiousDebugAndroidTest - Installs and runs the tests for obsequiousDebug on connected devices.
deviceAndroidTest - Installs and runs instrumentation tests using all Device Providers.
deviceCheck - Runs all device checks using Device Providers and Test Servers.
lint - Runs lint on all variants.
lintArrogantDebug - Runs lint on the ArrogantDebug build.
lintArrogantRelease - Runs lint on the ArrogantRelease build.
lintFriendlyDebug - Runs lint on the FriendlyDebug build.
lintFriendlyRelease - Runs lint on the FriendlyRelease build.
lintObsequiousDebug - Runs lint on the ObsequiousDebug build.
lintObsequiousRelease - Runs lint on the ObsequiousRelease build.
test - Run unit tests for all variants.
testArrogantDebugUnitTest - Run unit tests for the arrogantDebug build.
testArrogantReleaseUnitTest - Run unit tests for the arrogantRelease build.
testFriendlyDebugUnitTest - Run unit tests for the friendlyDebug build.
testFriendlyReleaseUnitTest - Run unit tests for the friendlyRelease build.
testObsequiousDebugUnitTest - Run unit tests for the obsequiousDebug build.
testObsequiousReleaseUnitTest - Run unit tests for the obsequiousRelease build.

Other tasks
-----------
jarArrogantDebugClasses
jarArrogantReleaseClasses
jarFriendlyDebugClasses
jarFriendlyReleaseClasses
jarObsequiousDebugClasses
jarObsequiousReleaseClasses
transformResourcesWithMergeJavaResForArrogantDebugUnitTest
transformResourcesWithMergeJavaResForArrogantReleaseUnitTest
transformResourcesWithMergeJavaResForFriendlyDebugUnitTest
transformResourcesWithMergeJavaResForFriendlyReleaseUnitTest
transformResourcesWithMergeJavaResForObsequiousDebugUnitTest
transformResourcesWithMergeJavaResForObsequiousReleaseUnitTest

To see all tasks and more detail, run gradlew tasks --all

To see more detail about a task, run gradlew help --task

BUILD SUCCESSFUL

That’s just under 100, and the problem only gets worse if you add flavor dimensions. In the book, I add client flavors — one for Wayne Enterprises and one for Stark Industries. That gives me 3 x 2 = 6 different flavors and 2 build types, or 12 different variants, with all the (nearly 200) associated tasks. Whew.

Wayne-Help

Here’s a sample of the build file, just to show what this looks like:

android {
    compileSdkVersion 23
    buildToolsVersion "23.0.3"
    defaultConfig {
        applicationId "com.kousenit.helloworld"
        minSdkVersion 16
        targetSdkVersion 23
        versionCode 1
        versionName "1.0"
    }
    buildTypes {
        // no changes to debug type, so no need to list it here
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'),
               'proguard-rules.pro'
        }
    }

    flavorDimensions 'attitude', 'client'

    productFlavors {
        arrogant {
            dimension 'attitude'
            applicationId 'com.kousenit.helloworld.arrg'
        }
        friendly {
            dimension 'attitude'
            applicationId 'com.kousenit.helloworld.frnd'
        }
        obsequious {
            dimension 'attitude'
            applicationId 'com.kousenit.helloworld.obsq'
        }
        stark {
            dimension 'client'
        }
        wayne {
            dimension 'client'
        }
    }
}

Say I want to skip a task. For example, when I’m doing a regular build, I don’t always need to run the lint task, which gives interesting results but takes time. In Gradle, excluding a particular task from the build is as simple as using the -x flag.

That sounds good, but unfortunately there are many lint tasks:

> ./gradlew tasks | grep lint
lint - Runs lint on all variants.
lintArrogantStarkDebug - Runs lint on the ArrogantStarkDebug build.
lintArrogantStarkRelease - Runs lint on the ArrogantStarkRelease build.
lintArrogantWayneDebug - Runs lint on the ArrogantWayneDebug build.
lintArrogantWayneRelease - Runs lint on the ArrogantWayneRelease build.
lintFriendlyStarkDebug - Runs lint on the FriendlyStarkDebug build.
lintFriendlyStarkRelease - Runs lint on the FriendlyStarkRelease build.
lintFriendlyWayneDebug - Runs lint on the FriendlyWayneDebug build.
lintFriendlyWayneRelease - Runs lint on the FriendlyWayneRelease build.
lintObsequiousStarkDebug - Runs lint on the ObsequiousStarkDebug build.
lintObsequiousStarkRelease - Runs lint on the ObsequiousStarkRelease build.
lintObsequiousWayneDebug - Runs lint on the ObsequiousWayneDebug build.
lintObsequiousWayneRelease - Runs lint on the ObsequiousWayneRelease build.

Excluding lint leaves out some of them, but runs others.

> ./gradlew build -x lint | grep lint
:app:lintVitalArrogantStarkRelease
:app:lintVitalArrogantWayneRelease
:app:lintVitalFriendlyStarkRelease
:app:lintVitalFriendlyWayneRelease
:app:lintVitalObsequiousStarkRelease
:app:lintVitalObsequiousWayneRelease

I’m not sure what the “vital” part of those release tasks is, but I don’t want it. I suppose I could try excluding the tasks one by one, but that’s starting to feel like a lot of work.

Instead, I can add the following to the build.gradle file, which waits for the task graph to be assembled and then removes the undesired name pattern.

gradle.taskGraph.whenReady { graph ->
    graph.allTasks.findAll { it.name ==~ /lint.*/ }*.enabled = false
}

Gradle assembles a directed acyclic graph of tasks, available through the gradle object via its taskGraph property. By calling the whenReady method, I wait until that graph is assembled before modifying it.

The whenReady method takes a closure, whose argument is the graph. I retrieve all the tasks into a list, find all the tasks whose name matches the given regex (meaning the name starts with the letters lint), and disable them all.

> ./gradlew build | grep lint
:app:lintVitalArrogantStarkRelease SKIPPED
:app:lintVitalArrogantWayneRelease SKIPPED
:app:lintVitalFriendlyStarkRelease SKIPPED
:app:lintVitalFriendlyWayneRelease SKIPPED
:app:lintVitalObsequiousStarkRelease SKIPPED
:app:lintVitalObsequiousWayneRelease SKIPPED
:app:lint SKIPPED

This works, but it’s a permanent solution to a temporary problem. I’d rather make excluding those tasks optional. Fortunately, I can do that through a project property.

gradle.taskGraph.whenReady { graph ->
    if (project.hasProperty('noLint')) {
        graph.allTasks.findAll { it.name ==~ /lint.*/ }*.enabled = false
    }
}

Now I can exclude the lint tasks by specifying a -P flag on the command line:

> ./gradlew build -PnoLint | grep lint
:app:lintVitalArrogantStarkRelease SKIPPED
:app:lintVitalArrogantWayneRelease SKIPPED
:app:lintVitalFriendlyStarkRelease SKIPPED
:app:lintVitalFriendlyWayneRelease SKIPPED
:app:lintVitalObsequiousStarkRelease SKIPPED
:app:lintVitalObsequiousWayneRelease SKIPPED
:app:lint SKIPPED

This strikes me as a clean, elegant solution to the problem, but maybe only because I can’t think of anything easier. If you can, please let me know, because I turned in the complete draft of the stupid book this week (!!) and this is in one of the chapters. If you find an error or a better idea, there’s still (barely) enough time to update it and even give you credit (if not necessarily cashy money, though I will be happy to purchase for you the libation of your choice next time I see you).

Either way, the idea of manipulating the task graph inside the build file is a really useful one, so you shouldn’t exclude it (get it?).

Retrofitting Groovy

I’m teaching an Android development class this week, and one of our primary references is the book Android 6 for Programmers, 3rd edition, which was released last December. One of the examples in the book accesses the Open Weather Map RESTful web service and builds a UI around the results, which is pretty much the default Android developer app.

The app accesses Open Weather Map by creating an instance of the URL class, invoking openConnection on the result, and downloading the response using the resulting InputStream. It then parses the response using various classes in the org.json package, including JsonObject and JsonArray.

As you might imagine, this is a tedious way to solve the problem. Java is already verbose; adding Android makes it worse, and then doing networking and JSON parsing “by hand” is just too much. As a teaching example it’s fine, but I wouldn’t recommend that as a long-term solution.

For RESTful web services, I’ve been a fan of the Spring for Android project, which includes a class called RestTemplate that has a method called getForObject. Once you map a set of Java classes to the expected JSON response, accessing the web service becomes a simple one-liner. Much better.

The problem, however, is that the Spring for Android project is now dormant to the point of being inactive. The 1.0.1 release is dated December, 2012, and the 2.0.0 M3 milestone hasn’t changed in years. That makes me reluctant to keep recommending it to new Android developers.

Instead, the primary library for working with RESTful services in Android appears to be Retrofit, from Square. It’s very powerful and current, and the only problem is that the documentation is, shall we say, thin.

I wanted to show the students in my class how to rewrite the book app to use Retrofit instead of doing the low-level networking and JSON parsing. That meant I had to experiment with the library, which is something I’d been planning to do for years but never actually did. The good news is that Retrofit can be used in a stand-alone Java app, so I could try it out myself before worrying about the Android aspects of the problem.

As often happens, that lead me to Groovy. Most Groovy apps are combinations of both Groovy and Java, and I like to say that while Java is good for tools, libraries, and basic infrastructure, Groovy is good for everything else. While it’s unlikely I can convince my students to use Groovy in their apps (it’s a very conservative company), I could certainly use it myself during my learning process.

The book code eventually produced a Java class called Weather, used to hold formatted strings for the day of the week, the min and max temperatures forecasted for that day, the humidity percent, a String description of the weather, and a URL to an icon showing the weather (sunny, cloudy, or whatever). My goal was to use Retrofit to access the Open Weather Map API, download the resulting JSON response, convert it to classes, and then create an instance of Weather for each of the forecast days.

First I created a new Gradle-based project that allowed me to mix Java and Groovy together. Here’s the build file, showing the Retrofit dependencies.

apply plugin: 'groovy'

sourceCompatibility = 1.8

repositories {
    jcenter()
}

dependencies {
    compile 'org.codehaus.groovy:groovy-all:2.4.6'
    compile 'com.squareup.retrofit2:retrofit:2.0.1'
    compile 'com.squareup.retrofit2:converter-gson:2.0.1'

    testCompile 'junit:junit:4.12'
}

I’m using the Gson converter, which automatically converts the JSON response to a set of classes once I’ve defined them.

Step 1 in any mapping operation is to look at the form of the JSON response. Here’s an abbreviated sample, from http://api.openweathermap.org/data/2.5/forecast/daily?q=Marlborough,CT&units=imperial&cnt=16&APPID=d82ee6zzzzzzz .

{"city":{"id":4844078,"name":"Terramuggus","coord":{"lon":-72.47036,"lat":41.635101},"country":"US","population":0},"cod":"200","message":0.0156,"cnt":16,"list":[{"dt":1460044800,"temp":{"day":51.48,"min":49.69,"max":51.48,"night":49.69,"eve":51.48,"morn":51.48},"pressure":984.62,"humidity":97,"weather":[{"id":501,"main":"Rain","description":"moderate rain","icon":"10d"}],"speed":9.98,"deg":170,"clouds":88,"rain":3.8}, { ... }, ... ]}

After the basic info, there is an array of 16 JSON objects representing the data I need, one for each day. (Note: to do this yourself, you’ll need to replace the APPID with your own, which you can get at the Open Weather Map site.)

Working top down, here is the set of POGOs (Plain Old Groovy Objects) I created to map to just the few parts I needed:

class Model {
    WeatherData[] list
}

class WeatherData {
    long dt
    TempData temp
    int humidity
    WeatherInfo[] weather
}

class TempData {
    double min
    double max
}

class WeatherInfo {
    String description
    String icon
}

To use Retrofit, I did what I normally do, which is to write a Groovy script and then eventually turn it into a class. That makes it easy to integrate with existing Java classes. Here’s the class I eventually created:

import retrofit2.Call
import retrofit2.Retrofit
import retrofit2.converter.gson.GsonConverterFactory

class DownloadForecast {
    private static final String KEY = 'd82ee6...'

    private final Retrofit retrofit = new Retrofit.Builder()
            .addConverterFactory(GsonConverterFactory.create())
            .baseUrl('http://api.openweathermap.org')
            .build()

    List<Weather> getWeatherList(String city='Marlborough', String state='CT') {
        OpenWeatherMap owm = retrofit.create(OpenWeatherMap)
        String address = "${URLEncoder.encode(city, 'UTF-8')},$state"
        Call<Model> model = owm.getData(q: address, units: 'imperial',
                cnt: '16', APPID: KEY)

        model.execute().body().list.collect { WeatherData wd ->
            Weather.parseData(wd)
        }
    }
}

I made both attributes private and final because I didn’t want Groovy to auto-generate and getters or setters for them. The instance of Retrofit is created using a builder, with its fluent syntax, in the recommended manner.

The getWeatherList method takes two strings representing the city and state. I gave both defaults (cool that you can do that in Groovy, isn’t it?), so I can invoke this method with zero, one, or two arguments, as the test cases will show.

The next requirement for Retrofit is that you provide an interface with the methods you want to invoke. In this case I called it OpenWeatherMap:

import retrofit2.Call;
import retrofit2.http.GET;
import retrofit2.http.QueryMap;

import java.util.Map;

public interface OpenWeatherMap {
    @GET("data/2.5/forecast/daily")
    Call<Model> getData(@QueryMap Map<String, String> params);
}

While I could have written that in Groovy, in this case I provided it in Java, just to make the integration cleaner. The GET annotation shows that relative to the base URL I need to access the given path, and the QueryMap annotation is applied to a map of parameters used to form the resulting query string. The return type is a Call.

Returning to the getWeatherList method, I used the create method on retrofit to return an implementation of OpenWeatherMap. Then to make the actual call, I need to invoke the execute method using my map of parameters. Groovy makes that part particularly easy:

Call<Model> model = owm.getData(q: address, units: 'imperial', cnt: '16', APPID: KEY)

That uses the normal Groovy native syntax for maps. You’ll note that I URL encoded the city when assembling the address, using the normal (Java) URLEncoder class in the standard library.

Once I executed the call, I traversed to the list child element, based on the attribute name used in the JSON response. That gave me my collection of WeatherData objects.

Then I needed to map the WeatherData class to my desired Weather class, which I did through a static method called parseData in Weather.

import groovy.transform.ToString
import java.text.NumberFormat

@ToString
class Weather {
    final static NumberFormat numberFormat = NumberFormat.instance
    final static NumberFormat percentFormat = NumberFormat.percentInstance

    String day
    String min
    String max
    String humidity
    String description
    URL iconURL

    static Weather parseData(WeatherData data) {
        numberFormat.setMaximumFractionDigits(2)

        new Weather(day: new Date(data.dt * 1000).format('EEEE'),
            min: numberFormat.format(data.temp.min) + '\u00B0F',
            max: numberFormat.format(data.temp.max) + '\u00B0F',
            humidity: percentFormat.format(data.humidity / 100),
            description: data.weather[0].description,
            iconURL: "http://openweathermap.org/img/w/${data.weather[0].icon}.png".toURL()
        )
    }
}

That (almost) matches the Java Weather POJO in the book, which I populated from the WeatherData values. The last line in the getWeatherList method:

model.execute().body().list.collect { WeatherData wd ->
    Weather.parseData(wd)
}

converts the array of WeatherData objects into a collection of Weather objects and returns it.

To make sure this is working, here’s my test case:

import org.junit.Test;
import java.util.List;

import static org.hamcrest.CoreMatchers.equalTo;
import static org.junit.Assert.*;

public class DownloadForecastTest {
    private DownloadForecast df = new DownloadForecast();

    @Test  // default city,state is Marlborough,CT
    public void getWeatherList_MarlboroughCT() throws Exception {
        List<Weather> weatherList = df.getWeatherList();
        assertThat(16, equalTo(weatherList.size()));
        System.out.println("Today's weather: " + weatherList.get(0));
    }

    @Test // specify just city defaults to state of CT
    public void getWeatherList_NewLondonCT() throws Exception {
        List<Weather> weatherList = df.getWeatherList("New London");
        assertThat(16, equalTo(weatherList.size()));
        System.out.println("Today's weather: " + weatherList.get(0));
    }

    @Test // the weather has got to be better in Honolulu
    public void getWeatherList_HonoluluHI() throws Exception {
        List<Weather> weatherList = df.getWeatherList("Honolulu", "HI");
        assertThat(16, equalTo(weatherList.size()));
        System.out.println("Today's weather: " + weatherList.get(0));    }
}

I used Java to write the test, mostly to demonstrate that I can access the Groovy classes from Java without any issues. All I’m testing is that I get 16 Weather objects in the results, as I expected (because of the supplied value of the cnt parameter). The printed output shows today’s weather in each location.

Today's weather: Weather(Thursday, 77.38°F, 79.36°F, 97%, scattered clouds, http://openweathermap.org/img/w/03n.png)
Today's weather: Weather(Thursday, 46.44°F, 48°F, 90%, moderate rain, http://openweathermap.org/img/w/10d.png)
Today's weather: Weather(Thursday, 49.69°F, 51.48°F, 97%, moderate rain, http://openweathermap.org/img/w/10d.png)

The first result is for Honolulu; the other two are in Connecticut. In other words, April hasn’t really made it’s way to Connecticut yet.

Now that the system is working, the next step would be to port everything to Java and add it to the Android app, making the REST call in an AsyncTask and so on. After coding in Groovy, however, the idea of porting all that easy code back into Java is just depressing, so I decided to blog about it instead.

Rough cut of Gradle Recipes for Android now available

My latest book, Gradle Recipes for Android, is now available as a “Rough Cut” at O’Reilly. You can get it at http://shop.oreilly.com/product/0636920032656.do.

gradle-recipes-for-android

Rough cuts are preliminary versions of O’Reilly books, which are released while a book is still in progress, without special effort taken for formatting or anything else.

In this case, however, the rough cut is pretty close to the final version. Recipe books at O’Reilly are like their cookbooks, only shorter. My book has about 27 recipes, which are short discussions of how to do specific tasks, in this case involving the Gradle build tool with the Android plugin, used to build Android applications. The book also contains information about how to use the only officially supported IDE, Android Studio.

The O’Reilly authoring system supports Asciidoc, which is wonderful. I found it much, much easier to write my book using Asciidoc, commit it to their git repository, and then generate the resulting pdf afterwards. The book is available in Safari Books Online, too, which show it formatted for HTML. The results are very nice, and allow you to copy and paste code from the browser to your editor of choice. For the record, I wrote the book using the Atom editor from GitHub, which has a nice Asciidoc preview mode.

The book is basically finished. I’m doing some editing and considering adding a recipe or two. If you have any suggestions, however, please feel free to send them along. I hope to complete the book in the next week or so, so it will be available in print form just in time for the holidays.:)

I’ve also been doing a lot of video course recording for O’Reilly over the past few months. So far the courses available are:

All the courses are available for sale, or on Safari. The last one (Gradle for Android) is similar to the book, but the book has more depth, a different approach, and lots of reference information. The book has appendices on Groovy and Gradle, while the video summarizes both topics.

I should mention in the midst of all this shameless self-promotion that there is also a Packt book called Gradle for Android by Kevin Pelgrims. Packt books vary wildly in quality, but Kevin’s book is excellent. In fact, it was so good, that (combined with the fact I’d already coincidently recorded a video course of the same name) I decided I needed to switch my book to the recipe style. His book is much more the traditional exposition, with chapters and depth. Mine is more the “here’s a problem, now here’s a solution” style. I think both books are complementary. At least I hope so.

On an unrelated note, I now have a revised home page, http://www.kousenit.com. It’s about time.

I hope you enjoy the book and/or video courses, or even just my new home page. As always, any errors or omissions are entirely my responsibility. I’m just happy to be making all this content available. Now I have to finish this weekend’s No Fluff, Just Stuff conference in Boston, travel to Raleigh for a Groovy course, and then get in line to see The Martian.

Upcoming Events, and “The Streak”

I’m really not a workaholic. I prefer days off as much as anybody. The problem is that there are all these things I want to do, so I volunteer to do them, and suddenly I’m overbooked so much I don’t have time for a break.

I think part of it comes down to an acronym that’s making the rounds these days. It’s called FOMO, and stands for Fear Of Missing Out. I suffer from it as much as anybody, I guess. I’ll need to work on that.

At any rate, I’m beginning a particularly busy part of my calendar, and I thought I’d mention it here in case I have the opportunity to meet any of you along the way.

On Tuesday, May 26, I’m giving my Groovy Vampires talk at the Austin Java Users Group. I really have a lot of fun with that talk. I plan to update it a bit, especially because a recording of the version I gave at last year’s SpringOne 2GX conference is currently available on InfoQ.

To give you a clue what it’s about, one day I was wandering in a Barnes and Noble and noticed that there was only one book shelf labeled “Computer”:

ComputerBooks

while there were three (yes, three) labeled “Teen Paranormal Romance”:

TeenParanormalRomance

Instead of lamenting the decline of western civilization, I decided that what this meant was that I needed to add some Groovy vampires to my book. The talk is based on that, and involves accessing RESTful web services, parsing JSON data, using a MongoDB database, creating a RESTful service using ratpack and Grails, and more.

The next day, May 27, I’ll be speaking at the Austin Groovy and Grails Users Group, this time on Testing Grails. While the bulk of the talk was created using Grails 2.3 and works under 2.4, I do plan to say a few words about Grails 3.0 as well. The testing mechanisms haven’t changed much in going from 2.3 to 2.4 to 3.0, though now they’ve added Geb as an included dependency, so you can make functional tests easily.

If you’re in the Austin, TX area this week, please drop by and say hi.

Also this week, starting tomorrow I’ll be hanging out at JavaRanch, because my Groovy Fundamentals video course will be featured in the Groovy forum this week. I’ve always been a fan of the ranch. I first joined way back in the late 90’s, when I was getting Java certified for the first time.

Speaking of the Groovy Fundamentals video (did I really need to link to that again? Yes. Yes I did), that’s the first of three Groovy video courses I recorded for O’Reilly recently. It covers the basics, from POGOs to collections to closures, in almost exactly four hours.

The second video course in the series is called “Practical Groovy”, and covers topics like integrating Java and Groovy, runtime metaprogramming, using AST transformations, the Groovy SQL class, and more. That one ought to go live within a week.

The third video in the series is called “Mastering Groovy”. It covers Spock testing (though I use Spock tests in the earlier videos), working with Spring, JPA, and NoSQL databases, functional programming, traits, and RESTful web services, among other topics. That one will go live when I finally finish reviewing all the video segments. It’s rather dismaying to discover that reviewing six hours of video takes at least six hours.

Though I must admit I’m tired of watching and listening to myself, I’m very proud of these videos and hope they’ll be helpful. I used to joke about selling the movie rights to Making Java Groovy, and speculate about who would I would recommend to play me.

(Best suggestion: Vincent D’Onofrio from his Men In Black days; most definitely not his Kingpin days.)

In essence, these video courses are that movie. They capture pretty much everything I’ve learned about Groovy over the years, condensed to an easily digestible form. If you’ve ever seen me give Groovy talks at No Fluff, Just Stuff conferences, you’ve seen a lot of what I’ve included, though with fewer jokes.

That brings me to next weekend, which is the Dallas NFJS event. I’ll be giving a Spock talk, and my “Managing Your Manager” talk, and a series on Android development. Again, if you’re in the neighborhood, please drop by and say hi.

Incidentally, some time this week (tentatively Wednesday, 5/27), Peter Ledbrook, Baruch Sadogursky, and I are planning to do another Groovy Podcast. I love doing those, so assuming that happens I’ll definitely tweet about it.

From Dallas I’m off to San Diego, where I’ll be teaching a (private) Groovy and Grails class. Teaching technical training classes is What I Do, practically my raison d’être, so feel free to contact me for details.

After the Grails class I’m heading to Sebastopol, CA, home of O’Reilly, to get back into the recording studio again. This time I’m working on a couple of Android videos, and if I manage to finish those I’ll also try to record something on Gradle for Android. That will all be the same week that culminates in the Gradle Summit in Santa Clara, where I’m doing an updated talk on the Gradle plugin for Android. I’m really looking forward to that conference, though I may miss the first day if we’re still recording.

(Yes, I’m making progress on the Gradle for Android book. Yes, it would be a lot easier if the Android plugin for Gradle, Android Studio, and even Android itself stopped changing so much so frequently, causing me to have to constantly rewrite chapters. Yes, the video course will be related, and will help me write the book. Yes, I’ll probably scowl if you ask me for an actual release date, but don’t let that stop you.)

When the Gradle Summit ends, I finally get to go home again, at least for a few hours, before I’m headed to South Carolina for another training class. I might have another one after that, too, but I haven’t decided.

Eventually I’m going to need a break, so it’s a good thing I scheduled one. Next year (!) my wife and I decided to go on the JoCo Cruise in the Caribbean, which is a sweet nerd cruise featuring Jonathan Coulton, Wil Wheaton, Paul and Storm, and many others. That really ought to be fun.

Finally, I need to say something about “The Streak”. Like many people in the I.T. field, I was a career changer. I came from engineering. More accurately, I should say that I was a research scientist, specializing in the field of turbomachinery aeroacoustics. What that really meant was I did lots and lots of math and wrote lots and lots of Fortran (the nightmares have ended, but it took a while). Ultimately I joined an AI group, went back to school at night, got my MS in CS and decided to leave my job.

My new job as a technical trainer started May 31, 2000. That day I helped teach a training course in Object Oriented Concepts (remember when did stuff like that?). I spent five years teaching classes (including roughly one Intro Java course a month back then) before I went out on my own in March of 2005.

We’re coming up on May 31, 2015, and in all that time, I have never missed a day of training. Not one. I call that The Streak, and since it looks like I’m going to make it to that date I figured it was okay to announce it here.

That journey, and the life changes that accompanied and preceded it, deserve their own blog post. I didn’t want the date to pass, though, without mentioning it. I’m rather inordinately proud of The Streak. Some of it is certainly luck, and it can’t last forever, but it means a lot to me. Right now my job and my career are everything I ever dreamed they could be, and I think The Streak is a side-effect. At the very least, it always gets me up in the morning.:)

Concurrent Kitties Using GPars

On today’s Groovy Podcast, I mentioned that I was teaching a Groovy training class this week at Research Now in Plano, TX. That’s not how I said it, though. I said that I was broadcasting live in front of a studio audience and that they were the most amazingly awesome group I’ve ever encountered.

(Yeah, okay, sometimes I shamelessly pander for good evals. I’ll let you know if it worked after the class ends. Unless it doesn’t, in which case I probably won’t.)

During the podcast, I told my inestimable co-host, Peter Ledbrook, that we got a chance to use GPars in class. The app we used it on was all about the primary goal of the internet, which is to display cat pictures.

Peter then shamed me into writing a blog post about it, which you’re reading now.

I’ve actually written about this app before, for another client. My post there was originally entitled, “The Reason The Internet Was Invented, Or Cat Pictures FTW”, but the host changed it to the far more mundane Calling RESTful Services in Groovy.

The basic idea is that Flickr (remember them? Me neither) has a RESTful API that lets you search for photos. The “flickr.photos.search” request doesn’t require authentication, but does require a whole set of query parameters, including an API key.

Funny story: in order to get a Flickr API key, you actually have to register at Yahoo! Remember them, too? Yeah, neither did I.

At any rate, I registered and got my key, so I can now do the searches. Here’s the start of my Groovy script to do it:

import groovy.json.*

String key = new File('flickr_key.txt').text
String endPoint = 'https://api.flickr.com/services/rest?'
def params = [method        : 'flickr.photos.search',
              api_key       : key,
              format        : 'json',
              tags          : 'kitty',
              nojsoncallback: 1,
              media         : 'photos',
              per_page      : 6]

// Build URL and download JSON data
String qs = params.collect { it }.join('&')
String jsonTxt = "$endPoint$qs".toURL().text

The query string is constructed from the map of params by running a collect on each element (which returns key=value for each Map.Entry) and then joining the resulting list with an ampersand. Notice the tags key was assigned to the word “kitty”.

The next part of my script writes out the results and appends them to a file.

// write formatted JSON data to file
File f = new File('cats.json')
if (f) f.delete()
f << JsonOutput.prettyPrint(jsonTxt)
println JsonOutput.prettyPrint(jsonTxt)

Here’s a sample formatted JSON response:

{
    "photos": {
        "page": 1,
        "pages": 127979,
        "perpage": 6,
        "total": "767873",
        "photo": [
            {
                "id": "17418175405",
                "owner": "31469819@N02",
                "secret": "9055856685",
                "server": "5453",
                "farm": 6,
                "title": "A Ghostly Cat",
                "ispublic": 1,
                "isfriend": 0,
                "isfamily": 0
            },
            {
                "id": "16795470464",
                "owner": "95966544@N07",
                "secret": "cc4af0d44f",
                "server": "8799",
                "farm": 9,
                "title": "Looking for a home",
                "ispublic": 1,
                "isfriend": 0,
                "isfamily": 0
            },
            {
                "id": "17228164988",
                "owner": "92936362@N06",
                "secret": "d42c68bbf3",
                "server": "8734",
                "farm": 9,
                "title": "peaches the cat",
                "ispublic": 1,
                "isfriend": 0,
                "isfamily": 0
            },
            {
                "id": "17208304157",
                "owner": "102705402@N02",
                "secret": "582fff8f44",
                "server": "8688",
                "farm": 9,
                "title": "This is the sweetest cat in the world!",
                "ispublic": 1,
                "isfriend": 0,
                "isfamily": 0
            },
            {
                "id": "17228717179",
                "owner": "37561081@N07",
                "secret": "eb8d0119fe",
                "server": "7722",
                "farm": 8,
                "title": "\u65e9\u5b89",
                "ispublic": 1,
                "isfriend": 0,
                "isfamily": 0
            },
            {
                "id": "17388635206",
                "owner": "127041099@N08",
                "secret": "6310c6012a",
                "server": "7745",
                "farm": 8,
                "title": "Tsim Tung Brother Cream (\u5c16\u6771\u5fcc\u5ec9\u54e5)",
                "ispublic": 1,
                "isfriend": 0,
                "isfamily": 0
            }
        ]
    },
    "stat": "ok"
}

Note that nowhere in the various photo elements do you find a URL for the actual image. It turns out that to assemble the image you have to plug various pieces of the photo elements into a string, which is something Groovy is good at. First, however, I have to parse this and grab the photo elements:

// parse JSON data and build URL for pictures
def json = new JsonSlurper().parseText(jsonTxt)
def photos = json.photos.photo

The photos variable is now a list of maps for each photo, which I can transform into URLs using a collect:

def images = photos.collect { p ->
    String url =
        "http://farm${p.farm}.staticflickr.com/${p.server}/${p.id}_${p.secret}.jpg"
    url.toURL().bytes
}

The Groovy string uses the farm, server, id, and secret elements of the response in each photo and builds a complete URL for the JPG image. Then I convert that to an actual URL and call getBytes() to return byte arrays.

I can then use a SwingBuilder to assemble a trivial GUI showing all the images:

// build UI using Swing
new SwingBuilder().edt {
    frame(title: 'Cat pictures', visible: true, pack: true,
            defaultCloseOperation: WC.EXIT_ON_CLOSE,
            layout: new GridLayout(0, 2, 2, 2)) {
        images.each {
            label(icon: new ImageIcon(it))
        }
    }
}

That requires some additional imports:

import groovy.swing.SwingBuilder

import java.awt.GridLayout
import javax.swing.ImageIcon
import javax.swing.WindowConstants as WC  // Ooh, aliased imports

Here’s where we improved the system using GPars. The download of the images can be done in a multithreaded fashion by adding a GParsPool:

import static groovyx.gpars.GParsPool.*

// ...

def images = []
withPool {
    images = photos.collectParallel { p ->
        String url =
                "http://farm${p.farm}.staticflickr.com/${p.server}/${p.id}_${p.secret}.jpg"
        url.toURL().bytes
    }
}

// ...

That uses the default pool size, which is the number of processors you have plus one. The images are now downloaded concurrently as part of transforming the photo elements into byte arrays using collectParallel.

Here’s the whole script together:

import static groovyx.gpars.GParsPool.*

import groovy.json.*
import groovy.swing.SwingBuilder

import java.awt.GridLayout
import javax.swing.ImageIcon
import javax.swing.WindowConstants as WC

String key = new File('flickr_key.txt').text
String endPoint = 'https://api.flickr.com/services/rest?'
def params = [method        : 'flickr.photos.search',
              api_key       : key,
              format        : 'json',
              tags          : 'kitty',
              nojsoncallback: 1,
              media         : 'photos',
              per_page      : 6]

// Build URL and download JSON data
String qs = params.collect { it }.join('&')
String jsonTxt = "$endPoint$qs".toURL().text

// write formatted JSON data to file
File f = new File('cats.json')
if (f) f.delete()
f << JsonOutput.prettyPrint(jsonTxt)
println JsonOutput.prettyPrint(jsonTxt)

// parse JSON data and build URL for pictures
def json = new JsonSlurper().parseText(jsonTxt)
def photos = json.photos.photo

def images = []
withPool {
    images = photos.collectParallel { p ->
        String url =
                "http://farm${p.farm}.staticflickr.com/${p.server}/${p.id}_${p.secret}.jpg"
        url.toURL().bytes
    }
}

// build UI using Swing
new SwingBuilder().edt {
    frame(title: 'Cat pictures', visible: true, pack: true,
            defaultCloseOperation: WC.EXIT_ON_CLOSE,
            layout: new GridLayout(0, 2, 2, 2)) {
        images.each {
            label(icon: new ImageIcon(it))
        }
    }
}

Here is the result of a sample run:
Cat-pictures

So there you have it, except for the stupid Flickr key, which I decided to let you register for on your own. Hey, I had to go through that pain, so everybody else does, too.

Well, not everybody. As part of my pandering for evals technique, I did give my key to the students in my class, who no doubt will reward me with stellar evals once we’re done. Probably. It could happen. Either way, at least there were cat pictures, and that’s a Good Thing.

Groovy posts in other places

Recently I’ve been writing about Groovy and Grails for my friends at Accelebrate. I do that because:

  1. They’re a great client
  2. They support what I do
  3. They pay me CASH MONEY

The only problem is, whenever I post there, I don’t post here. I thought, therefore, that I’d add some links here to let everyone know what I’ve been writing about on their blog.

  • Calling RESTful services with Groovy is all about using Groovy to invoke Flikr’s API for the reason the Internet was invented: Cat Pictures.
  • That Which We Call A POGO, By Any Other Name, is a historical review of POJOs and POGOs, with Groovy AST transformations, and lots and lots of links to wildly unlikely resources that form the word POGO.
  • Building a Geolocation Web Application with Groovy and Grails was originally entitled, “Where in the World is Steve Heckler?”, but they changed it. In that post, I describe writing a Grails app to display locations on a map. (It helps to know that Steve Heckler is the owner of Accelebrate.) The best part is that I even managed to use pictures of Steve as custom markers on the resulting Google Map.:)
  • Kicking AST and Taking Names is all about using Groovy’s abstract syntax tree (AST) transformations to dramatically simplify coding. That sounds dry, but it does have figures that reference CaddyShack, South Park, and even Gladiator, so it’s got that going for it. Which is nice.

So if you’re enjoying any of my posts and you’re wondering why I haven’t been more active here, there you go. And wherever you go — (waits a beat) — there you are.

If a method arg is a closure, for crying out loud pass it a closure

This is a (mildly) embarrassing post, because it demonstrates how in my transition to functional programming I missed something important. I get it now, though, and I might as well help others avoid the same mistake I made.

I teach a lot of Grails training classes, and one question that always comes up is how to map to existing databases rather than generate the schema from the Grails application. Hibernate (and therefore Grails) would love to generate the database schema for you, for a variety of reasons, but mostly because going the other way isn’t unique. You have to make a lot of decisions about a domain model when you reverse-engineer an existing schema, ranging from multiplicities in relationships to how to map many-to-manys to directionality and more.

In my standard Grails course, rather than create a database we can map to, I go with an existing, freely available sample. For years I’ve chosen the Sakila sample database from MySQL.

(As an aside, it took me years to realize that the word Sakila is actually intended to be a subtle joke. It’s the letters SQL with some vowels thrown in to make it possible to pronounce the word, with the “Q” replaced by a “k”. Go figure. As it turns out, it’s also the name of their dolphin emblem, but that’s neither here nor there.)

Over the years I’ve tried to map the Sakila schema to a Grails domain model, with varying degrees of success. I think I have it all worked out now, but that’s the subject for another blog post. Here, instead, I want to talk about stored procedures.

Many of my clients want to use Grails with a database that has lots of stored procedures in it. Hibernate doesn’t like that. Hibernate wants to map all the domain classes directly to tables, without going through any possible intermediaries. While it is possible to invoke a stored procedure from Hibernate, it’s a bit awkward, mostly because Hibernate doesn’t want to do it.

(Length of Java Persistence Using Hibernate: about 850 pages. Number of pages that discuss stored procedures: about 5.)

That’s where Groovy (as opposed to Grails) comes in. The Groovy API includes the wonderful groovy.sql.Sql class, which is a simple, but effective, façade over JDBC. That class will open and close connections, eliminate all the annoying boilerplate code associated with JDBC, and more. If I ever have to write raw SQL code, I always switch to groovy.sql.Sql rather than use JDBC.

(That’s not quite true. The JdbcTemplate class from Spring is nearly as helpful. If I don’t have Groovy available and I have to do JDBC, I go with that.)

Let me get specific. The Sakila database represents a video store (which shows how old that is). It has tables representing stores, and inventory, and actors, and films, and more. But it also contains a handful of stored procedures, one of which is called film_in_stock. According to the documentation, the film_in_stock procedure takes two input parameters (the id of the film and the id of a store) and one output parameter (the number of copies of that film currently at that store). The example they show is:

mysql> call film_in_stock(1, 1, @count);
+--------------+
| inventory_id |
+--------------+
| 1            |
| 2            |
| 3            |
| 4            |
+--------------+
4 rows in set (0.06 sec)

Query OK, 0 rows affected (0.06 sec)

mysql> select @count;
+--------+
| @count |
+--------+
| 4      |
+--------+
1 row in set (0.00 sec)

There are four copies of film 1 (“Academy Dinosaur”) at store 1 (a store at “47 MySakila Drive” in Alberta). This provides me with a test case:

import spock.lang.*

class StoreProcServiceIntegrationSpec extends Specification {
    def storedProcService  // inject the service

    void "call film_in_stock stored procedure"() {
        expect:
        storedProcService.callFilmInStock(1, 1) == 4
    }
}

This assumes my call to the stored procedure happens in a class called StoredProcService. I have to use an integration test here, because I want to use the real database, but that means I can rely on Spring dependency injection to get the service into the test.

To access the Sakila database from a Grails app, I configured my conf/DataSource.groovy file to include the dataSource properties:

import org.hibernate.dialect.MySQL5InnoDBDialect

dataSource {
    pooled = true
    jmxExport = true
    driverClassName = "com.mysql.jdbc.Driver"
    username = "root"
    password = ""
    dialect = MySQL5InnoDBDialect
}
// ...
environments {
    development {
        dataSource {
            dbCreate = "validate"
            url = "jdbc:mysql://localhost:3306/sakila"
        }
    }
    test {
        dataSource {
            dbCreate = "validate"
            url = "jdbc:mysql://localhost:3306/sakila"
        }
    }
// ...
}

(Yeah, I have super high security on this database. Don’t even think about breaking in. You’d never guess that the root user has no password whatsoever. I mean, who would do such a ridiculous thing?)

So the default data source in my Grails app is the Sakila database, using a MySQL driver that I have listed in the conf/BuildConfig.groovy file:

dependencies {
    runtime 'mysql:mysql-connector-java:5.1.29'
    test "org.grails:grails-datastore-test-support:1.0-grails-2.4"
}

Again, nothing terribly unusual here.

The key observation is that the dataSource reference in the config file refers to a Spring bean, which can be injected in the usual manner. So I created a Grails service called StoredProcService and specified the dataSource bean as a dependency.

@Transactional
class StoredProcService {
    def dataSource
    // ...
}

The GroovyDocs for the Groovy API show that the groovy.sql.Sql class has a constructor that takes a data source, so I’m all set. Then, to call the stored procedure, I need to invoke the call method from that class.

This is where life gets interesting, and ultimately tricky. The procedure I want to call has both input and output parameters, so my call has to take that into account. It turns out that the way to call a stored procedure with both inputs and outputs is to invoke:

void call(String sql, List<Object> params, Closure closure)

The SQL string uses the JDBC escape syntax for stored procedure calls. The params list take the input parameters and placeholders for the output parameters, and the closure is then invoked with the output parameters.

Wait, what? That’s a little fast. Here’s the actual call, or at least this is the version I had before last weekend:

import groovy.sql.Sql

@Transactional
class StoredProcService {
    def dataSource
    
    int callFilmInStock(filmId, storeId) {
        String sqlTxt = '{call film_in_stock(?,?,?)}' // JDBC escape syntax
        Sql db = new Sql(dataSource)  // use the injected datasource
        db.call(sqlTxt, [filmId, storeId, Sql.INTEGER]) { count ->
            // use the count here
        }
    }
}

The Sql class provides constants like “static OutParameter INTEGER” to represent output variable in the stored procedure call. Then the last argument is a closure that has as many arguments as there are output parameters (in this case, one, which I called count).

Now, though, I have a problem. I’d like my callFilmInStock method to return the result of invoking the film_in_stock stored procedure, i.e., the count of films at that store. The problem is, I can’t just say return count inside that closure, because returning from a closure returns only from the closure, not the calling method.

I’ve blogged about that before, and even submitted a variation to Baruch Sadogursky as a Groovy Puzzler, but this is where I first encountered that problem.

My solution was always to create a local variable outside the call, assign the count to that variable, and return the variable:

int callFilmInStock(filmId, storeId) {
    String sqlTxt = '{call film_in_stock(?,?,?)}' // JDBC escape syntax
    Sql db = new Sql(dataSource)  // use the injected datasource
    int result = 0
    db.call(sqlTxt, [filmId, storeId, Sql.INTEGER]) { count ->
        result = count
    }
    return result
}

This works, and (sigh) I’ve been teaching this approach for some time now. I’ve always been uncomfortable with it, though, and when I started digging into Java 8 lambdas I realized why. I’m using a closure here and relying on it having a side effect, which is to modify a variable defined outside the closure. In fact, in Java 8, lambdas are definitely not supposed to do that. The only variables lambdas are supposed to access from outside the lambda are those that are “effectively final”, meaning they’re never modified. Groovy closures don’t have that limitation, but I still had this nagging feeling that I was, as they say, doing it wrong.

Last weekend, the new NFJS season started for me in Minneapolis, and I got a chance to talk about this to the inimitable Venkat Subramaniam, who writes books on this stuff (among other things). After I explained the problem, he thought about it for about five whole entire minutes and said that I should be passing the method a closure, not returning an integer.

The right way to write my service function is to add a closure to the method signature and use it in the sql call:

void callFilmInStock(filmId, storeId, closure) {
    String sqlTxt = '{call film_in_stock(?,?,?)}' // JDBC escape syntax
    Sql db = new Sql(dataSource)  // use the injected datasource
    db.call(sqlTxt, [filmId, storeId, Sql.INTEGER], closure)
}

Wow, that’s so much simpler. The client now passes in a closure that uses the returned value, rather than trying to kludge returning the value itself. That means the test case becomes:

void "call film_in_stock stored procedure"() {
    expect:
    storedProcService.callFilmInStock(1, 1) { count ->
        assert count == 4
    }
}

(This uses the normal Groovy idiom where a method taking a closure as its last argument can put the closure after the parentheses.)

This test passes just as the previous one did, even though the method return type is now void.

That’s the part I never got before. The groovy.sql.Sql.call(String, List, Closure) method takes a closure as its last argument, which means I can supply the closure myself and the method will use it with the returned count.

When I learned about functional programming, I got the idea of transforming collections rather than iterating over them, and using filters to select only the values I want, and reducing the results down to single values. I got that I’m supposed to favor immutability wherever possible. What I apparently hadn’t grokked was sending closures as method arguments and how that simplified everything. I’m still not sure I’m all the way there, but I’m certainly a lot closer now.

Over the weekend I tweeted that Venkat had fundamentally changed my understanding of functional programming, which is both cool and seriously annoying. Frankly, those are also words I would use to describe Venkat. If he wasn’t such a nice guy, he would be insufferable, mostly due to sheer jealousy. But we rapidly learn on the NFJS tour never to compare ourselves to Venkat in any way or you’ll just drive yourself crazy. Just don’t tell him I said that.

Now I have to go update my Grails materials…

Follow

Get every new post delivered to your Inbox.

Join 1,978 other followers

%d bloggers like this: