GR8Conf US 2016 Recap

This was my first time attending GR8Conf and it was a really great experience. I primarily focus on Android Development now, but I enjoy dabbling in some API Development when I find the time. This conference was very inspiring and I plan on investigating many of the new things that I learned. Here are a few highlights of the event:

Talk: From Grails to Android

One of the key new features of Grails 3 is the use of Gradle for its build system. The Android Framework also uses the Gradle build system to build, test, run and package applications. In this talk, learn how to move from developing applications for Grails 3 to Android.

Grails: 6 Must Read Security Tips

Courtesy: Yuri Samoilov, Flickr

Grails comes packaged with several application security gems. However, by default some of these features are not enabled. This is especially the case in older versions (2.2.x) of the Framework. 

In this blog post I will review six features that can help support your efforts to protect your application. If you have access to +lynda.com, I also highly recommend the Web Security course by +Kevin Skoglund. 

Grails: Functional Testing with Geb and Spock

Functional testing allows you to execute HTTP requests against your running application and verify if it's performing the desired behavior.

Since Grails doesn't ship with any direct support for functional testing (using Grails 2.2.4) I decided to use a common configuration of Geb with Spock for my testing needs. My configuration files are included below.  I had to modify BuildConfig.groovy and then include a separate configuration file for Geb.




BuildConfig.groovy

    repositories {
       mavenRepo "http://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-support"
    }

    def gebVersion = "0.9.2"
    def seleniumVersion = "2.35.0"

    dependencies {
        test "org.seleniumhq.selenium:selenium-chrome-driver:$seleniumVersion"
        test "org.spockframework:spock-grails-support:0.7-groovy-2.0"
        test "org.gebish:geb-spock:$gebVersion"
        test "org.seleniumhq.selenium:selenium-support:2.40.0"
    }

    plugins {
        test ":geb:$gebVersion"
        test (":spock:0.7") {
            exclude "spock-grails-support"
        }
    }

The tests and config files are stored in the test/functional folder of the main application. In the primary functional folder you can place your GebConfig.groovy file. There are various configuration options available to you. I prefer to use a very simple version, consisting of the following:

GebConfig.groovy

import org.openqa.selenium.chrome.ChromeDriver

reportsDir = "target/geb-reports"
driver = { new ChromeDriver() }

You will also need to install the ChromeDriver.  Info available here.

Once that's done, you should define some Geb Pages that represent the corresponding gsp pages in your application. I typically define three key components: the url, the at closure, and the content closure. These elements will be used in your Spock Specification to navigate to the desired page in your application, verify that you are at the correct page and then interact with elements on the page that you define in the content closure.

Sample Page

import geb.Page

class LoginPage extends Page {

    static url = "login/auth"

    static at = { title == "Login" }

    static content = {
        loginForm { $("#loginForm") }
        loginButton { $("#loginButton") }
        registerLink { $("a[href*='register/index']") }
    }
}

Sample Spec

import geb.spock.GebReportingSpec
import spock.lang.Stepwise

@Stepwise
class LoginSpec extends GebReportingSpec {

    def "invalid login"() {
        given: "I am at the login page"
        to LoginPage

        when: "I am entering invalid password"
        loginForm.j_username = "me@gmail.com"
        loginForm.j_password = "ioguffwf"
        loginButton.click()

        then: "I am being redirected to the login page"
        at LoginPage
        !loginForm.j_username
        !loginForm.j_password
    }

    def "admin login"() {
        given : "I am at the login page"
        to LoginPage

        when: "I am entering valid username and password"
        loginForm.j_username = "me@gmail.com"
        loginForm.j_password = "me"
        loginButton.click()

        then: "I am being redirected to the homepage"
        at HomePage
    }
}

Spock specifications are very simple to read and should be written in such a way that it's very clear what you are attempting to test and verify.  In this case, the LoginSpec, the specification has only two methods which are executed in order.  The first makes sure that when a user enters invalid login credentials they are redirected back to the log in page.  The second verifies that they are taken to the HomePage when the login succeeds.

In order to run your functional tests you can use the following commands:

grails test-app functional: -https
grails test-app -clean functional: LoginSpec -https

I add the -https flag because my application has support for https and I want to be able to test that functionality as well.

Happy testing!

Grails: Displaying messages from messages.properties in Quartz Job

When working with Quartz Jobs, you may want to take advantage of Strings that are located in the messages.properties file.   In order to do this, you need to get a reference to the messageSource bean and then call the getMessage() method with the name of the message that you want to use, whatever objects you need to pass to the message and finally the locale. (examples below)

def messageSource 
...
messageSource.getMessage('my.msg', [myObject] as Object[], Locale.US)

Without any parameters in the message:

def messageSource 
...
messageSource.getMessage('my.msg.no.params', null, Locale.US)

Grails: Cannot create JDBC driver of class 'com.mysql.jdbc.Driver' for connect URL 'jdbc:hsqldb:mem:grailsDB'

Recently received the following error when attempting to deploy a new war file on my Glassfish server:

Cannot create JDBC driver of class 'com.mysql.jdbc.Driver' for connect URL 'jdbc:hsqldb:mem:grailsDB'

The solution to the issue however, has nothing to do with MySQL and drivers whatsoever.  The problem was that I mistyped the environment name when creating the war file.

I typed grails rstaging war instead of grails staging war.  This is just another example of extremely useful error messages that I've encountered while developing for Grails.

Storing JSON in External EhCache Server

If you would like to store JSON data in an external EhCache Server you would need to use either REST or SOAP calls.  The below example uses REST calls and the HTTPBuilder library.  It assumes that you have your EhCache Server running at the following location:
http://localhost:8080/ehcache/rest.

The name of the cache that I'm using in this example is just sampleCache.

def getCachedItem(key) {
    def uri = "http://localhost:8080/ehcache/rest/sampleCache/"
    def restClient = new RESTClient(uri)

    def res
    try {
      res = restClient.get(path:key)
    }
    catch (Exception e) {
      log.error("Error retrieving item from cache for this key: $key")
    }
    
    return res?.data
  }

  def putCachedItem(key, html) {
    def url = "http://localhost:8080/ehcache/rest/sampleCache/"

    try {
      def json = html as grails.converters.JSON

      def http = new HTTPBuilder(url)
      http.request(Method.PUT) { req ->
        uri.path = key
        send('application/json', grails.converters.JSON.parse(json?.toString()))
        
        response.'201' = { resp ->
          log.info("success in adding item to cache: ${resp.status}")
        }

        response.success = { resp, object ->
          log.info("success in adding item to cache: ${resp.status}")
        }

        response.failure = { resp ->
          log.error("error adding item to cache: ${resp.statusLine}")
        }
      }
    }
    catch (Exception e) {
      log.error("Error setting the cache for this key: $key")
    }
  }

Removing jar files from your Grails war

Recently I encountered a situation where a plugin was bringing in a version of a jar file that was causing my application to misbehave.  In order to fix this issue I had to remove the errant jar file from the final build.  This is the code that you can use to do that.  You have to place this code snippet in your BuildConfig.groovy.  Hope this helps someone else.

grails.war.resources = { stagingDir ->
  delete(file:"${stagingDir}/WEB-INF/lib/<name>.jar")
}

Updated: Mocking Hibernate Create Criteria in Grails' Unit Test with GMock

Using the GMock Library I have updated my unit tests to mock out the create criteria methods the following way:

void testSomeFunction() {
    def results = []

    def mockCriteria = mock() {
      list(instanceOf(Closure)).returns(results)
    }

    mock(Book).static.createCriteria().returns(mockCriteria)

    play {
      assertEquals null, bookService.getDefaultBook(null)
    }
  }

You need to add the following to the top of the unit test file with the import statements:

import org.gmock.*
import static org.hamcrest.Matchers.*

@WithGMock

Also add the following to your BuildConfig.groovy:

dependencies {
    test "org.gmock:gmock:0.8.0"
    test "org.hamcrest:hamcrest-all:1.0"
}

I learned this from the following blog post: http://adhockery.blogspot.com/2010/01/using-gmock-to-complement-grails.html

Why you should upgrade to Grails 1.3.5

I did a profile of my Grails application with JVisualVM running Grails 1.3.4 and then again running Grails 1.3.5.  What a difference!  The memory leak issue that was referred to here http://jira.codehaus.org/browse/GRAILS-6622 was fixed in the 1.3.5 release.  I believe that a similar memory leak was affecting my application as well.  Across the board, the application is faster and has a much smaller memory footprint.  I have attached a snapshot of the memory comparison for the two versions to this post.

Memory Comparison of Grails 1.3.4 and Grails 1.3.5

3 Tips to Help Avoid the Hibernate Flush

When using Grails, the default behavior of Hibernate is to do a flush before queries, at the end of requests, and pretty much whenever Hibernate feels like it. So here are three tips that can help you avoid some of the flushing and/or at the very least decrease the number of database connections that Hibernate attempts to get:

1. Use read() function when possible: If you are only going to read some field of a domain object and will make no changes to it, then just use object.read(), instead of the get() method
2. Make sure your services are only transactional if they need to be, so if you are not updating, saving, or deleting records in the db chances are you do not need your Service to be transactional
3. If you do have some transactional logic, put it inside a domain.withTransaction {} closure instead of making the entire function or class transactional
   

Configuration of DataSource.groovy for Highly Concurrent Grails Application

In developing a highly concurrent Grails Application, I ran into an issue where many connections were being created to the database and were subsequently not released. After reading several blog posts it seems that the library used to handle the connection pooling was not designed for highly concurrent systems. I found that using the following setup in your DataSource.groovy file eliminates the stale connection issue and allows a high level of concurrency.

dataSource {
pooled = true
driverClassName = "com.mysql.jdbc.Driver"
dialect = "org.hibernate.dialect.MySQL5InnoDBDialect"
properties {
maxActive = 50
maxIdle = 10
initialSize = 10
minEvictableIdleTimeMillis = 10000
timeBetweenEvictionRunsMillis = 20000
maxWait = 10000
validationQuery = "/* ping */"
testWhileIdle = true
numTestsPerEvictionRun = 3
testOnBorrow = true
}
}

Hope this can help someone else.

Monitor Hibernate Cache Statistics in Grails Application v2

In an earlier post I wrote about monitoring your cache hits in a Grails Application using a programmatic approach, this time I will show you how to use jconsole and have it done for you automatically.

All you need to do is start your Grails Application: grails run-app. Then start jconsole and choose your running Grails' process. Go over to the MBeans tab and navigate to the area of your application server where you can see the cache.

What I really love about this is that you can perform actions in your application and watch the cache hits over time.

Using Grails' mockParams with Associations

If you have a class Book which contains a list of authors and would like to test this functionality in your Controller's unit tests. This is how you would do it, using the Grails mockParams.

void testSaveOfBookWithAuthor() {
mockRequest.method = 'POST'
mockParams.title = "Book Title"
mockParams["authors"] = [["author.id":"123", "author":["id":"123"]]]
controller.save()
}

Spring Security vs. Apache Shiro

This is a high level comparison of Spring Security and Apache Shiro, two Security frameworks that have plugins available in Grails.

Spring Security (a.k.a. ACEGI)

• has users with roles
• to restrict users on a more fine grained level, use interceptors in your controller
• LDAP support
• integration with CAS single-sign-on
• has a new ACL option for the Grails Plugin
  
• somewhat 'heavy-handed'
  

Example (Using Grails Interceptor):
def beforeInterceptor = [action:checkViewableSites, except: "list"]

def checkViewableSites = {
if(!user.viewableSites.contains(Site.get(params.id) )) {
response.sendError 403
return false
}
}

Apache Shiro (a.k.a. JSecurity)

• has support for permissions out of the box
• LDAP support with Shiro plugin
  
• easy to customize
• Grails Nimble plugin that allows permissions on three levels: users, groups, and roles
• Nimble plugin has an easy to use UI component for adding/modifying users and permissions
• Nimble plugin has support for authentication through Facebook and OpenID out of the box
  

Example (Using Shiro Permissions):

• profile:edit:1 - this means a person can edit only the Profile object with id 1
• profile:*:* - this means that a person can do anything to any Profile object
  

Monitor Hibernate Cache Statistics in Grails Application

After going through the effort of adding cache:true all over your application to take advantage of the Hibernate caching you will want to monitor the results. In order to do so you only need to modify two files, DataSource.groovy and a desired Controller.

//In your DataSource.groovy
hibernate {
generate_statistics=true
}

//In your Controller
def sessionFactory

def showCacheStatistics = {
def statistics = sessionFactory.statistics
log.info(statistics)
render statistics
}

Here is the output of calling the showCacheStatistics method on your controller:
Statistics[start time=1269058393252,sessions opened=27,sessions closed=25,transactions=18,successful transactions=3,optimistic lock failures=0,flushes=21,connections obtained=15,statements prepared=219,statements closed=219,second level cache puts=45,second level cache hits=1,second level cache misses=0,entities loaded=40,entities updated=5,entities inserted=48,entities deleted=0,entities fetched=26,collections loaded=20,collections updated=0,collections removed=0,collections recreated=12,collections fetched=18,queries executed to database=93,query cache puts=6,query cache hits=8,query cache misses=6,max query time=89]

Currently I'm showing just a direct rendering of the statistics object, but you could specify which parameters are important to you and only show those, or perhaps send the data to a gsp file and have it formatted in some way.

Improve Performance of Grails Queries in MySQL

A web application that I was developing would become extremely slow as the number of requests increased per minute. I couldn't figure out why because with my local database everything seemed fine. Here is the process that I used in MySQL to determine the bottleneck in my database. The final problem was a missing index on a table that was used in the where clause of a select statement over and over again. I included a snapshot the MySQL Process list at the end of the post. Hope this helps someone.

Steps to Follow:

• show full processlist;
  • shows each database connection that has been allocated for your database
  • contains the time a connection has been Sleep
  • contains how long a Query is taking to execute, you want this to be less than a second ideally
• describe <query>;
  • this gives you the database's plan for executing the query
  • you want the number of rows that will be looked at to be as small as possible
• show index from <table>;
  • this will list all indices contained on a table
  • this is a quick way to discover if you are missing a key index that could improve query performance
• create index <tablename_fieldname_idx> on <table>(field);
  
  • this will add a new index to a table

Mocking Hibernate Criteria in Grails App

Testing is always a sore spot in many applications, and Grails is no exception, but mocking objects and methods is actually not as bad as you think. Here is a simple way to mock the Hibernate Criteria object that is available from Grails domain classes.

Example:
If this is the code that you want to test, and for this particular test you want to have no results, this how you would mock it out.

Code:


Test Code:


The key piece of code is the creation of the bookCriteria object, in this case it is a member of the test class that is why it is not declared locally in the function. However, you will notice that the bookCriteria is simply a HashMap that returns a empty list for the 'list' parameter. This way when testing 'c.list' a empty list will be returned. Simple hunh!

Using ConfigSlurper Scripts in Grails

In Java .properties files are used to configure an application. You can use similar notation to configure Grails applications as well.
A sample appName-config.groovy file that sets up the configuration items that we need.

abtesting {
   amazon = new Double(0.5)
   ebay = new Double(0.5)
}

A sample function in a controller that will access the configuration properties and modify them.

def abtesting = {
 def propValue = grailsApplication.config.abtesting.amazon
 println propValue


 grailsApplication.config.abtesting.amazon = new Double(1.0)
 println grailsApplication.config.abtesting.amazon
}

This code produces the following output:
>> 0.5
>> 1.0

Using Grails' JSON and HTML Builders

In order to generate JSON in groovy that can then be passed to a Grails controller you should use the JSONBuilder class. This class is part of the Grails util package. The code below demonstrates how to generate JSON and HTML.

Resulting JSON that is produced:

ResultSet {['text':'google', 'url':'http://www.google.com'], ...}

def writer = new StringWriter();
new grails.util.JSonBuilder(writer).json {
  ResultSet {
    links.each { link ->
      Links(text: link.text, url: link.url)
    }
  }
}
writer.toString()

This is the code to do the same with HTML, you would use the Groovy XML MarkupBuilder instead.

StringWriter w = new StringWriter()
def builder = new groovy.xml.MarkupBuilder(w)

builder.div (class: 'className'){
  links.each { link ->
    p {
      b 'Title: '
      builder.yield link.text
    }
  }
}
w.toString()

Resulting HTML that is produced:
<div class="className">
<p><b>Title: </b>google</p>
</div>