profile for Stefan Kendall at Stack Overflow, Q&A for professional and enthusiast programmers

Hostnames are names, too.

No Comments »

A long time ago, in a galaxy not too far away, humans realized that everyone in the world needs a great selfie stick to admire themselves. Then, they began naming things; objects, concepts, and even people themselves were branded with some tag identifying their person. Identifying the specific identity of a person was clearly important.

Why, then, are there so many John Smiths in the world? Are parents just not creative enough. Well, probably not. John Smith is pretty easy to remember, and can you name one you know? Two? In a fixed geography, name collisions are tiny and aren’t an issue. Enter the internet. Suddenly, uniqueness is a problem again. People adopt “screennames” or “aliases” to give themselves a more permanent and unique identifier. Since the name only appears in text, it doesn’t even have to be particularly memorable or pronounceable, although the more consistent the name is spelled, the more likely it is someone will be able to find you online.

My gamertag, for instance, is “iftrue”. Seems pretty hard to mis-spell if I spoke the name out loud. xX99friendlyfire99Xx, on the other hand, probably has issues.

But to my point.

Hostnames, like human names, need to be unique.

But only to a fixed-size area.

What, then, are the implications of this?

1.) Choose hostnames that express purpose.

My HTPC’s hostname, for example, is HTPC. Sure, there may be hundreds or thousands of such machines in the world, but who cares? On my network of a handful of devices, there will be one HTPC. Its name will be HTPC. In class IT-ology, one would choose an entirely arbitrary naming scheme for the machines. Types of candy, animal names, cartoon characters – all would be fair game. But why should my HTPC be named “Hersheys”. It just doesn’t make sense! From the day of naming forward, I would need to maintain an internal mapping of candy names to machine purpose. Why not build the purpose into the name?

2.) Make hostnames unique.

Technically, there’s no particular difference between HTPC and HersheyHTPC3838__4. With just HTPC, however, there could be issues as my network grows. How, then, do I distinguish HTPC from new home theater PCs that might crop up on the network? Going back to the human names and geography example, why not use the room? Surely more than one HTPC won’t ever invade the same room, so livingroom-HTPC will likely never conflict with future machines. In a business sense, a room is more likely akin to your team or project. Think for a moment; odds are, your project or team has a name you use internally to identify yourself. That name is your hostname identifier.

3.) There is no fight club.

Honestly, there’s no reason to name your servers after pokemon. It may be fun at first, but it isn’t manageable. Eventually, you’ll run out of pokemon, and long before that you’ll be creating spreadsheets to map bad names to good names. Get rid of the spreadsheets and the wikis, and just use names that make sense.

My trip down the rabbit hole – NFJS, Grails, and borked plugins

No Comments »

This past weekend, I attend NFJS 2011 Columbus, and I came back excited and ready to go implement everything I saw in action during the presentations given over the three days of the conference. Specifically, I was excited about the integration of blueprint, Compass, and grails.

It just works, I was told, and hey look, there’s even a plugin that supports the integration!

Well, that last bit was mostly a lie. Over the past two days, I’ve been rebuilding the plugin from the ground up. Version 0.2, it seems, had the following awesome “features”:

Broken on Windows
That’s right. The plugin didn’t work *at all* on windows. It relies on the existence of applications on the path which simply cannot exist in any cross-platform manner, so windows just doesn’t work.
Directory watching for SASS changes doesn’t work, or crashes
You would think that the ability to modify a file and test it out immediately without compiling would be a prerequisite to even consider releasing a grails plugin, but alas, it is not.
JRuby and the Compass gem have to be installed on your build system for the plugin to work
With the power of grails dependency management at your finger tips, why would you ever possibly do this? Even in the worst case scenario, you could package a gradle build script and run that on application startup. On a team of more than one developer, arbitrarily adding build dependencies is cumbersome and non-ideal.

Well, I fixed the first two issues on my own fork of the plugin, but I still need to go do the third. Once all these issues are resolved, the plugin will be viable again, and maybe some unfortunate programmer soul won’t fall down the rabbit hole of breakage I did. I may re-release the plugin, as it’s changing into something entirely different than what it was originally.

SASS and Compass are cool; REALLY cool, in fact, and we grails gimps deserve a sane pipeline too.

Documentation always lies.

No Comments »

Yes, this has been covered before, and yes, much smarter people than I have already made the case against documentation.

But they don’t go far enough.

Published interfaces, for example, are supposed to be the exception to the rule. Surely developers using your very public API need to be able to trust documentation, right? Right? Wrong.

To illustrate what I’m talking about, here’s an excerpt from Ext.form.BasicForm in the ExtJs 3.3.1 API:

getValues( [Boolean asString] ) : String/Object
Returns the fields in this form as an object with key/value pairs as they would be submitted using a standard form submit. If multiple fields exist with the same name they are returned as an array.

Note: The values are collected from all enabled HTML input elements within the form, not from the Ext Field objects. This means that all returned values are Strings (or Arrays of Strings) and that the value can potentially be the emptyText of a field.


asString : Boolean
(optional) Pass true to return the values as a string. (defaults to false, returning an Object)




getFieldValues( [Boolean dirtyOnly] ): Object
Retrieves the fields in the form as a set of key/value pairs, using the getValue() method. If multiple fields exist with the same name they are returned as an array.

Note: The values are collected from all enabled HTML input elements within the form, not from the Ext Field objects. This means that all returned values are Strings (or Arrays of Strings) and that the value can potentially be the emptyText of a field.


dirtyOnly : Boolean
(optional) True to return only fields that are dirty.


The values in the form

Got all that? Both sound like they do about the same thing, except that getValues allows for the form values to be returned as a string. Upon first usage, this seems to be correct. After you start testing, however, you find this nasty little bug:

getValues turns all spaces in form values into plus-signs.

See that anywhere in the documentation? No? Neither do I.

But surely, Stefan, this is the exception and not the norm, right? Unfortunately no. Only blog posts, stackoverflow answers, and book examples can be trusted at all to be even somewhat reliable when it comes to any technology in the process of change. Known bugs, outdated documentation, incorrect documentation can lure you into choosing a product that claims to do more than it does, or avoid a technology which does more than it claims.

Using a library or method yourself is the absolute 100% only way to determine if something works or not. The best you can hope for is a clean API, widespread usage, and a handful of stackoverflow and blog posts describing working implementations. Luckily for me, the ExtJS API browser is quite beautiful, and troubleshooting took only a few minutes.

I’ve recently dealt with other libraries, however, that weren’t so clean, and a very simple assumption, like, say, a simple factory class only instantiates and returns a single instance, can be wildly wrong and lead to hours of pain. In the linked example, the factory class attempts to read XML mapping files from the classpath, and throws exceptions if the mappings don’t exist. Simply newing a DozerBeanMapper instance doesn’t. What the bloody hell.

All documentation is wrong, and RTFM just can’t apply anymore, supposing it ever did. If I don’t see it working, it doesn’t work. I don’t care what your release notes say.

Wine and Parellism with GPars

No Comments »

This weekend, I finished watching The Social Network, which prompted me to start on a project I’ve been futzing with for a while. Several glasses of wine into a box of Bota Box, I had created a rudimentary grails project to scan, aggregate, and display ammunition prices for arbitrarily defined [and hot-code droppable] ammunition sellers. With each additional caliber and provider, however, I realized that time-until-results was increasing linearly. Clearly, serializing network IO while page scraping was the culprit, as processing time was minimal compared to the amount of time each page took to respond.

At first, the code looked like this:

For each provider
  For each caliber
    Build a list multimap of Provider -> Caliber : List of ammo entries

This sort of thing works for most academic exercises, but this was slow. Really slow. How slow, you ask?

1 Provider 2 Providers
1 Caliber ~3 seconds ~6 seconds
2 Calibers ~7 seconds ~12 seconds

Since my end goal is tens of providers and tens of calibers, an all-search would wind up taking 3n2 seconds, or 5 full minutes for 10 sites and 10 calibers.

At first, I decided that I could split out every ammo provider and parallelize those sequence of requests, making my algorithm this:

For each provider [asynchronously]:
  For each caliber:
     Scan and return results

This worked well, but I knew I could do better. Trying again, I parallelized caliber requests.

For each provider [asynchronously]:
  For each caliber [asynchronously]:
     Scan and return results

With this code to back it:

  List<AmmoResult> findAllAmmoResults()
	   (List<AmmoResult>) withPool {
		 def ammoProviderCallable = { AmmoProvider ammoProvider ->
			return new AmmoResult(provider: ammoProvider, caliberEntriesMap: buildCaliberEntriesMap(ammoProvider))

		 List<AmmoResult> ammoResults = ammoProviders.collect(ammoProviderCallable)*.get()

		 return ammoResults

   ListMultimap<AmmoCaliber, AmmoEntry> buildCaliberEntriesMap(AmmoProvider ammoProvider)
	  (ListMultimap<AmmoCaliber, AmmoEntry>) withPool {
		 //For each caliber, build out a 1 : N multimap that represents one unit of work of building the ammo entry
		 //multimap. That is, each key is built in parallel and combined later.
		 def caliberCallable = { AmmoCaliber caliber ->
			ListMultimap<AmmoCaliber, AmmoEntry> subMap = ArrayListMultimap.create();
			subMap.putAll(caliber, findAmmoEntriesFor(, caliber))
			return subMap

		 List<ListMultimap<AmmoCaliber, AmmoEntry>> listToJoin = AmmoCaliber.values().collect(caliberCallable)*.get()

		 //Combine the keys and values into one super multimap.
		 ListMultimap<AmmoCaliber, AmmoEntry> combinedCaliberEntryMap = ArrayListMultimap.create();
		 for (ListMultimap<AmmoCaliber, AmmoEntry> caliberListMap in listToJoin)

		 return combinedCaliberEntryMap

With the magic of GPars, or

runtime 'org.codehaus.gpars:gpars:0.11'

with Ivy, I was able to turn ~12seconds of runtime into ~4seconds of runtime.

I should probably still dive deeper, however, as many of the providers require multiple further requests to build their data – none of which are currently parallelized. Some algorithms perform well when parallelized, and anything that does heavy I/O will likely qualify. Next time you’re hitting web services, ask yourself if you should step back and kick open a thread. What passes performance metrics today likely won’t tomorrow, so plan for the worst.

Anyway, back to code, booze, and my pandora seed of the week.

Functional Programming – in Java!

No Comments »

If you’ve ever used Ruby, Groovy, Scala, Javascript, Python, or really any language worth its salt, you’ve come to love and cherish a few very basic functional ways to handle data structures. filter (find) and map (apply), specifically, cut out reams of copied code, and chained together provide powerful, expressive ways of handling collections.

With Guava, those language concepts are available in Java.


Example: Transform a list of strings into a list of the first character of that string.

List list = Lists.newArrayList( "1", "2345", "3239802", "a" );


List firstChars = Lists.newArrayList();
for( String s : list ){
   firstChars.add( s.charAt(0) );


return Lists.transform( list,
   new Predicate(){
             public Character apply( String s ){ return s.charAt( 0 ); }


Example: Filter out all strings with a length > 1

List oneLengthStrings = Lists.newArrayList();
for( String s : list ){
   if( s.length() == 1 )
       oneLengthStrings.add( s );


return Collections2.filter( list,
    new Predicate(){
        public boolean apply( String s ){ return s.length() == 1; }

Without closures, the syntax isn’t pretty, but semantically, it makes more sense, and after a while it’s easier to spot the the logic than that in wayward loops, which are often coupled to other logic unrelated to collection manipulation. When you use filter and transform, the logic is obvious and you can be fairly certain you won’t need to search for unusual side effects.

Here’s to waiting for JDK8. At least we’ve got Scala and Groovy in the meantime.

Selenium ain’t just for tests

No Comments »

Do you…

Need to fix a web UI bug, but need to deal with dynamic content nested in business logic?
Want to view a workflow repeatedly in an automated fashion?
Want to improve your web development as a whole?

Well, then, Selenium IDE is just the tool for you!

Using Selenium IDE, you can script tasks to navigate through dynamic content, without scripting at all! The firefox plugin records actions as you do them in the browser, allowing you to write once and replay back as many times as necessary. Let’s go through an example, to explain how this all works.

Suppose you want to search for a particular book on amazon, but for whatever reason, the URLs are no longer consistent and static. That is, when you search for a book, business logic could take you to a number of different screens, each with similar output but different URL mappings. This rarely happens (unless Oracle buys your company), but it does illustrate how scripted tasks overcome the issue of content-baseed naviation. Let’s go ahead and script an amazon search.

First, I pull open my trust Selenium IDE. Yee-haw.

Notice how selenium is already recording! Now, anything I click or type gets recorded automatically. How nice.

Step 1Step 2

Now, I’m going to do a simple search for “Hibernate”

Look at the simple syntax. If you want to modify parts of this yourself, you can. It’s easy to insert statements as necessary to do more complex tasks, as you’ll see in a minute.

Step 3Step 4

Ack! Our script broke? What happened?

Selenium doesn’t automatically handle AJAX. To fix the breakage, we can wait for the elements to appear for any elements coming back from AJAX. Notice the “waitForElementPresent” statements.

Step 5Step 6

And we’re done!

In a few clicks, we can now replay our actions and get back to the same state we did before, without manually jumping through hoops.

Selenium-IDE is easy to use, simple, and it’s a great browser-scripter. Good luck and have fun. – My iPhone web apps

No Comments »

Recently, I’ve become very interested in iPhone web app development and Grails, so I decided to merge the two into a little project. Using Grails, jQTouch, and jQuery, I built an application to search a variety of torrent portals and aggregate the results. I could describe the application, but why not just head over to and check it out?

If you don’t have an iPhone or Android-like device, here are some screenshots to show you what the interface looks like on a mobile device:

iPhone Web App Screenshot 1iPhone Web App Screenshot 2iPhone Web App Screenshot 3

Working with this project, I discovered a few interesting bits of information.

  1. jQTouch is incomplete
    I had to work through the source code of the CSS files and demo page to make any headway into this library. There is very little documentation, and some of the pre-built “apple” styles don’t look truly native. This is sad, but it does do some things pretty well. For instance, if you bind $.tap event to a jQuery object, and the target browser doesn’t support the $.tap javascript event, the event is bound to $.click instead. This is great for testing. I also found that while Safari for Windows is a close approximation for the iPhone browser, it’s not the same.

    For instance, if your page has any HTML errors, such as improperly closed <input /> tag, Safari for Windows won’t complain. Safari for the iPhone, however, will break all to hell and respond haphazardly to javascript events. To prevent this from happening to you, I highly recommend enabling Safari developer mode. See the screenshots below.

    How to enable Safari Developer modeHow to enable Safari Developer mode

  2. Groovy has built-in JSON conversion support
    At first, I thought I was going to need to use something like GSON to automatically convert my objects into JSON, but Groovy can do this with built-in libraries. Check this out:
    		def namedResults = [:]
    		results.each { ProviderService key, value ->
    			namedResults[key.getName()] = value
    		render namedResults as JSON

    This produces

    {"Demonoid":["Result1", "Result2",...],...}

    which is trivially parseable via the javascript JSON parser at

  3. GPath is a decent substitute for XPath
    I’m generally a big fan of XPath, but GPath ain’t so bad. Specifically, trying to find all the deepest nodes which match a certain criterion is pretty easy.
    		def slurper = new XmlSlurper()
    		slurper.setFeature("", false)
    		def doc = slurper.parseText(xml)
    		//Find anchor tags for torrent links
    		def allAnchorTags = doc.depthFirst().findAll { GPathResult node ->
    			node["@class"] == "detLink"

    By combining depthFirst() and findAll {…}, one can filter all nodes in an XML document succinctly and with ease. The equivalent XPath //a[@class='detLink'] would require some extra lifting to parse the results so neatly.

  4. Auto-wiring is magical
    In the course of building the TorrentSearch application, I used nothing but auto-wiring to configure my services and controllers. This probably doesn’t scale all that well, but it worked well enough for my purposes. I tend to use the path of least resistance until it no longer works, and auto-injection via field name is definitely the path of least resistance. Code snippet:
    class TorrentSearchController
    	def torrentProviderService

    Simply by defining the service, it gets auto-injected with a singleton instance of the class named TorrentProviderService.


Grails is a really neat way to develop applications, and it makes putting together simple web services incredibly easy. In the future, I’ll see how well this scales in the future, but for now it seems like the way to build super quick web applications on the JVM. jQTouch, on the other hand, is rough around the edges, although the potential of native web applications for mobile devices is promising.

Easy HTML web scraping with Groovy and Java. (w/XOM)

No Comments »

Ever need to parse some HTML is Java or Groovy? No matter what the source, you’re almost always guaranteed to get bad, unformed garbage as a response when scraping. Rather than ditch XML readers and bust out regex, you can transform this data into good xhtml with tools like TagSoup.

The following class is a utility class meant to abstract away the nonsense of cleaning HTML. I’ve found that HTML data can come from a number of different sources (even files), so being able to clean strings containing html is immensely useful.

HtmlParsingUtils.cleanHtml(String html) cleans the HTML passed as an argument and returns parse-able XHTML. It does not validate entities to avoid the 503 return errors if you attempt to hit the w3 servers too frequently when validating XML entities – which is exactly what happens when using TagSoup when doing massive scraping.

XOM is my XML handling library of choice, although the code can be adapted if requirements dictate no-XOM.

Example usage:

String htmlData = obtainHtmlFromElsewhere();
String xml = HtmlParsingUtils.cleanHtml( htmlData );

Builder builder = HtmlParsingUtils.createNonValidatingXmlBuilder();
Document doc =,HtmlParsingUtils.xhtmlBaseUri);
//Query: Find all anchor tags
Nodes nodes = doc.query("//html:a",HtmlParsingUtils.htmlCtx);

Presented below is the Groovy adaptation of HtmlParsingUtils. The Java version is identical, except the dummy entity resolver is built as private class declaration inline, rather than a closure.

import nu.xom.Builder
import nu.xom.XPathContext
import org.jdom.input.SAXBuilder
import org.xml.sax.EntityResolver
import org.xml.sax.InputSource
import org.xml.sax.XMLReader
import org.xml.sax.helpers.XMLReaderFactory

 * Utility class for handling html
class HtmlParsingUtils
	public static final EntityResolver dummyEntityResolver = { String publicId, String systemId ->
		return new InputSource(new StringReader(""))
	} as EntityResolver;

	public static final xhtmlBaseUri = ""

	//Required for querying
	public static final XPathContext htmlCtx = new XPathContext("html", xhtmlBaseUri)

	 * @param str possibly non-well-formed html
	 * @return xhtml representation of the argument data
	public static String cleanHtml(String str)
		SAXBuilder builder = new org.jdom.input.SAXBuilder("org.ccil.cowan.tagsoup.Parser"); // build
		Reader input = new StringReader(str);
		org.jdom.Document doc =;
		String cleanXmlDoc = new org.jdom.output.XMLOutputter().outputString(doc);

		return cleanXmlDoc;

	 * @return XOM Builder that does not validate or resolve entities.
	public static Builder createNonValidatingXmlBuilder()
		XMLReader reader = XMLReaderFactory.createXMLReader();
		Builder builder = new Builder(reader);

		return builder;

Grails: Absolutely pure productivity.

No Comments »

Your IDE can do more.

Recently, I decided I had a need to buy a portable charger for iPhone that was at least 10000mah.Hopefully where the jQTouch front-end work pretty simple, but I just wanted a super fast way to get a charger up and running that would be maintainable and easy to use on the go. I considered Spring with Tomcat/Hibernate, but I didn’t want to spend a week configuring XML. I’ve attempted to Ruby/Rails in the past, but I always missed being off the JVM. Then, I came upon Grails. It’s fast, it’s easy, and it’s Java (…ish), so in that sense there’s little to no learning curve aside from that of the Grails framework itself.

To begin my quest to utilize Grails, I attempted to setup Eclipse with the grails plugin. After a pound of headache, I just gave up. Then, I came upon Netbeans grails integration.

Check this out.

  1. Create a project.
    Step 1
  2. Let grails do its magic.

    Step 2

  3. Click the magic run arrow.
    Step 3
  4. Watch Grails start automagically under Jetty.
    Step 4
  5. Create a Domain class.
    Step 5
  6. Step 6

Now, any changes you make in the IDE to the controllers, services, or domain classes gets picked up immediately by the running webserver. In one click, you can deploy your entire application. After that one click, everything just works. This is how all IDEs should handle server deployment. Aside from this ridiculously easy setup, grails gives you dependency management off the bat with Ivy, which can be configured to use the maven2 repositories.

Check it out.

Dependency management

FURTHERMORE, grails deploys WARs for your application, so there’s no trickery to get your web application running. Develop under grails, then drop your WAR in your application container and be done with it.

Use Grails. It’s awesome.

Still Alive.

No Comments »

I’m currently working on a few side projects while trying to learn and use/deploy applications on Grails. I’ve also purchased another domain for my conquests. Hopefully I’ll have more to say about my side projects in the next few months, but I’ll hopefully get a Grails post up in early May. Expect pretty screenshots.