If you don’t have an iPhone or Android-like device, here are some screenshots to show you what the interface looks like on a mobile device:

Working with this project, I discovered a few interesting bits of information.

1. jQTouch is incomplete

   I had to work through the source code of the CSS files and demo page to make any headway into this library. There is very little documentation, and some of the pre-built “apple” styles don’t look truly native. This is sad, but it does do some things pretty well. For instance, if you bind $.tap event to a jQuery object, and the target browser doesn’t support the $.tap javascript event, the event is bound to $.click instead. This is great for testing. I also found that while Safari for Windows is a close approximation for the iPhone browser, it’s not the same.

   For instance, if your page has any HTML errors, such as improperly closed <input /> tag, Safari for Windows won’t complain. Safari for the iPhone, however, will break all to hell and respond haphazardly to javascript events. To prevent this from happening to you, I highly recommend enabling Safari developer mode. See the screenshots below.

   

2. Groovy has built-in JSON conversion support

   At first, I thought I was going to need to use something like GSON to automatically convert my objects into JSON, but Groovy can do this with built-in libraries. Check this out:

   		def namedResults = [:]
   
   		results.each { ProviderService key, value ->
   			namedResults[key.getName()] = value
   		}
   
   		render namedResults as JSON
   

   This produces

   {"Demonoid":["Result1", "Result2",...],...}

   which is trivially parseable via the javascript JSON parser at JSON.org.

3. GPath is a decent substitute for XPath

   I’m generally a big fan of XPath, but GPath ain’t so bad. Specifically, trying to find all the deepest nodes which match a certain criterion is pretty easy.

   		def slurper = new XmlSlurper()
   		slurper.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false)
   
   		def doc = slurper.parseText(xml)
   
   		//Find anchor tags for torrent links
   		def allAnchorTags = doc.depthFirst().findAll { GPathResult node ->
   			node["@class"] == "detLink"
   		}
   

   By combining depthFirst() and findAll {…}, one can filter all nodes in an XML document succinctly and with ease. The equivalent XPath //a[@class='detLink'] would require some extra lifting to parse the results so neatly.

4. Auto-wiring is magical

   In the course of building the TorrentSearch application, I used nothing but auto-wiring to configure my services and controllers. This probably doesn’t scale all that well, but it worked well enough for my purposes. I tend to use the path of least resistance until it no longer works, and auto-injection via field name is definitely the path of least resistance. Code snippet:

   class TorrentSearchController
   {
   	def torrentProviderService
              ...
   }
   

   Simply by defining the service, it gets auto-injected with a singleton instance of the class named TorrentProviderService.

Conclusion

Grails is a really neat way to develop applications, and it makes putting together simple web services incredibly easy. In the future, I’ll see how well this scales in the future, but for now it seems like the way to build super quick web applications on the JVM. jQTouch, on the other hand, is rough around the edges, although the potential of native web applications for mobile devices is promising.

The following class is a utility class meant to abstract away the nonsense of cleaning HTML. I’ve found that HTML data can come from a number of different sources (even files), so being able to clean strings containing html is immensely useful.

HtmlParsingUtils.cleanHtml(String html) cleans the HTML passed as an argument and returns parse-able XHTML. It does not validate entities to avoid the 503 w3.org return errors if you attempt to hit the w3 servers too frequently when validating XML entities – which is exactly what happens when using TagSoup when doing massive scraping.

XOM is my XML handling library of choice, although the code can be adapted if requirements dictate no-XOM.

Example usage:

String htmlData = obtainHtmlFromElsewhere();
String xml = HtmlParsingUtils.cleanHtml( htmlData );

Builder builder = HtmlParsingUtils.createNonValidatingXmlBuilder();
Document doc = builder.build(xml,HtmlParsingUtils.xhtmlBaseUri);
//Query: Find all anchor tags
Nodes nodes = doc.query("//html:a",HtmlParsingUtils.htmlCtx);

Presented below is the Groovy adaptation of HtmlParsingUtils. The Java version is identical, except the dummy entity resolver is built as private class declaration inline, rather than a closure.

import nu.xom.Builder
import nu.xom.XPathContext
import org.jdom.input.SAXBuilder
import org.xml.sax.EntityResolver
import org.xml.sax.InputSource
import org.xml.sax.XMLReader
import org.xml.sax.helpers.XMLReaderFactory

/**
 * Utility class for handling html
 */
class HtmlParsingUtils
{
	public static final EntityResolver dummyEntityResolver = { String publicId, String systemId ->
		return new InputSource(new StringReader(""))
	} as EntityResolver;

	public static final xhtmlBaseUri = "http://www.w3.org/1999/xhtml"

	//Required for querying
	public static final XPathContext htmlCtx = new XPathContext("html", xhtmlBaseUri)

	/**
	 * @param str possibly non-well-formed html
	 * @return xhtml representation of the argument data
	 */
	public static String cleanHtml(String str)
	{
		SAXBuilder builder = new org.jdom.input.SAXBuilder("org.ccil.cowan.tagsoup.Parser"); // build
		builder.setEntityResolver(dummyEntityResolver);
		Reader input = new StringReader(str);
		org.jdom.Document doc = builder.build(input);
		String cleanXmlDoc = new org.jdom.output.XMLOutputter().outputString(doc);

		return cleanXmlDoc;
	}

   /**
	 * @return XOM Builder that does not validate or resolve entities.
	 */
	public static Builder createNonValidatingXmlBuilder()
	{
		XMLReader reader = XMLReaderFactory.createXMLReader();
		reader.setEntityResolver(dummyEntityResolver);
		Builder builder = new Builder(reader);

		return builder;
	}
}

Recently, I decided I had a need to build a quick web service to back an AJAX application for iPhone that I’m developing. jQTouch is making the front-end work pretty simple, but I just wanted a super fast way to get a web service up and running that would be maintainable and easy to scale. I considered Spring with Tomcat/Hibernate, but I didn’t want to spend a week configuring XML. I’ve attempted to Ruby/Rails in the past, but I always missed being off the JVM. Then, I came upon Grails. It’s fast, it’s easy, and it’s Java (…ish), so in that sense there’s little to no learning curve aside from that of the Grails framework itself.

To begin my quest to utilize Grails, I attempted to setup Eclipse with the grails plugin. After a pound of headache, I just gave up. Then, I came upon Netbeans grails integration.

Check this out.

1. Create a project.
   
2. Let grails do its magic.

   

3. Click the magic run arrow.
   
4. Watch Grails start automagically under Jetty.
   
5. Create a Domain class.
   

Now, any changes you make in the IDE to the controllers, services, or domain classes gets picked up immediately by the running webserver. In one click, you can deploy your entire application. After that one click, everything just works. This is how all IDEs should handle server deployment. Aside from this ridiculously easy setup, grails gives you dependency management off the bat with Ivy, which can be configured to use the maven2 repositories.

Check it out.

FURTHERMORE, grails deploys WARs for your application, so there’s no trickery to get your web application running. Develop under grails, then drop your WAR in your application container and be done with it.

Use Grails. It’s awesome.

An open protocol to allow secure API authorization in a simple and standard method from desktop and web applications.

Unfortunately, as it turns out, OAuth is neither simple nor standard in practice. In case you don’t care to read the tutorials, OAuth works something like this:

1. You request a consumer key and consumer secret from a “service provider” (such as Twitter).
2. You use this consumer key and secret to request a “request key” (insecure).
3. You use the request key to generate a URL that the user can go to to authenticate.
4. You either do, or don’t, pass a callback URL to the above request to receive the response token.
5. Your callback URL gets hit, which contains the token needed to request an “access token” and “access token secret”, which may or may not expire or become invalid at any time.

These steps, however, are not so clear cut when you actually start using service providers which offer OAuth authentication.

Let’s look at the behavior of some big-name web services which have adopted OAuth.

Twitter, I think, has taken a progressive approach to OAuth authentication. If you don’t pass a callback URL, you get a pretty screen prompting the user to deliver a PIN code to the application from which you’ve come.

This looks great, right? Well, if you try the same stuff on TripIt, you just get the return data.

Response:

oauth_token=935394b819ab32c9558e7063dbb3bcda9c274a00

TripIt presents no user display, so you’re stuck if you didn’t want to a provide a callback URL. Once you understand that Twitter just does things differently, you begin to assume that using a callback URL would work swimmingly. Well, if you wanted Twitter support, you just lost it. Look at Twitter’s response to a request with a callback URL specified:

Response URL:

http://stefankendall.com:8080/test.jsp?oauth_token=0yLxRrUoxHJxA0tF8hJSxP2EHfFaYyw1a3nW4Dqg&oauth_verifier=Alzk3ehPRq6bbjvISlgXqn9Jk7yZJlmst0jtXpg

That’s right. TWO values, whereas TripIt just provided one. You’ll notice that oauth_token is now the token you requested with, and oauth_verifier is the real data you need. This is a direct contradiction in the naming and response schemes of TripIt and Twitter. Oy vey.

I would go on about Netflix, but that’s a jump down the rabbit hole I’m not willing to take right now.

OAuth is a joke, and as it stands, it’s neither simple nor a standard. Good grief.

str = 'an' * 3000 + 'garbage' + 'an'*3000
r = /(?<pattern>.*) #The first part
    (?:.*?) #Garbage
    \k<pattern>/x #The first part, repeated.
str =~ r
puts $1.length

First, we create a repeated string, followed by some garbage data, followed by that string again. A pattern is built to find the largest prefix which is also a suffix, given a break in data. Ruby lets us name the regex

Consider the following:

<posts>
   <post>
      <title>XSLT!</title>
      <description>A short post about XSLT</description>
      <content date = "01/03/2009">
      Lorum ipsum
      </content>
   </post>
<posts>

The naive approach of displaying this to the user is to manually parse the data with an XML parser of some kind. I’ve done this in previous projects; it works, but it’s generally stuck in code, which means that changes to the output format require recompilation or some other elaborate build and deployment. This is inherently wrong given the process that’s taking place. A transformation of XML can and should be independent from the XML itself.

Here’s some pseudocode for the rendering of an XML document into HTML:

html = '<html><body>'
for node in document.find( 'post' )
    title = node.find( 'title' ).text
    descriptionNode = node.find( 'description' )
    postDate = descriptionNode.attribute( 'date' )
    descriptionText = descriptionNode.text

    html += '<div>'
    html += '<h1>' + title + '</h1>'
    html += '<p>' + descriptionText + '</p>'
    html += '<p><em>' + postData + '</em></p>'
    html += '</div>'

html += '</body></html>'
render(html)

While this is straightforward enough in the trivial example, it falls apart when parsing large, complex documents. Consider now the XSLT equivalent:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0">
    <xsl:template match="/">
        <html>
            <body>
                <xsl:apply-templates />
            </body>
        </html>
    </xsl:template>
    <xsl:template match = "post">
        <div>
            <xsl:apply-templates />
        </div>
    </xsl:template>
    <xsl:template match = "title|description">
        <p><xsl:apply-templates /></p>
    </xsl:template>
    <xsl:template match = "content">
        <p><xsl:apply-templates /></p>
        <p><em><xsl:value-of select="@date" /></em></p>
    </xsl:template>
</xsl:stylesheet>

Although the trivial example is longer in terms of characters, there are numerous benefits to this approach.

1. The transformation is externalized. If you decide that all post divs now need a “post” class, you don’t need to recompile, and you know exactly where to make the change.
2. Presumably, your XSLT editor can validate XML, or at least tell you when you’re missing a closing angle bracket or two. Most IDEs will not see unclosed angle brackets within strings as a problem, so in you get a proper environment in which to craft your output HTML. Sure, you could use an XML library that hides this implementation from you, but you still run the risk of missing an element or forgetting to append nodes to one another. This wastes time. With a XSLT document, you can very clearly easily about your output before testing. This emphasizes another issue.
3. Good XSLT editors will let you quickly and easily run XSLT against XML. Using oXygen, I was able to develop the above document in under a minute. Actually testing manually-manipulated XML, on the other hand, takes much more time.

All of this said, there are still reasons to avoid XSLT in certain situations. All browsers do not support XSLT, and there are few XSLT 2.0 processors at the moment. Ergo, you must perform your transformations on the server, rather than on the client, but you should probably be doing that anyway. Furthermore, maintenance programmers will need to know XSLT to debug and extend your application, so if you’re worried about limited knowledge handling your software, it may be best to simply avoid XSLT altogether.

I’ve quickly found that XSLT is often the right tool for the job, however, so it would be a sin to not learn enough of the technology to recognize when its applicable and when it isn’t. Don’t craft your HTML by hand, but only use XSLT when it’s appropriate.

Specifically, Spring’s MultiActionController and XOM’s XML building utilities let me write the minimal AccountController given below.

public class AccountController extends MultiActionController
{
private AccountManagementService accountManagementService;

public ModelAndView login( HttpServletRequest request, HttpServletResponse response ) throws IOException
{
   String username = StringEscapeUtils.escapeHtml( request.getParameter( "username" ) );
   String password = StringEscapeUtils.escapeHtml( request.getParameter( "password" ) );
   Document xmlDoc = accountManagementService.login( username, password );

   response.getWriter().println( xmlDoc.toXML() );

   return null;
}

public ModelAndView register( HttpServletRequest request, HttpServletResponse response ) throws IOException
{
   String username = StringEscapeUtils.escapeHtml( request.getParameter( "username" ) );
   String password = StringEscapeUtils.escapeHtml( request.getParameter( "password" ) );
   Document xmlDoc = accountManagementService.register( username, password );

   response.getWriter().println( xmlDoc.toXML() );

   return null;
}

public void setAccountManagementService( AccountManagementService accountManagementService )
{
   this.accountManagementService = accountManagementService;
}
}

The URL mapping for service requests was also dead-simple:

<bean id="urlMappingDeployment" class="org.springframework.web.servlet.handler.SimpleUrlHandlerMapping">
      <property name="mappings">
         <props>
             <prop key="/login.do">accountController</prop>
             <prop key="/register.do">accountController</prop>
             <prop key="/getInventory.do">inventoryController</prop>
             <prop key="/updateInventory.do">inventoryController</prop>
          </props>
      </property>
   </bean>

By utilizing Spring, I saved reams of code, although using a more expressive language like Groovy with Grails may have been quicker and more expressive. At any rate, I’m off to go learn about XSLT. Cheers.

]]> http://www.progamonth.com/?feed=rss2&p=125 0 http://www.progamonth.com/?p=108 http://www.progamonth.com/?p=108#comments Mon, 12 Oct 2009 13:19:22 +0000 Stefan Kendall http://www.progamonth.com/?p=108 If you’re like me, you heard about this language on the Java Posse Pod-cast (link), and you’re just now trying to figure out what all the hubbub is about. Well, Scala is functional, declaration, strongly typed (although it doesn’t feel like it), and most importantly, a JVM language. it compiles to Java classes, and with the inclusion of a single library, you can work Scala quite easily into your web application, either by design or postfixed.

Aside from ease of integration, Scala also offers many functional concepts: first-class functions, curried functions, partially applied functions, and in simple cases, tail recursion! Object-Oriented purists can continue to work as they would in Java, but those of us willing to dabble in the dark-arts can do things like this:

class MyApplication extends Application {
   println( "Hello, World!" )
}

See what’s happening? The closure between the braces is passed as a constructor argument to MyApplication, which then executes the function. It’s not that much of a convenience, but for small applications, public static void main( String[] args ) is just annoying.

Tomcat

Remember how I said Scala could run under Tomcat as easily as Java? Well, I setup a servlet on my server to proxy file downloads. Try this:

http://stefankendall.com:8080/download?url=http://stefankendall.com/files/test.txt

It’s not much to see, but the code is pretty simple!

import javax.servlet.http.{HttpServlet, HttpServletRequest, HttpServletResponse}
import javax.servlet.ServletOutputStream
import java.net.URL
import java.io.{BufferedInputStream, PrintStream, StringWriter, PrintWriter }

class ProxyDownloadServlet extends HttpServlet
{
	override def doGet( request: HttpServletRequest,	response: HttpServletResponse )
	{
		//get the file url to download from the GET request parameter
		val fileUrl = request.getParameter( "url" );

		//if no url was provided, write that to the user
		if( fileUrl == null )
		{
			response.getWriter().println( "<html><body><p>No download url!</p></body></html>" )
			return
		}

		try
		{
			//download the file
			val inputStream = new BufferedInputStream( new URL( fileUrl ).openStream() )

			//set the content type of the response so the
			//user will be prompted to download
			response.addHeader( "Content-Type","application/octet-stream");

			//get the output stream
			val outputStream = response.getOutputStream();

                                  //data buffer
			var buffer = new Array[byte](4 * 1024)

			var bytesRead = inputStream.read( buffer )
			while( bytesRead != -1 )
			{
				outputStream.write( buffer, 0, bytesRead )
				bytesRead = inputStream.read( buffer )
			}
		}
		catch
		{
			case e => {
				println( "Attempted to download:" )
				println( fileUrl )
				e.printStackTrace()
				response.setContentType("text/html")
				outputStream.println( "<html><body><p>Bad file url!</p>" )
				outputStream.println( "<p>File: " )
				outputStream.println( fileUrl );
				outputStream.println( "</p>" )
				outputStream.println( "</body></html>" )
			}
		}
		finally
		{
			inputStream.close()
			outputStream.close()
		}
	}
}

I had some trouble getting things like String concatenation working under Tomcat (Yeah, I know.), and the tools for Scala are rather weak right now (at least the eclipse plugin), but it’s definitely not Java! There’s probably a more Scala way to do this, but that just illustrates the point: you can use as much, or little, Java-isms as you see fit. Remember that this compiles to .class files, so it can mingle with other Java servlets as easily as other Java could.

]]> http://www.progamonth.com/?feed=rss2&p=108 0 http://www.progamonth.com/?p=93 http://www.progamonth.com/?p=93#comments Wed, 16 Sep 2009 19:26:08 +0000 Stefan Kendall http://www.progamonth.com/?p=93 Doesn’t that subject line look crazy? Well, in Objective-C, methods, properties, and members are accessed with maniacal brackets. This is fine, aside from the fact that you need to know how deep you’re going to chain a method call BEFORE you actually do it, as otherwise you have to go back and add the correct number of left brackets. Objective-C 2.0, as supported on the iPhone, supports dot notation, but when in Rome, I hear you use crazy bracket syntax. So, then, how does this all play out? Well, iPhone development can really be sectioned off into its constituting parts: Objective-C programming, Interface Builder, and release to the app store on iTunes.

Objective-C

Objective-C 2.0 has properties, strong typing, and memory management. Objective-C for the iPhone only has the first two. For this reason, we’re forced to deal with memory management or deal with the terrible results. In making a simple business calculator, I was able to leak 6 UI elements each time the program was run and take my 3GS iPhone down to a few MB of remaining ram in only 7 or so test runs. The solution, of course, was simply releasing the UI objects when the application was closing, but this is a significant annoyance when trying rapidly design software.

True properties, of course, with synthesized property methods, save a decent amount of time and just look nicer than having your IDE generate getters and setters.

Consider the following:

//Tester.h
@interface Tester
{
   MyClass * myClass;
}
@property MyClass * myClass;
@end

//Tester.m
@implementation Tester
@synthesize myClass;
@end

Now, [testObj myClass] calls the implicitly created getter of Tester, and [testObj setMyClass: myClass] calls the implicitly generated setter. Isn’t that cool?

iPhone Development

Aside from the Objective-C nonsense, the iPhone SDKs and Interface Builder make life pretty easy for app development. Interface Builder provides a full visual interface for designing iPhone applications, and XCode even provides an iPhone emulator with which to test applications. Sure, the emulator won’t tell you if you have a memory leak, or anything of that nature, but there are other tools to monitor such.
/* TODO: add screenshots */

iTunes and iTunes Connect

The release process to iTunes, while relatively painless, requires a wealth of information and takes about two weeks to process (at least it did for FoodWatcher, my “Hello World” iTunes application to measure weightwatchers points).

The following is required to submit an application to the app store:

• application name
• application version
• 512×512 image to represent the application. This is used if the application gets “featured.”
• 57×57 iPhone icon for the application. You can either add your own gloss or let the iPhone render it for you.
• At least 1 screenshot of the application in progress
• A support website for the application
• A support email for the application
• A list of user names/passwords if your application requires it

and for non-free applications, you need to provide full tax/bank information. The process is a bit of a hassle, but the requirements make sense given the context of iTunes and the App Store.

]]> http://www.progamonth.com/?feed=rss2&p=93 0 http://www.progamonth.com/?p=56 http://www.progamonth.com/?p=56#comments Wed, 19 Aug 2009 01:39:22 +0000 Stefan Kendall http://www.progamonth.com/?p=56 Lorum Ipsum and such

jQuery has plugins.

Lots of them, in fact. The jQuery plugin page lists hundreds of additions to the base jquery package. These plugins make web development even easier and more glamorous than jQuery alone, doing splendid things like Show dialogs.

Now isn’t that cool? jQuery plugins let external authors write neat libraries to extend the language, much as firefox extensions improve the core of firefox. As jQuery has become the javascript go-to tool of choice, jQuery plugins have become the library tools of choice for productive, external-library-oriented developers, such as you and me. So fly, ye’ programmer, and explore the world of jQuery plugins.

]]> http://www.progamonth.com/?feed=rss2&p=56 0