Monday, August 9, 2010

Continuous Integration for iPhone

I was talking with a co-worker recently about Continuous Integration stacks for various platforms (because we are interesting like that, and because Chariot is running a CI event this fall). When we got onto the subject of iPhone he was surprised to hear how much of a complete stack was available, including unit, integration tests and full support for mocking, all of which we can bundle into our CI build.

A brief run down of what we currently use:
  • CruiseControl.rb - CruiseControl.rb is a continuous integration tool
  • OCUnit - unit test framework already integrated with XCode 3.2.3
  • OCMock - Objective-C implementation of mock objects
  • iCuke - Integration testing for iPhone applications with Cucumber
Admittedly the iCuke part is relatively new, only 5/6 months old, and there have been no updates in the last month. We haven't used this on a project before but the potential is there. There is always potential when using Cucumber :)

This post is part of a series which guide you through setting up the individual pieces. After all, having continuous builds is sweet, but have having continuous builds of fully tested builds is sweeter.
Check back for the sister post to this where we will look at testing Android with JUnit andRobotium, then hooking that up into a CI environment, maybe CruiseControl.rb for consistency or maybe Hudson for a change :)

Monday, August 2, 2010

Machine Learning: Google Prediction API

Loosely speaking, machine learning is using a computer to recognize patterns in data, and then make predictions about new data based on what it has learned. It is like a marriage between computer science and statistics. Besides its most obvious application (an army of sentient robots which wages war against mankind a.k.a. Skynet), there are a lot of uses for machine learning including:
  • Predicting housing prices
  • Recommendation engine for a retail website based on past customer purchases
  • Spam filter or any document classifier
  • Autonomous vehicles such as cars or helicopters
  • Forecasting electricity demand based on historical data
  • Computers playing chess
The best way to understand how machine learning works is to see an example. Google has released a machine learning software-as-a-service product called the Prediction API. I wrote a simple spam filter using the Prediction API, and it works surprisingly well. The basic usage is 2 steps: train using historical data, predict using new data.

Training sets in machine learning are generally tables or matrixes where each row in the table represents a single training example. The first column in the table is the classification for that example. The classification values of my spam filter are either "spam" for a spam email or "good" for a non-spam email. The additional columns in the table are features of that training example. In the case of predicting house prices, the features would be attributes about the house such as the square footage, number of bed rooms, whether it has a pool, etc. In the case of the spam filter, the text of the email's subject and body are the features. Other features of an email could be header information, but I kept it simple. The Prediction API requires the training set to be a comma-separated file. Here's what the training set looked like for the spam filter:

# classification, email text
good, "Some email text..."
spam, "Do you want to buy a degree?"
good, "Hi mom, ...."
spam, "Click on this shady link..."

To get a decent quantity of real data to test with, I wrote a little IMAP email exporter that downloaded a month's worth of my good email and my spam email from Gmail. My training set was about 3000 emails. I also put 1500 additional emails in another CSV file to use for testing later. It is important that the testing data set be independent of the training set. Then I uploaded the training set to Google Storage which is another cool software-as-a-service product that Google is developing. Google Storage is similar to Amazon S3. It is easy to use, and I like the interface, but that's another topic.

The Prediction API is REST-based. Once the training set is uploaded, you call a simple request to start training:

www.googleapis.com/prediction/v1/train/{my bucket}

There is another call to check the status of the training. For my data set, training only took a couple of minutes. After the training has completed, you make another REST call to get a prediction. The payload of the request is JSON object that has the same features as your training set. My spam filter only had one feature: text. If you used 10 features, the JSON object would have 10 fields. In practice, you can have hundreds or thousands of features.

www.googleapis.com/prediction/v1/train/{my bucket}/predict

{"data":{
"input": {
"text" : ["Want to buy a degree?"]}}}}

The Prediction API returns the most probable classification for that request. The response looks like:

{"data":{
"output" :{
["output_label":"spam"]}}}

Conclusion

I wrote a little Groovy script that ran through 1500 test emails and checked to see if the Prediction API would pick the correct classification. So how did it do? It correctly identified spam 91% of the time. I thought that was really good considering I only used the text of the email as an input feature. I was able to create a simple spam filter that is 91% accurate and it only took me a couple of hours. Keep in mind the Prediction API documentation states that machine learning is the ultimate case of "garbage in, garbage out". The quality of your training examples has a huge effect on the accuracy of your predictions.

Machine learning algorithms are not one-size-fits-all by any means. The art of ML is using the right algorithm for your data and tuning that algorithm appropriately. Google has taken an interesting approach by making the Prediction API appear to be a "universal learner" that works on any data set. During the learning process, it must try several different algorithms and pick the best fit. However Google is keeping mum on the internals of the Prediction API.

I ran into a few snags when I was running my tests, but the Prediction API message boards are responsive and I got help quickly. The Prediction API is currently experimental, so it should interesting to see where Google takes it, and if it sees the light of day as a commercial offering.

Machine learning is an interesting and valuable field that has a lot of uses in software development. In my next machine learning post, we'll look at the open source ML toolkit Weka.

References and Further Reading

Saturday, July 31, 2010

Why Open Source Can Mean Better Support


Why Open Source Can Mean Better Support

One of the first questions enterprise clients ask us when they evaluate an open source platform is, "How good is the support?"

The assumption is that a commercial platform means reliable 24/7 support while open source means you’re more or less on your own.

But the opposite is often true: Not only are many open source platforms backed by enterprise-grade professional support, they are also backed by communities of users ready to respond to questions over the web.

Take the example of Mule ESB, an open source software platform that one of our clients asked us to compare with a major commercial platform.

The client asked us to develop two different proof-of-concept versions of their software, one using Mule ESB and the other using the commercial platform. The client wanted us to report back on the robustness of each platform and the quality of the support.

While working with Mule ESB, we discovered an issue with the SMTP Transport in the Mule ESB Enterprise 2.2.5 version with the Mule Management Console (MMC) agent installed (we would learn that the problem was with the MMC agent and not with the core Mule ESB).

MuleSoft, the makers of Mule ESB, not only have an enterprise-level support option, but also an active community support forum where developers offer peer-to-peer troubleshooting.

We posted our question to the community forum and received a response saying that we had likely encountered a bug. The user recommended that we log the bug into MuleSoft's bug tracking system.

After logging the bug, MuleSoft sent us this solution to our problem within four hours. A member of our team has since written a more detailed technical explanation on the solution on his blog.

The lesson here is that open source platforms have an advantage when it comes to support: a layer of community-powered troubleshooting on top of traditional paid support.

So when it comes time to compare open source and commercial options, don’t be surprised if support is one of the reasons you decide to go open source.

Wednesday, July 14, 2010

Mobile Musings from July

July seems to have been a busy month on many fronts in the world of mobile, then again, isn't every month?

Some hardware news that tickled my fancy:
The continued hubbub around the Dell Streak: you can now sign up for an 'exclusive' 24 hour pre-sales period. It was also spotted on engadet running Android 2.1, was slated for a 'July end' releas; but wait... It is currently on the Dell site for sale "soon". 2

BlackBerry bought the "BlackPad" domain, and it is rumored to be a 9.7 inch tablet with a November 'introduction'. BlackBerry also scheduled a August 3rd event which is supposedly to introduce a new blackBerry slider running OS6 (<-- that's the exciting bit). Personally I can't wait to see either of these devices, especially if they are running OS6; which, from various screen shots and movies floating around the information super highway look's really, really nice. It would be great to see BlackBerry make some (much needed) ground up on the iPhone and Android which seem to be having their way with the mobile market currently. 3

Speaking of BlackBerry OS6, some media for your perusing pleasure:

HP applied for a "PALMPAD" trademark which would cover "Computers, computer hardware, computer software, computer peripherals, portable computers, handheld and mobile computers, PDAs, electronic notepads, mobile digital electronic devices", not sure what else needs to be said really :) 5 6

In software news my favorite announcement came from the Symbian who are going to be partnering with PhoneGap. This will presumably mean effortless support for devices running Symbian, meaning you can have most of your mobile platform bases covered with PhoneGap. I wrote an article a while back regarding 'Rapid Application Prototyping with PhoneGap', and this news seems to solidify it's place as a framework to seriously consider if you are developing for multiple platforms. 7

The fun news from July was seeing Ubuntu on the Nexus one, however this also came with sad news; you can no longer buy the device :( 8 9

Monday, July 12, 2010

Spring Roo - News from Ken Rimple and Gordon Dickens

Chariot's Ken Rimple and Gordon Dickens are following in the path of several of our architects - they are becoming published authors. Both are working, along with Ben Alex, on a book for Manning, Spring Roo in Action.

Here is a video interview with them. They give a brief overview of Roo and discuss why they wanted to write the book.



(sorry for the link, but YouTube was giving me trouble with the embedding today.)

As they said in the video, you can get the early chapters of the book at:
http://www.manning.com/dickens. It may not have been clear in the video.

We will have more interviews with Ken and Gordon on Roo and Spring Integration. As you can tell, they are pretty passionate about the topic.

Thursday, July 1, 2010

Mobile Musings from June

June has been a busy month for mobile, primarily because of Apple's launch of iOS4 and the new iPhone (antenna woes and all). However, presenting an update on that would be too easy; here are a few things you might have missed in all the hubbub.

CNET reported that merely weeks after Microsoft launch of Kin, they would be stopping development, and focusing on Windows Phone 7. 1

Android released 2.2 to developers with the notable additions being:
  • Improved performance
  • Use as a portable hotspot (certain devices)
  • Exchange support
  • Multiple keyboard languages 2
Rhodes 2.0 was launched under its new MIT license allowing developers far more flexibility and a broader scope to use Rhodes in developing applications. RhoSync remained under Rhomobile commercial license. The launch of Rhodes 2.0 brought with it some new features, primarily the metadata framework, which I have yet to get my teeth into.3

MeeGo, the joint venture between Intel's Moblin and Nokia's Meamo announced its "Handset Project Day 1" complete with video (http://www.youtube.com/watch?v=EW5wpg5epMs), which I thought looked pretty darn slick. 4

I'll be keeping a closer eye on MeeGo as its growth continues, and looking out for new Intel Atom-based devices, until then though; I'll just have to hope Santa comes early with a new Aava smartphone for me :)

Tuesday, June 15, 2010

Making the Most of Maven: Nexus, Hudson, Sonar

Maven is a build tool. I like Maven because it supports dependency management, consistent builds and a modular code base. But if you are just using Maven without using Nexus, Hudson, Sonar you are really missing out on a lot of goodness.

Nexus is a Maven repository manager, Hudson is a CI server, and Sonar is a code metrics server. All three share some commons attributes:
  1. They are web-based server applications
  2. They have an open source license
  3. They are easy to install -- just unzip and run the embedded app server (Jetty or Winstone) or deploy as a WAR into your existing app server
  4. They have very professional looking user interfaces. Don't underestimate the value of a good paint job when getting buy-in for a new product.
  5. They are fairly easy to extend. I've written plugins for all three.
  6. They have active communities
  7. They have commercial support available if you want it
  8. They are continually being updated with new releases containing features and bug fixes
OK, great -- all three are web-based and awesome. But what do they actually do?


Nexus is a Maven repository manager. A vanilla Maven repository is just a web server with a directory structure of versioned jars with some XML metadata. The default Maven repository is a server called "central" with the URL: http://repo2.maven.org/maven2. Maven central is a good repository of a large number of open source jars.

However many organizations don't want their builds to be dependent on and constantly downloading jars from central, which is a public Internet site, for performance, security, and even legal purposes. What if central goes down or gets really slow? What if central gets hacked? What if developers use a jar in central that is licensed in a way your organization doesn't permit?

Running your own Maven repository that caches jars from central is a good idea. And what about your various projects and the jars/wars they produce? Running your own Maven repository within your organization allows you to host and version your own jars and make full use of Maven dependency management features.

While you can use any old web server as a Maven repository, there are many administrative headaches to consider:
  • Security -- who has access to what
  • Searching -- how can you find out what's in the repository
  • Purging old SNAPSHOT versions -- eventually your web server will fill up with old SNAPSHOTs jars
  • Scalability -- what happens when you can't fit everything on a single web server
  • Maintenance of settings.xml -- restructuring repositories on your server can force you to push out a new version of settings.xml to all of clients
Nexus is a Maven repository + a bunch of administrative features that address all of these problems.
  • Security -- Nexus has a fine-grained roles/permissions model, and integrates with LDAP
  • Searching -- Nexus has a search feature based on Lucene. You can search for a particular artifact by group ID and artifact ID, or you can search for a particular class and find the jar(s) that contains that class.
  • Purging old SNAPSHOTs -- Nexus allows you to schedule jobs to perform periodic tasks like purging old SNAPSHOTs
  • Scalability -- Nexus has a proxy feature that allows you to run multiple instances that proxy to each other for redundancy or just to distribute load.
  • Maintenance -- You can create logical repositories, called groups, which are the aggregate of many physical repositories. The effect is similar to DNS -- you give a logical name to something that never changes which allows you to change the underlying physical structure without adversely effecting users. This means you don't have to change the repositories listed in settings.xml for your users as often.
  • Programmable API -- Nexus provides a REST API to perform most functions, and even allows you to extend the REST API via plugins which is very useful.

Hudson is a continuous integration or CI server. If you aren't familiar with CI, the goal of CI is to shorten the feedback loop from the time a developer makes a change until the time he/she sees the result. In practice this means a CI server periodically polls a version control system at short intervals, e.g. 15 minutes, and if it detects any code changes, it downloads the new code and tries to build it. If the build fails, then emails are sent to everyone on the development team pointing out the offending code and person who did it. The principle is "Better code through shame" and it works well.

There are other open source CI servers in existence. CruiseControl is a popular one, and was one of the first. The advantage Hudson has over any other that I've used is ease of use. It literally takes me 2 minutes or less to setup a new Hudson job via the user interface -- all you need is the Subversion URL of the project that you want Hudson to watch and continuously build. Hudson also has a decent security model although it isn't as flexible as Nexus'. The Hudson REST API is very useful for managing large numbers of Hudson jobs programmatically. And Hudson has a large number of useful, free plugins.


Sonar is a code metrics server. It analyzes your code using a variety of metrics (cyclomatic complexity, Bob Martin's OO metrics, dependency cycles, duplicate code, lines of code, etc) and runs various rules checkers to detect bad code such as PMD, Checkstyle, and FindBugs. Then it tracks those metrics and rules violations over time so you can see whether your code is getting better or worse.

Similarly to Hudson, the best part about Sonar is that it is incredibly easy to use. To get it setup, you first run a Sonar server instance. Then you use the Sonar maven plugin to analyze your Maven project by running "mvn sonar:sonar" and *magically* your code gets analyzed and the results appear in a beautiful dashboard on the Sonar server. That's it. And to make this even easier, there is a nice Sonar Hudson plugin that allows you run Sonar on any Hudson job by checking a checkbox. Gathering metrics and trending them over time across many projects is generally a huge hassle so it never gets done. With Sonar, it is completely built into your CI process so it couldn't be easier.

Conclusion

I've been using Nexus, Hudson, and Sonar for about 2 years now, and I wouldn't consider using Maven without them. They are easy to use and provide so much value -- especially considering they are free open source -- that they really are worth an evaluation if you've never used them.