Monday, August 30, 2010

Monthly mobile musings for August

August, appeared to me, to be the month of 'Samsung'. They released/previewed/teased numerous handsets including the Seek, Flight II, Eternity II, Epic and Galaxy, and a few days ago released the Bada 1.0 SDK into the wild. Checkout the Samung Bada home for more details. There is also a developer book to go with it available via Amazon coming in September.

More importantly they put up a teaser video for the 'Galaxy Tab' which made me think I want one, but I have no idea why.

Not really August related directly, but I stumbled across another cross platform mobile development framework, the dual license product 'Sencha Touch' from Sencha. A HTML 5, CSS 3, and Javascript framework, it boasts an impressive list of enhanced touch events like:
  • Tap
  • Double Tap
  • Swipe
  • Tap and hold
  • Pinch and
  • Rotate
There is a decent article on InfoQ which included Sencha and some of the other usual suspects when it comes to mobile development frameworks.

Something else that caught my attention during August was the 'App Inventor for Android'. I'm now eagerly awaiting my login :)

Monday, August 9, 2010

Continuous Integration for iPhone

I was talking with a co-worker recently about Continuous Integration stacks for various platforms (because we are interesting like that, and because Chariot is running a CI event this fall). When we got onto the subject of iPhone he was surprised to hear how much of a complete stack was available, including unit, integration tests and full support for mocking, all of which we can bundle into our CI build.

A brief run down of what we currently use:
  • CruiseControl.rb - CruiseControl.rb is a continuous integration tool
  • OCUnit - unit test framework already integrated with XCode 3.2.3
  • OCMock - Objective-C implementation of mock objects
  • iCuke - Integration testing for iPhone applications with Cucumber
Admittedly the iCuke part is relatively new, only 5/6 months old, and there have been no updates in the last month. We haven't used this on a project before but the potential is there. There is always potential when using Cucumber :)

This post is part of a series which guide you through setting up the individual pieces. After all, having continuous builds is sweet, but have having continuous builds of fully tested builds is sweeter.
Check back for the sister post to this where we will look at testing Android with JUnit andRobotium, then hooking that up into a CI environment, maybe CruiseControl.rb for consistency or maybe Hudson for a change :)

Monday, August 2, 2010

Machine Learning: Google Prediction API

Loosely speaking, machine learning is using a computer to recognize patterns in data, and then make predictions about new data based on what it has learned. It is like a marriage between computer science and statistics. Besides its most obvious application (an army of sentient robots which wages war against mankind a.k.a. Skynet), there are a lot of uses for machine learning including:
  • Predicting housing prices
  • Recommendation engine for a retail website based on past customer purchases
  • Spam filter or any document classifier
  • Autonomous vehicles such as cars or helicopters
  • Forecasting electricity demand based on historical data
  • Computers playing chess
The best way to understand how machine learning works is to see an example. Google has released a machine learning software-as-a-service product called the Prediction API. I wrote a simple spam filter using the Prediction API, and it works surprisingly well. The basic usage is 2 steps: train using historical data, predict using new data.

Training sets in machine learning are generally tables or matrixes where each row in the table represents a single training example. The first column in the table is the classification for that example. The classification values of my spam filter are either "spam" for a spam email or "good" for a non-spam email. The additional columns in the table are features of that training example. In the case of predicting house prices, the features would be attributes about the house such as the square footage, number of bed rooms, whether it has a pool, etc. In the case of the spam filter, the text of the email's subject and body are the features. Other features of an email could be header information, but I kept it simple. The Prediction API requires the training set to be a comma-separated file. Here's what the training set looked like for the spam filter:

# classification, email text
good, "Some email text..."
spam, "Do you want to buy a degree?"
good, "Hi mom, ...."
spam, "Click on this shady link..."

To get a decent quantity of real data to test with, I wrote a little IMAP email exporter that downloaded a month's worth of my good email and my spam email from Gmail. My training set was about 3000 emails. I also put 1500 additional emails in another CSV file to use for testing later. It is important that the testing data set be independent of the training set. Then I uploaded the training set to Google Storage which is another cool software-as-a-service product that Google is developing. Google Storage is similar to Amazon S3. It is easy to use, and I like the interface, but that's another topic.

The Prediction API is REST-based. Once the training set is uploaded, you call a simple request to start training:

www.googleapis.com/prediction/v1/train/{my bucket}

There is another call to check the status of the training. For my data set, training only took a couple of minutes. After the training has completed, you make another REST call to get a prediction. The payload of the request is JSON object that has the same features as your training set. My spam filter only had one feature: text. If you used 10 features, the JSON object would have 10 fields. In practice, you can have hundreds or thousands of features.

www.googleapis.com/prediction/v1/train/{my bucket}/predict

{"data":{
"input": {
"text" : ["Want to buy a degree?"]}}}}

The Prediction API returns the most probable classification for that request. The response looks like:

{"data":{
"output" :{
["output_label":"spam"]}}}

Conclusion

I wrote a little Groovy script that ran through 1500 test emails and checked to see if the Prediction API would pick the correct classification. So how did it do? It correctly identified spam 91% of the time. I thought that was really good considering I only used the text of the email as an input feature. I was able to create a simple spam filter that is 91% accurate and it only took me a couple of hours. Keep in mind the Prediction API documentation states that machine learning is the ultimate case of "garbage in, garbage out". The quality of your training examples has a huge effect on the accuracy of your predictions.

Machine learning algorithms are not one-size-fits-all by any means. The art of ML is using the right algorithm for your data and tuning that algorithm appropriately. Google has taken an interesting approach by making the Prediction API appear to be a "universal learner" that works on any data set. During the learning process, it must try several different algorithms and pick the best fit. However Google is keeping mum on the internals of the Prediction API.

I ran into a few snags when I was running my tests, but the Prediction API message boards are responsive and I got help quickly. The Prediction API is currently experimental, so it should interesting to see where Google takes it, and if it sees the light of day as a commercial offering.

Machine learning is an interesting and valuable field that has a lot of uses in software development. In my next machine learning post, we'll look at the open source ML toolkit Weka.

References and Further Reading

Saturday, July 31, 2010

Why Open Source Can Mean Better Support


Why Open Source Can Mean Better Support

One of the first questions enterprise clients ask us when they evaluate an open source platform is, "How good is the support?"

The assumption is that a commercial platform means reliable 24/7 support while open source means you’re more or less on your own.

But the opposite is often true: Not only are many open source platforms backed by enterprise-grade professional support, they are also backed by communities of users ready to respond to questions over the web.

Take the example of Mule ESB, an open source software platform that one of our clients asked us to compare with a major commercial platform.

The client asked us to develop two different proof-of-concept versions of their software, one using Mule ESB and the other using the commercial platform. The client wanted us to report back on the robustness of each platform and the quality of the support.

While working with Mule ESB, we discovered an issue with the SMTP Transport in the Mule ESB Enterprise 2.2.5 version with the Mule Management Console (MMC) agent installed (we would learn that the problem was with the MMC agent and not with the core Mule ESB).

MuleSoft, the makers of Mule ESB, not only have an enterprise-level support option, but also an active community support forum where developers offer peer-to-peer troubleshooting.

We posted our question to the community forum and received a response saying that we had likely encountered a bug. The user recommended that we log the bug into MuleSoft's bug tracking system.

After logging the bug, MuleSoft sent us this solution to our problem within four hours. A member of our team has since written a more detailed technical explanation on the solution on his blog.

The lesson here is that open source platforms have an advantage when it comes to support: a layer of community-powered troubleshooting on top of traditional paid support.

So when it comes time to compare open source and commercial options, don’t be surprised if support is one of the reasons you decide to go open source.

Wednesday, July 14, 2010

Mobile Musings from July

July seems to have been a busy month on many fronts in the world of mobile, then again, isn't every month?

Some hardware news that tickled my fancy:
The continued hubbub around the Dell Streak: you can now sign up for an 'exclusive' 24 hour pre-sales period. It was also spotted on engadet running Android 2.1, was slated for a 'July end' releas; but wait... It is currently on the Dell site for sale "soon". 2

BlackBerry bought the "BlackPad" domain, and it is rumored to be a 9.7 inch tablet with a November 'introduction'. BlackBerry also scheduled a August 3rd event which is supposedly to introduce a new blackBerry slider running OS6 (<-- that's the exciting bit). Personally I can't wait to see either of these devices, especially if they are running OS6; which, from various screen shots and movies floating around the information super highway look's really, really nice. It would be great to see BlackBerry make some (much needed) ground up on the iPhone and Android which seem to be having their way with the mobile market currently. 3

Speaking of BlackBerry OS6, some media for your perusing pleasure:

HP applied for a "PALMPAD" trademark which would cover "Computers, computer hardware, computer software, computer peripherals, portable computers, handheld and mobile computers, PDAs, electronic notepads, mobile digital electronic devices", not sure what else needs to be said really :) 5 6

In software news my favorite announcement came from the Symbian who are going to be partnering with PhoneGap. This will presumably mean effortless support for devices running Symbian, meaning you can have most of your mobile platform bases covered with PhoneGap. I wrote an article a while back regarding 'Rapid Application Prototyping with PhoneGap', and this news seems to solidify it's place as a framework to seriously consider if you are developing for multiple platforms. 7

The fun news from July was seeing Ubuntu on the Nexus one, however this also came with sad news; you can no longer buy the device :( 8 9

Monday, July 12, 2010

Spring Roo - News from Ken Rimple and Gordon Dickens

Chariot's Ken Rimple and Gordon Dickens are following in the path of several of our architects - they are becoming published authors. Both are working, along with Ben Alex, on a book for Manning, Spring Roo in Action.

Here is a video interview with them. They give a brief overview of Roo and discuss why they wanted to write the book.



(sorry for the link, but YouTube was giving me trouble with the embedding today.)

As they said in the video, you can get the early chapters of the book at:
http://www.manning.com/dickens. It may not have been clear in the video.

We will have more interviews with Ken and Gordon on Roo and Spring Integration. As you can tell, they are pretty passionate about the topic.

Thursday, July 1, 2010

Mobile Musings from June

June has been a busy month for mobile, primarily because of Apple's launch of iOS4 and the new iPhone (antenna woes and all). However, presenting an update on that would be too easy; here are a few things you might have missed in all the hubbub.

CNET reported that merely weeks after Microsoft launch of Kin, they would be stopping development, and focusing on Windows Phone 7. 1

Android released 2.2 to developers with the notable additions being:
  • Improved performance
  • Use as a portable hotspot (certain devices)
  • Exchange support
  • Multiple keyboard languages 2
Rhodes 2.0 was launched under its new MIT license allowing developers far more flexibility and a broader scope to use Rhodes in developing applications. RhoSync remained under Rhomobile commercial license. The launch of Rhodes 2.0 brought with it some new features, primarily the metadata framework, which I have yet to get my teeth into.3

MeeGo, the joint venture between Intel's Moblin and Nokia's Meamo announced its "Handset Project Day 1" complete with video (http://www.youtube.com/watch?v=EW5wpg5epMs), which I thought looked pretty darn slick. 4

I'll be keeping a closer eye on MeeGo as its growth continues, and looking out for new Intel Atom-based devices, until then though; I'll just have to hope Santa comes early with a new Aava smartphone for me :)