Tuesday, November 19, 2013

The DataPhilly Meetups:

So many great user groups, but so little time!  This year I've started going to the DataPhilly meetups, and I think I'm hooked.  The bottom line is DataPhilly talks are very intriguing, expose you to topics you don't encounter everyday, and give you the chance to meet "non-traditional" developers (scientists and statisticians), whose ranks are rapidly growing.  

First off, a lot of the talks typically use Python (and sometimes R):  There are a ton of libraries in Python when it comes to statistics, data analysis, machine learning, NLP, etc. so that's not surprising.  What I like is getting exposure to a language I don't code in frequently and the ecosystem around it.  In addition, a lot of times the real gems are hidden in the questions and discussions during the talks and the lightning round sessions at the end.

What follows is a summary of the last meetup:

The first half was on mrjob (https://github.com/Yelp/mrjob), a pretty lightweight (from what I can tell) map reduce library written in Python that lets you run Hadoop streaming jobs locally, on Amazon EMR (Elastic MapReduce), or your own Hadoop cluster.  The talk started with word counting, which I swear is the "Hello World" of big data, but as promised quickly moved on to discuss detailed and specific examples (backed by code) of how its used at Monetate to generate product recommendations and gather statistics on user behavior.  On the surface it seems super easy to write the MR jobs.  The best part was when they discussed user behavior statistics and calculating variance: a Temple U. professor in the audience got on his soapbox and vehemently warned of the dangers of calculating variance using the "single pass algorithm" with single precision numbers due to underflow and overflow. (More on that at the end)

The second half was on Scrapy (http://scrapy.org/), a Python library - scratch that - actually an "appliance" to perform crawling and web scraping.  It was a super basic intro that went over its high level architecture, spiders, and how to parse the HTML.  For those who didn't know anything about it, it was a decent intro.  Of course "Little Bobby" asked the obvious question of how do you maintain the heaping pile of scraping code?  The speaker mentioned how Monetate used it heavily back in the day when they were smaller, but now some of their clients give them direct data feeds instead so they don't always have to resort to scraping.  Its sort of crazy to to think how much data is still "trapped" in web pages.  I don't think that problem is going away anytime soon. 

At the end of the night, two people presented during the lighting round:

1. "How airline crew schedules are made" - The speaker presented a modern twist on this combinatorial problem that now tries to minimize pilot fatigue when generating schedules.  He discussed the features of a schedule that take this into account (pilot awake time, trips at night, # of timezones crossed, etc) and combining these features along with empirical data on fatigue (cognitive tests based on sleep deprivation, etc.) to come up with a  "best fit" formula to use in a cost function.  Hey pilots are human - they get sleepy and jet lagged just like the rest of us!

2. The aforementioned Temple U professor quickly set up an example in R to back up his claim earlier, showing how dangerous it is to calculate variance with single precision numbers and using the single-pass algorithm - you can end up with NEGATIVE variance, which makes no sense.  He then showed it again using double precision and the two pass algorithm.  Here's a wikipedia link that describes it in more detail: http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance  -  What I personally took away from this: when you perform anything more than very simple computations, know what the library does under the hood and use Google to find out if there are good numeric algorithms out there already.  Remember computers are limited precision machines!

BTW, last month, one of the talks was on Python scikit-learn (http://scikit-learn.org/stable/) - a machine learning library.  The example used Wikipedia articles written in different languages as training data into scikit-learn and showed a program that identifies which language an arbitrary body of text is written in.  It was a gentle intro to machine learning, "document vectorizing", and classification.

Here is a link to the meetup:  http://www.meetup.com/DataPhilly/

Seriously, go check it out!

Thursday, October 31, 2013

5 Tips for Big Software Projects

Software development isn't easy.  And the bigger the software gets, the harder it is to build right from the ground up.  A string of high-profile failures has given us a timely reminder of this.  But it's not hopeless -- here are five things to consider to bump the odds in your favor:

1. Great software is never built on a poor architecture

The only way to build a large project is to divide and conquer.  But ten teams working in ten different ways will just produce ten times the mess.  The bigger a project gets, the more important it is to build on the best architecture and tools, modularizing where it makes sense without simply introducing more discrepancies and more complexity.  And the more people working on top of an architecture, the more important that it doesn't introduce obstacles that every developer may find different ways to avoid.

2. Generated code is terrible

Many situations land programmers with large amounts of tedious boilerplate code.  The gut reaction is to code-generate your way past it.  But if you think a junior programmer writes bad code, you should see a programmer try to program a computer to program.  It's just as convoluted as it sounds.  Worse, it has to be done over and over again, and it's one way to guarantee that some large chunk of code is rarely up to date.  Whether it's persistence or XML or other boilerplate code, there's a different set of tools that will eliminate the tedium with better code instead of worse code.

3. Best practices aren't always best

If you want to be really cruel, you can write terrible code that adheres to multiple best practices, and just watch an architect splutter.  The problem is that many individual best practices don't make sense when used outside of their associated tools and frameworks, or when used in combination, or when otherwise taken out of context.  Part of what makes experienced programmers better is that they can review all the tools and practices at their disposal, and use their judgement to select the right combination.

4. Software without automated tests is just a pile of code

A large project is built, never adequately tested, and rushed into production.  It doesn't work well.  Why is this a surprise?  But testing is also tedious, and requires a whole additional staff.  That is, if you don't automate it from the beginning.  Your computer is perfectly capable of driving an application with simulated load, an actual Web browser, or real-world integration requests.  There's no reason not to build a comprehensive automated test suite to ensure the software operates as intended.  And the more people who have their hands in the code, the more important it gets.

5. Integration begins with testing

Projects with many integration points tend to have a large number of interested parties, many of whom may be developing their parts of the system in parallel.  Then everybody finishes, and guess what?  It doesn't all work together.  But there's an easy way to avoid that.  If everybody starts by providing simulated requests and replies then testing can begin immediately, and each group can phase in real implementations over time.  Each system gets immediate feedback as to whether anything's breaking, and there's much less debugging required to fix a small change from a working state.

Friday, September 20, 2013

Come to Chariot's Data I/O Event on October 30th and learn about all-things data

We are holding an all-day event on October 30th, downtown in the Philadelphia Cira Centre, that shines a light on large-scale data processing and application management. In this article I'm going to explain a bit about the event's goals, and some information on the speakers and talks we've been lining up.

First, the logistics. You can find out the full details and register on this page, but here are the basics:

Date/Time, etc

When: October 30, 2013
Where: The Cira Centre, Philadelphia, PA
Cost: $80 - includes breakfast and lunch

Speakers include

  • Camille Fournier - Camille is Head of Engineering at Rent the Runway, and a committer for Apache Zookeeper. She's focused on open-source infrastructure. Zookeeper is a set of glue services that keeps a distributed application platform like Hadoop running across multiple nodes.
  • Lars George - Lars George is speaking about big data from experience. He's been working on the HBase, a Hadoop database tool, since 2007, serving as the EMEA (Europe, the Middle East, and Africa) Chief Architect for Cloudera. He also authored HBase - the Definitive Guide.
  • Lance Ball - The word on the street lately has been all about node.js and Javascript. Node uses a great event-driven model to process web requests. However, while Node has gotten all the hype, Vert.x has been quietly creating a storm. Vert.x is also event-driven, but it easily scales - using the JVM and the Hazelcast-based clustering mechanism to quickly expand and manage application nodes. Vert.x isn't just programmable in Javascript, but in Java, Groovy, Ruby and even Python. It can run stand-alone, in clusters, even embedded in your own applications. Lance is the creator of Vert.x, which recently moved to the Eclipse Foundation where it enjoys a vendor-neutral standing. It is girded by the Java Netty fast native-I/O API. Come see how you can get Polyglot programming, Java, Javascript, AND scaling all in one platform.
  • Max De Marzi - Max is passionate about graph-based data. Any items that can be related together in a relational model usually have inferred hierarchy, but are stuck in the world of the Tuple. In neo4j, the Graph-based database from Neo Technology, you design your datasets using objects and relationships in a graph-oriented way. Max will review some key use cases, such as social networking, recommendation engines, personalization, and a number of others to see how you can leverage Neo4j to quickly handle large, interconnected datasets.
  • Claudia Perlich - Claudia was a keynoter at Chariot's 2013 Emerging Technologies for the Enterprise conference, where she talked about data science as an industry. She has a strong data analysis background, holding a PhD in Information Systems from NYU, and is the Chief Data Scientist at Media Six Degrees. Claudia holds several patents, has spoken at dozens of conferences and is an author on a number of papers, journals and books.
  • Grant Ingersoll - As a founder of LucidWorks, he supports the active Lucene search community. He is a committer to Solr, which is an enterprise search platform that allows for full-text searching of large volumes of traffic.
  • Eric Snyder Amazon Redshift is a SQL-based clustered data warehouse with instant on-demand scaling and pay-as-you-use techniques. Eric will show you the pros and cons of this service.
  • Walt Mankowski - Angle - Walt Mankowski will review several Python APIs that can do math and science calculations at the speed of C/Fortran, but with simplicity of scripting. There is NumPy, a math library that can deal with multi-dimensional data, and SciPy, which extends NumPy with scientific calculation features.

We are awaiting abstracts from a few of our speakers, but hopefully this gives you an idea of the types of topics they'll be speaking about. For $80 and a day of your time, we are providing you with a day of talks in a number of key data analysis and processing areas. Sign up today and secure your seat.

Details and registration links

emergingtech.chariotsoutions.com/dataio2013.

Tuesday, July 30, 2013

Automated Testing of HTML5 Canvas/Single-Page Applications with Geb

This is the second in a series of articles about using automated testing tools on a Canvas-based Web application. Each article covers a different testing tool or technique. (See the first on WebDriver)

The specific application I'm testing is sized for a tablet, and uses a combination of a Canvas taking up roughly 2/3 of the space and a set of JQuery Mobile widgets taking up most of the remaining 1/3 of the space (there's also a small space on top showing a read-only "status bar" with key statistics). The basic concept is an editable diagram in the Canvas, with specific edit controls in the JQuery Mobile area. You can hover, drag, and click parts of the diagram in the Canvas, which can change what's displayed in the edit area, and changes you make in the edit area can add, remove, or change elements of the diagram in the canvas as well as updating the values shown in the status bar.

My goals for the automated testing are:

  • Make it easy to write tests
  • Run the tests in a real browser, to ensure I'm testing what a user would actually see
  • Make it easy to run a suite of tests and drill into specific test failures
  • Make the tests run fast enough that it's not terribly onerous to run before every commit
  • Be able to verify the visible state of the HTML showing on the screen (e.g. in the edit area, in the status area)
  • Be able to verify the underlying diagram model that dictates what is drawn to the Canvas. (I haven't attempted to go as far as testing the actual Canvas state -- as in the pixel color at some specific coordinate. The bugs I get aren't that the Canvas is drawing something that does not accurately represent its model state.)

Summary of Geb

Good: All the advantages of WebDriver (which it uses under the covers), plus lots of syntactic sugar for writing tests, plus you can write functional tests in Spock or traditional tests in JUnit, TestNG, or etc. using the lovely Geb syntax. (WebDriver launches the browser and runs through the tests, clicking on various things on the screen, waiting for screens to appear, executing JavaScript as needed, etc.) Supports multiple browsers.

Bad: There's no built-in way to record Geb tests by interacting with an existing Web site. Inherits the limited WebDriver support for interacting with a Canvas (requiring JavaScript workarounds). The documentation doesn't include the single simple step you need to drive single-page applications.

Bottom Line: All the advantages of WebDriver, plus fixes a lot of the WebDriver limitations. If I was considering WebDriver, I'd definitely use Geb instead.

Detailed Review

Instead of providing many language bindings like WebDriver, Geb focuses on producing an outstanding syntax using Groovy. On the one hand, it means that you have to learn and use Groovy. On the other hand, the syntax is really amazing. I think it's worth the trade.

However, most of the Geb examples assume you are using Spock and writing functional tests. The tests I'm writing are probably best considered integration tests -- they're long sequences of browser activity and I expect it all to work. I find this to be a better fit for JUnit (run a long sequence of mixed actions and validations) than Spock (break up the tests into small given/when/then type groups). So for my purposes I chose to write JUnit tests.

Installing Geb

I chose to set up a Maven project so I could run my tests via "mvn test" as well as generating an IntelliJ project and using IntelliJ to write my test code. The POM has several entries to include Geb, Selenium, JUnit, and Groovy, and to enable the Groovy compiler:

Geb JUnit Maven 3 POM

You should be able to run "mvn install" on that and at least confirm that the plumbing works.

Learning Curve:

Geb makes heavy use of Groovy closures, and the ability to invoke properties and methods that weren't explicitly defined. At first, I just followed the examples in the Book of Geb. Eventually things started to make a little more sense.

I'm not going to describe all the Geb syntax here, as the manual does a decent job of it. I'm just going to hit the things the manual didn't cover for me.

Some of my issues were:
  • The manual does clarify how to get Geb to work with single-page applications. The issue for me is that page transitions are animated and therefore are not instantaneous, so the "at checker" ran before the new page was actually visible (and therefore failed).
  • I had to piece together the config file since I didn't see a complete example
  • I wasn't quite clear on how to spread my tests and page object across Groovy source files
  • I wasn't quite clear on whether to use inheritance or modules to reflect that fact that the Canvas is visible for every "page" (meaning a JQuery Mobile page) of the application
  • It wasn't obvious how to handle a Canvas click that caused a page change, since the basic click operation in WebDriver is broken for the Canvas (as described in the previous article)
  • I wanted to automate the recording of tests like I did for WebDriver directly

Config File

Here's my Geb config file for Firefox. The last two lines are necessary if you don't want to configure them as system properties for all the different ways you might run the tests. Note the code within the "driver" block is running against the WebDriver Java API.

Single Page Applications

Since the browser never requests a new URL, there is no need to use "to" or "via" or set the URL for a page. You can simply point the browser to the base URL with go() and then navigate around within that.

It doesn't mean you don't use multiple "pages" within the Geb tests, though. I created a Page object for every JQuery Mobile screen. I just handle the page transitions via "at SomePage" rather than "to SomePage" to tell the test that I now expect a different JQuery Mobile screen to be showing even though the URL hasn't changed.

My first stab at an "at checker" looked like this:
That doesn't account for the page animation, though. The fix turns out to be easy:
That gives the page 5 seconds (by default) to show up, more than enough. Putting the wait logic in the "at checker" is important because some other things call the at checker as a side effect, and that would have to be avoided if the at checker itself didn't handle the waiting.

Many Pages Sharing a Canvas

Since my Canvas is always visible, no matter what JQuery Mobile page you're on, I ended up creating a base Page class with all the Canvas logic (clicking on the Canvas, clicking various widgets within the canvas such as the menu button or toolbar buttons shown there, etc.). All my other Page objects inherited from that one (except the dialog pages, because you can't interact with the Canvas while a dialog is showing on the screen).

Handling Canvas Clicks

I put my same JavaScript workaround for Canvas clicks in my base Page class:
Then I wanted my toolbar clicks to automatically change the current Page and run the new page's at checker, just like you can do with regular clicks in Geb. I ended up invoking the Canvas click manually, and then returning some innocuous page element that Geb could use its click-and-change-page logic on.

Normal link or button that forwards to a new page:
Toolbar button in the Canvas:
Since the content entry returns a legitimate Geb Navigator object (after clicking the Canvas), you can call click() on the content entry. When you call click on it, it changes the page per the to: NewPage directive (including calling the at checker on the new page), even though the click on #someBoringElement did nothing at all.

And the test ends up looking like this:
That's not strictly necessary -- you could put in all the at checks explicitly, and I suppose it could make the test clearer. But I prefer to have them all run as a convenient side effect of navigating.

Source Code Organization

I ended up putting all my Page definitions in a single TestSupport.groovy file. They were all pretty short and closely related, and I didn't want to clutter up my project with lots of tiny files. I think a consequence of this is that I needed to do a "full build" more often since the dependencies across files weren't as obvious, but builds were fast and that didn't bother me.
For my tests, I created several XYZTest.groovy files, since Maven by default runs tests with the *Test.* naming convention. It was easy to put several related tests in each file with the JUnit @Test annotation:

Recording Tests

There's an up side and a down side here. The up side is, I was able to record tests by including a custom "QA" JavaScript file in my app, which added listeners to buttons and links and pages and so on. It emits the appropriate Geb code to the Web console, which then I copy and paste into a test class. The down side is, it relies a lot on referencing page elements by their ID. That works fine, but it sort of misses the point of the PageObjects pattern. It's very easy in Geb to define all the page widgets in a content block, but when recording the tests, I know that an element with a particular ID was clicked, but I don't know what you decided to call that element in the content block of the page element.

It's possible I could settle on a convention of always naming elements in the content block the same as their ID, but even that avoids some of the nice syntax available with Geb.

For instance, I have a screen with four items in a glorified list, but every time you go to the screen, the specific content of the four items might be different (one screen tries to pick the best four for you, another is the same screen no matter which of many categories you're viewing, etc.). So the IDs are pretty generic, like ListItem0, ListItem1, ListItem2, ListItem3. It makes it easy to loop over them and so on.

I can set up a content entry in Geb like this:
Then my test can click an item and cause an implicit navigation to the detail page and run the at checker for the detail page all very easily:
But again, the test recorder doesn't know about that, so it can only generate code like this:
Does it do the same thing? Yes. But it's more lines, uglier lines, and the test would break if the list IDs ever changed -- exactly the reasons the PageObjects pattern exists.

So I end up with a mix of somewhat uglier tests that I recorded, and somewhat prettier ones that I wrote by hand.

Here's what the recorder script looks like now:
And a generated test:
Whereas a handcrafted test might look a little cleaner:

Running Tests

Running tests is as easy as "mvn test" or right-clicking on a test file or test method in IntelliJ (or however else you prefer to run JUnit tests). I get a summary of tests run, passed, and failed, green or red bars, etc. I find that IntelliJ is a little better at putting the specific error in front of me for a test failure, whereas I have to hunt a little more with Maven, but either way works. It's a big improvement over plain WebDriver code.

Thursday, July 25, 2013

PhoneGap 3.0

PhoneGap 3.0 was released at PhoneGap day in Portland, OR last week. There are some great changes with this new release. Historically PhoneGap and Cordova, the open source project behind PhoneGap, were almost identical. This has changed with the release of 3.0.

Prior to 3.0, PhoneGap was distributed as an archive which was downloaded and unzipped to install. Now, both PhoneGap and Cordova are distributed as command line tools, which are installed via the node.js package manager, npm. Both Cordova and PhoneGap can be installed on your machine at the same time.

$ npm install phonegap -g
$ npm install cordova -g

The command line tools provide scripts to create projects, add platforms (iOS, Android, BlackBerry, Windows Phone, etc), compile and deploy projects. The new structure makes it easier to support multiple platforms with one project.

$ cordova create hello com.example.hello Hello
$ cd hello
$ cordova platform add ios
$ cordova platform add android
$ cordova emulate

With 3.0, the Cordova distribution provides the core functionality to embed a webview (browser) into a native application. Cordova is distributed without any of the plugins installed. This allows you to only install the functionality you need, simplifies your code base, and reduces the amount of code in your app.

To use the Camera APIs, we'd install the camera plugin.

$ cordova plugin add https://git-wip-us.apache.org/repos/asf/cordova-plugin-camera.git

Raymond Camden's blog post about Cordova 3.0 for a list of core plugins.

PhoneGap is a distribution of Cordova, similar to how Safari and Chrome are based on Webkit. The PhoneGap distribution of Cordova come with all of the core plugins installed. Additionally the PhoneGap command line adds features like PhoneGap build support. If you don't have a native SDK installed, the compiling the project can be delegated to PhoneGap Build.

$ phonegap create example
$ cd example
$ phonegap run wp7

Overall these are great changes for PhoneGap, but may require a changes to your workflow. The payoff is that it's easier to create projects, manage plugins, and support multiple platforms.

Friday, July 12, 2013

Android Advanced Cursors

If you have been doing Android development for even a little while, you have probably used the provided SQLite functionality to store and retrieve data in a local database on the Android device.

If not, you can head over to Lars Vogel's excellent tutorial website to see how it all works, or to get a refresher.

For many applications, the basic Cursor is all you need. But what about more advanced requirements? What follows is an overview of some of the more advanced cursor options.

CursorJoiner

If you need data from two or more database tables, the most straightforward thing to do is to use a SQL query that joins the two tables. However, sometimes you already have sources for two separate queries (such as a CursorLoader based on a ContentProvider), and you may wish to just re-use them, without creating a new query (without, e.g., modifying the ContentProvider).

This is where CursorJoiner comes in handy. Unlike the other options listed below, CursorJoiner doesn't produce a new cursor. Instead, it provides an Iterable interface over the joined cursors.

If the only reason that you want to join the two tables is to produce a Cursor, say, for use by a CursorAdapter, then you should probably just stick with writing SQL that joins the two tables. However, if you just want to iterate the data once, then CursorJoiner can be very helpful.

One of the more interesting uses that I've seen for CursorJoiner is for determining what has changed when a LoaderManager.LoaderCallbacks<Cursor> implementation gets an updated Cursor in it's onLoadFinished method.

You can use the CursorJoiner to join the 'old' cursor and the 'new' cursor on their primary key, and then compare them. As you iterate the CursorJoiner, you get a CursorJoiner.Result enum that indicates whether the current row exits in the left cursor, the right cursor, or both.

Here is some sample code that determines what has been added to or deleted from a ContentProvider as a CursorLoader driving a CursorAdapter is updated:

   
@Override
public void onLoadFinished(Loader<cursor> loader, Cursor updatedCursor) {
   if(adapter.getCursor() != null) {
      Cursor oldCursor = adapter.getCursor();
      CursorJoiner joiner = 
         new CursorJoiner(oldCursor, 
             new String[] {Thing.ID},
             updatedCursor,
             new String[] {Thing.ID}
      );
            
      for(CursorJoiner.Result joinerResult : joiner) {
         switch (joinerResult) {

            case RIGHT:
               int thingId = updatedCursor.getInt(
                  updatedCursor.getColumnIndex(Thing.ID)
               );
               
               String thingName = updatedCursor.getString(
                  updatedCursor.getColumnIndex(Thing.NAME)
               );
                      
               Log.d(TAG, "new Thing added - id: " + thingId 
                    + ", name: " + thingName);
            break;

            case LEFT:
               int thingId = updatedCursor.getInt(
                  oldCursor.getColumnIndex(Thing.ID)
               );
               
               String thingName = updatedCursor.getString(
                  oldCursor.getColumnIndex(Thing.NAME)
               );
                      
               Log.d(TAG, "Thing deleted - id: " 
                    + thingId + ", name: " + thingName);
            break;

            case BOTH:
               int thingId = updatedCursor.getInt(
                  oldCursor.getColumnIndex(Thing.ID)
               );
               
               String thingName = updatedCursor.getString(
                  oldCursor.getColumnIndex(Thing.NAME)
               );  

               Log.d(TAG, "Thing unchanged - id: " 
                    + thingId + ", name: " + thingName);
            break;                                   
      }
   }        
   updatedCursor.moveToFirst();
   adapter.swapCursor(updatedCursor);
}


MatrixCursor

MatrixCursor is useful if you have a collection of data that is not in the database, and you want to create a cursor for it.  You simply construct it with an array of column names, and optionally an initial capacity. Then, you can add one row at a time to it by passing either an array of objects or an Iterable to its addRow() method. Then use it like any other cursor:
  
String[] columnNames = {"Column1", "Column2"};
MatrixCursor matrixCursor = new MatrixCursor(columnNames);
matrixCursor.addRow(new String[]{"value1", "value2"});
adapter.swapCursor(matrixCursor);

MergeCursor

MergeCursor allows you to present two or more cursors as a single cursor. You could get almost the same effect by doing a sql UNION query. However, MergeCursor allows you to have different columns in the various cursors, and still merge them.

This is very useful if you are presenting a list of heterogeneous items. I've used it with Emil Sjölander's StickyListHeaders library, using a different cursor for each 'section', all presented to the StickyListHeader implementation as a single cursor via a MergeCursor.

One thing to be aware of when presenting heterogeneous items through a MergeCursor - you'll probably want to add a pseudo-column to each of the cursors to make it easy for your ListAdapter (or whatever will be using the cursor) to be able to determine which type of data it's dealing with on each row. A seemingly little-known trick here is that you can include literal columns when creating a cursor using the ContentProvider's query method (and similar methods in other database classes). Just include the literal in the projection parameter to the query:
  
String projection = {Thing.ID, Thing.NAME, "'TYPE_THING' as TYPE"};
Cursor c = qBuilder.query(mDb,
                projection,
                selection,
                selectionArgs,
                groupBy,
                having,
                sortOrder);

CursorWrapper

Finally, CursorWrapper is a Wrapper class for Cursor that delegates all of its calls to the actual cursor object. As the documentation stated, the primary use for this class is to extend a cursor while overriding only a subset of its methods.

One example for the use of CursorWrapper is when you already have a cursor, but only want to present the first few rows of the cursor to a CursorAdapter. Override the getCount() method to return a number smaller than the actual cursor's count, and the adapter will only present that number of results. This is similar to what you would have gotten had you added a LIMIT clause to the sql that produced the cursor.

With this collection of advanced Cursor tricks, you should be able to accomplish whatever you want with Cursors and Adapters.

Tuesday, July 9, 2013

Automated Testing of HTML5 Canvas Applications with Selenium WebDriver

This is the first in a series of articles about using automated testing tools on a Canvas-based Web application. Each article covers a different testing tool or technique.  (See the second on Geb.)

The specific application I'm testing is sized for a tablet, and uses a combination of a Canvas taking up roughly 2/3 of the space and a set of JQuery Mobile widgets taking up most of the remaining 1/3 of the space (there's also a small space on top showing a read-only "status bar" with key statistics). The basic concept is an editable diagram in the Canvas, with specific edit controls in the JQuery Mobile area. You can hover, drag, and click parts of the diagram in the Canvas, which can change what's displayed in the edit area, and changes you make in the edit area can add, remove, or change elements of the diagram in the canvas as well as updating the values shown in the status bar.

My goals for the automated testing are:
  • Make it easy to write tests
  • Run the tests in a real browser, to ensure I'm testing what a user would actually see
  • Make it easy to run a suite of tests and drill into specific test failures
  • Make the tests run fast enough that it's not terribly onerous to run before every commit
  • Be able to verify the visible state of the HTML showing on the screen (e.g. in the edit area, in the status area)
  • Be able to verify the underlying diagram model that dictates what is drawn to the Canvas. (I haven't attempted to go as far as testing the actual Canvas state -- as in the pixel color at some specific coordinate. The bugs I get aren't that the Canvas is drawing something that does not accurately represent its model state.)

Summary of Selenium WebDriver

Good: It has a model that I like -- you write tests, and when you run them, it launches a browser and runs through the tests, clicking on various things on the screen, waiting for screens to appear, etc. It supports multiple browsers (though with varying quality). You can evaluate JavaScript within the context of the page (e.g. "return mymodel.somevalue;") to get at the underlying state in addition to testing the visible component state.

Neutral: You write the tests in a programming language such as Java, C#, Ruby, or Python. (I used Java.)

Bad: All the examples show a test that runs from as a standalone application. You have some decisions if you want to run from within a unit testing framework -- do you restart the browser between tests (~10 seconds), reload the page (but it's not like a user does that for every action), or try to recover from an unknown page state if a test fails? The detailed event support is poor -- you can cause a mouse move to a specific location, for instance, but not a mouse down, mouse up, or click at specific coordinates. It's fine if all you do is click buttons, but terrible for a Canvas (there is a workaround). As far as I know there's no way to record WebDriver tests from the Selenium IDE or a tool like that.

Bottom Line: If you're willing to write a lot of setup code, you can record and execute tests easily. They run in a browser, 99% like a user would experience. I just hope to find some still higher-level tools that wrap WebDriver to record tests, run multiple tests and show results, etc. I also hope future releases resolve some of the differences in the browser-specific hookups that make tests work differently in different browsers.

Detailed Review

The basic model is that you write a test in code, compile (if needed), and execute the test. The test launches a real browser (Firefox, Safari, etc.) and interacts with it, simulating mouse movements, clicks, etc. This causes the page to function just as if a user was interacting with it. A test can also execute JavaScript within the context of the page, to retrieve information on the JavaScript state, to execute JavaScript functions defined for the page, or whatever.

Installing WebDriver:

Since I'm just writing Java code, all I had to do was put the dependency in my Maven POM:
<dependency>
    <groupid>org.seleniumhq.selenium</groupid>
    <artifactid>selenium-java</artifactid>
    <version>2.33.0</version>
</dependency>
For Chrome support, I also had to download the Chrome driver.

Learning Curve:

My first problem was with the event support. A perfectly suitable API is there for the tests to use:
new Actions(driver).moveToElement(canvas, xWithinCanvas, yWithinCanvas)
                   .click().perform();
However, this API is not reliable. In Firefox, every mouse down, mouse up, or mouse click happens at the center of the element. So the code above produces a mouse move event to the provided x,y, then a mouse move event to the center of the Canvas, then a mouse down, mouse up, and click all at the center of the Canvas. That may be fine for a button, but is unworkable for a Canvas, where you want to be able to hover, click, etc. at a specific location. The situation is even worse in Safari, where it just produces an exception indicating that mouse move events aren't supported. Chrome, meanwhile, works fine.

The workarounds are ugly -- manually dispatching synthesized mouse events using JavaScript:
driver.executeScript("var evt = $.Event('click', "+
                             "{pageX: "+x+", pageY: "+(y+55)+"} );"+
                     "$('#diagramCanvas').trigger(evt);");
Here I'm manually creating a mouse event with pageX and pageY using offsets I happen to know from the provided mouse coordinates. Yuck!

For Firefox for my purposes, things were a little smoother. My Canvas event processing is to record the subcomponent of the diagram that the mouse is over on a mouse move, and do something with the selected subcomponent on a click, so I don't actually use the mouse coordinates on a click event. Since the move event works OK in Firefox, something like this works for me:
new Actions(driver).moveToElement(canvas, xWithinCanvas, yWithinCanvas)
                   .perform();
driver.executeScript("$('#diagramCanvas').click();");
I also ran into some other differences between browsers. In Firefox I could click a widget that closed a JQuery Mobile dialog and immediate click on the diagram. In Chrome, the same occasionally failed unless I introduced a slight delay to allow the dialog to get out of the way.

Writing Tests:

Once I got past that, I started writing tests. Writing one little test was fine, and it was exciting to see it run in the browser. But I quickly determined that it was going to be incredibly obnoxious to write any substantial quantity of tests by hand. Too much code, and languages like Java and Ruby don't describe a user's interaction with HTML very well.

When I ended up doing was:
  1. Write a helper class with methods like waitForFooPage(), clickToolbarButtonFoo(), replaceTextIn(id, text), clickElement(id), clickCheckbox(id), confirmModelState(x, y, z), and so on. All the WebDriver API calls are hidden in the helper class.
  2. Write an additional JavaScript "QA" file that added a bunch of event listeners to emit (via console.log) calls to the helper class -- for instance a click on an a emits a clickElement(id) and a change on an input generates a clickCheckbox(id) or replaceTextIn(id, text) and a JQuery Mobile pageShow generates a waitForFooPage() and so on. Then it adds a listener to the very bottom left-hand corner of the canvas (otherwise unused) to toggle the recording state.
So when that file is included in the page, I can basically record a script, though I still have to copy and paste it into a test file to use. Still much better than writing tests by hand. My recorder has some basic page state validation in there (for instance for screens where the links have different content depending on the current state when you load it -- like a category browse type page). But I still have to write any detailed validation by hand.

The QA JavaScript has a little custom logic but basically ends up looking like this, emitting test code to the console as you interact with the app:
$('#diagramCanvas').click(function(e) {
    if(e.pageX >= 0 && e.pageX <= 10 &&
                e.pageY >= 55+590 && e.pageY <= 55+600) {
        QA.recording = !QA.recording;
        console.log("Recording: "+(QA.recording ? "on" : "off"));
    } else if(QA.recording) {
        console.log('helper.clickOnDiagram('+
                        e.pageX+','+(e.pageY-55)+');');
        QA.lastClickToolbar = false;
    }
});
$('a').on('click', function() {
    if(QA.recording && $(this).attr('id')) {
        var id = $(this).attr('id');
        console.log('helper.clickOnElement(\"'+id+"\");");
        QA.lastClickToolbar = false;
    }
});
$('body').on('change', 'input', function() {
    if(QA.recording) {
        var type = $(this).attr('type');
        if(type === 'checkbox' || type === 'radio')
            console.log('helper.clickOnCheckbox(\"'+
                        $(this).attr('id')+"\");");
        else if(type === 'text' || type === 'number')
            console.log('helper.replaceTextIn(\"'+
                        $(this).attr('id')+"\",\""+
                        $(this).val()+"\");");
    }
});
$('#page1').on('pageshow', function() {
    if(QA.recording && !QA.lastClickToolbar)
        console.log('helper.waitForPageOne();');
});
$('#page2').on('pageshow', function() {
    if(QA.recording && !QA.lastClickToolbar)
        console.log('cw.waitForPageTwo();');
});
And the helper class that it emits commands for looks something like this:
private JavascriptExecutor js;
private WebElement canvas;
private WebElement page1;
private WebElement page2;
private Actions ui;

public TestHelper(WebDriver driver) {
    js = (JavascriptExecutor)driver;
    canvas = driver.findElement(By.id("diagramCanvas"));
    page1 = driver.findElement(By.id("page1"));
    page2 = driver.findElement(By.id("page2"));
    ui = new Actions(driver);
}

public void clickOnDiagram(int x, int y) {
    ui.moveToElement(canvas, x, y).perform();
    clickOnCanvas(x, y);
}
private void clickOnCanvas(int x, int y) {
//    js.executeScript("$('#upperSchematic').click();");
    js.executeScript("var evt = $.Event('click', { pageX: "+
                     x+", pageY: "+(y+55)+" } );\n" +
            "$('#diagramCanvas').trigger(evt);");
}
public void waitForPageOne() {
    waitForPage(page1);
}
public void waitForPageTwo() {
    waitForPage(page2);
}
private void waitForPage(WebElement page) {
    wait.until(ExpectedConditions.visibilityOf(page));
}
public void clickOnCheckbox(String id) {
    // In JQuery Mobile you need to click on the label
    WebElement element = (WebElement)js.executeScript(
                         "return $(\"label[for='"+id+"']\")[0];");
    ui.click(element).perform();
}
public void clickOnElement(String id) {
    WebElement element = page.findElement(By.id(id));
    if(!element.isDisplayed())
        throw new RuntimeException(
            "Cannot click on '#"+id+"' because it is not displayed!");
    if(!element.isEnabled())
        throw new RuntimeException(
            "Cannot click on '#"+id+"' because it is not enabled!");
    ui.click(element).perform();
}
public void replaceTextIn(String id, String text) {
    WebElement element = page.findElement(By.id(id));
    String old = (String) js.executeScript(
                                   "return $('#"+id+"').val();");
    ui.click(element);
    for(int i=0; i<old.length(); i++) {
        ui.sendKeys(Keys.BACK_SPACE);
        ui.sendKeys(Keys.DELETE);
    }
    ui.sendKeys(text);
    ui.sendKeys(Keys.TAB);
    ui.perform();
}
A little copy and paste and I end up with a test method that looks like this:
helper.mainMenu();
helper.clickOnElement("MainMenuNew");
helper.wait(100);
helper.clickOnDiagram(393, 412);
helper.waitForPageXYZ();
helper.replaceTextIn("SomeTextFieldID", "MyNewText");
helper.clickOnElement("SomeEditWidgetID");
confirm(helper.getElementText("SomeSpanID").startsWith("MyNewText"));
...

Running Tests:

I wasn't too interested in writing a dozen tests as command-line executables and running them one after the other. I wanted to run one command and have it run all the tests, counting successes and failures, logging stack traces for any failures, and so on. I really wanted a little GUI to show red and green bars and so on, but it wasn't worth the time to recreate that. I ended up writing a little wrapper to run and capture the results of all my individual tests, but it didn't seem like something I should have to be doing.

The bottom line is that I can issue a command, watch a browser flash through interactions with my app, and get a list of failures and stack traces at the end. When I want to write a new test I include my extra JavaScript file, click the obscure corner of the Canvas to turn on recording, and then interact with my app as usual. At the end, I copy and paste the script from the Web Console into a new source file and add any additional validation I want. Then I can comment out the QA JavaScript and run more tests.

It's what I wanted, but less flashy, and I had to write a lot of setup code to get there.

Here's the code from the master class that runs the tests:
public static void main(String[] args) throws InterruptedException {
    FirefoxProfile profile = new FirefoxProfile();
    profile.setEnableNativeEvents(true);
    DesiredCapabilities capabilities = new DesiredCapabilities();
    capabilities.setCapability("nativeEvents", true);
//    WebDriver driver = new SafariDriver(capabilities);
//    WebDriver driver = new ChromeDriver(capabilities);
    WebDriver driver = new FirefoxDriver(profile);
    driver.manage().window().setSize(new Dimension(1040, 850));
    driver.get("http://localhost:9393/");
    TestHelper helper = new TestHelper(driver);
    helper.waitForDefaultPage();
    List<Throwable> exceptions = new ArrayList<Throwable>();
    // NOT SHOWN: Introspect the test class and run the test methods
    if(exceptions.size() == 0)
        System.out.println("All tests passed!");
    else
        for (Throwable exception : exceptions) {
            exception.printStackTrace();
        }
    driver.quit();
}

Areas for Further Investigation:

The next thing I'd like to try is Geb as a wrapper for WebDriver (Update: see results here.)

It looks like a standard pattern for wrapping all the interesting application actions and state is PageObjects. My gut reaction is that it would have been way too much work to do that for all the "pages" in my application, but it's a much more organized approach and better than just scattering various IDs and coordinates through a bunch of individual tests. I feel like the approach has merit, I just can't put my finger on the line where the provided value overcomes the necessary work. Probably if you're writing all the tests by hand, but I'm not willing to go there anyway.