Showing posts with label jquery. Show all posts
Showing posts with label jquery. Show all posts

Tuesday, July 9, 2013

Automated Testing of HTML5 Canvas Applications with Selenium WebDriver

This is the first in a series of articles about using automated testing tools on a Canvas-based Web application. Each article covers a different testing tool or technique.  (See the second on Geb.)

The specific application I'm testing is sized for a tablet, and uses a combination of a Canvas taking up roughly 2/3 of the space and a set of JQuery Mobile widgets taking up most of the remaining 1/3 of the space (there's also a small space on top showing a read-only "status bar" with key statistics). The basic concept is an editable diagram in the Canvas, with specific edit controls in the JQuery Mobile area. You can hover, drag, and click parts of the diagram in the Canvas, which can change what's displayed in the edit area, and changes you make in the edit area can add, remove, or change elements of the diagram in the canvas as well as updating the values shown in the status bar.

My goals for the automated testing are:
  • Make it easy to write tests
  • Run the tests in a real browser, to ensure I'm testing what a user would actually see
  • Make it easy to run a suite of tests and drill into specific test failures
  • Make the tests run fast enough that it's not terribly onerous to run before every commit
  • Be able to verify the visible state of the HTML showing on the screen (e.g. in the edit area, in the status area)
  • Be able to verify the underlying diagram model that dictates what is drawn to the Canvas. (I haven't attempted to go as far as testing the actual Canvas state -- as in the pixel color at some specific coordinate. The bugs I get aren't that the Canvas is drawing something that does not accurately represent its model state.)

Summary of Selenium WebDriver

Good: It has a model that I like -- you write tests, and when you run them, it launches a browser and runs through the tests, clicking on various things on the screen, waiting for screens to appear, etc. It supports multiple browsers (though with varying quality). You can evaluate JavaScript within the context of the page (e.g. "return mymodel.somevalue;") to get at the underlying state in addition to testing the visible component state.

Neutral: You write the tests in a programming language such as Java, C#, Ruby, or Python. (I used Java.)

Bad: All the examples show a test that runs from as a standalone application. You have some decisions if you want to run from within a unit testing framework -- do you restart the browser between tests (~10 seconds), reload the page (but it's not like a user does that for every action), or try to recover from an unknown page state if a test fails? The detailed event support is poor -- you can cause a mouse move to a specific location, for instance, but not a mouse down, mouse up, or click at specific coordinates. It's fine if all you do is click buttons, but terrible for a Canvas (there is a workaround). As far as I know there's no way to record WebDriver tests from the Selenium IDE or a tool like that.

Bottom Line: If you're willing to write a lot of setup code, you can record and execute tests easily. They run in a browser, 99% like a user would experience. I just hope to find some still higher-level tools that wrap WebDriver to record tests, run multiple tests and show results, etc. I also hope future releases resolve some of the differences in the browser-specific hookups that make tests work differently in different browsers.

Detailed Review

The basic model is that you write a test in code, compile (if needed), and execute the test. The test launches a real browser (Firefox, Safari, etc.) and interacts with it, simulating mouse movements, clicks, etc. This causes the page to function just as if a user was interacting with it. A test can also execute JavaScript within the context of the page, to retrieve information on the JavaScript state, to execute JavaScript functions defined for the page, or whatever.

Installing WebDriver:

Since I'm just writing Java code, all I had to do was put the dependency in my Maven POM:
<dependency>
    <groupid>org.seleniumhq.selenium</groupid>
    <artifactid>selenium-java</artifactid>
    <version>2.33.0</version>
</dependency>
For Chrome support, I also had to download the Chrome driver.

Learning Curve:

My first problem was with the event support. A perfectly suitable API is there for the tests to use:
new Actions(driver).moveToElement(canvas, xWithinCanvas, yWithinCanvas)
                   .click().perform();
However, this API is not reliable. In Firefox, every mouse down, mouse up, or mouse click happens at the center of the element. So the code above produces a mouse move event to the provided x,y, then a mouse move event to the center of the Canvas, then a mouse down, mouse up, and click all at the center of the Canvas. That may be fine for a button, but is unworkable for a Canvas, where you want to be able to hover, click, etc. at a specific location. The situation is even worse in Safari, where it just produces an exception indicating that mouse move events aren't supported. Chrome, meanwhile, works fine.

The workarounds are ugly -- manually dispatching synthesized mouse events using JavaScript:
driver.executeScript("var evt = $.Event('click', "+
                             "{pageX: "+x+", pageY: "+(y+55)+"} );"+
                     "$('#diagramCanvas').trigger(evt);");
Here I'm manually creating a mouse event with pageX and pageY using offsets I happen to know from the provided mouse coordinates. Yuck!

For Firefox for my purposes, things were a little smoother. My Canvas event processing is to record the subcomponent of the diagram that the mouse is over on a mouse move, and do something with the selected subcomponent on a click, so I don't actually use the mouse coordinates on a click event. Since the move event works OK in Firefox, something like this works for me:
new Actions(driver).moveToElement(canvas, xWithinCanvas, yWithinCanvas)
                   .perform();
driver.executeScript("$('#diagramCanvas').click();");
I also ran into some other differences between browsers. In Firefox I could click a widget that closed a JQuery Mobile dialog and immediate click on the diagram. In Chrome, the same occasionally failed unless I introduced a slight delay to allow the dialog to get out of the way.

Writing Tests:

Once I got past that, I started writing tests. Writing one little test was fine, and it was exciting to see it run in the browser. But I quickly determined that it was going to be incredibly obnoxious to write any substantial quantity of tests by hand. Too much code, and languages like Java and Ruby don't describe a user's interaction with HTML very well.

When I ended up doing was:
  1. Write a helper class with methods like waitForFooPage(), clickToolbarButtonFoo(), replaceTextIn(id, text), clickElement(id), clickCheckbox(id), confirmModelState(x, y, z), and so on. All the WebDriver API calls are hidden in the helper class.
  2. Write an additional JavaScript "QA" file that added a bunch of event listeners to emit (via console.log) calls to the helper class -- for instance a click on an a emits a clickElement(id) and a change on an input generates a clickCheckbox(id) or replaceTextIn(id, text) and a JQuery Mobile pageShow generates a waitForFooPage() and so on. Then it adds a listener to the very bottom left-hand corner of the canvas (otherwise unused) to toggle the recording state.
So when that file is included in the page, I can basically record a script, though I still have to copy and paste it into a test file to use. Still much better than writing tests by hand. My recorder has some basic page state validation in there (for instance for screens where the links have different content depending on the current state when you load it -- like a category browse type page). But I still have to write any detailed validation by hand.

The QA JavaScript has a little custom logic but basically ends up looking like this, emitting test code to the console as you interact with the app:
$('#diagramCanvas').click(function(e) {
    if(e.pageX >= 0 && e.pageX <= 10 &&
                e.pageY >= 55+590 && e.pageY <= 55+600) {
        QA.recording = !QA.recording;
        console.log("Recording: "+(QA.recording ? "on" : "off"));
    } else if(QA.recording) {
        console.log('helper.clickOnDiagram('+
                        e.pageX+','+(e.pageY-55)+');');
        QA.lastClickToolbar = false;
    }
});
$('a').on('click', function() {
    if(QA.recording && $(this).attr('id')) {
        var id = $(this).attr('id');
        console.log('helper.clickOnElement(\"'+id+"\");");
        QA.lastClickToolbar = false;
    }
});
$('body').on('change', 'input', function() {
    if(QA.recording) {
        var type = $(this).attr('type');
        if(type === 'checkbox' || type === 'radio')
            console.log('helper.clickOnCheckbox(\"'+
                        $(this).attr('id')+"\");");
        else if(type === 'text' || type === 'number')
            console.log('helper.replaceTextIn(\"'+
                        $(this).attr('id')+"\",\""+
                        $(this).val()+"\");");
    }
});
$('#page1').on('pageshow', function() {
    if(QA.recording && !QA.lastClickToolbar)
        console.log('helper.waitForPageOne();');
});
$('#page2').on('pageshow', function() {
    if(QA.recording && !QA.lastClickToolbar)
        console.log('cw.waitForPageTwo();');
});
And the helper class that it emits commands for looks something like this:
private JavascriptExecutor js;
private WebElement canvas;
private WebElement page1;
private WebElement page2;
private Actions ui;

public TestHelper(WebDriver driver) {
    js = (JavascriptExecutor)driver;
    canvas = driver.findElement(By.id("diagramCanvas"));
    page1 = driver.findElement(By.id("page1"));
    page2 = driver.findElement(By.id("page2"));
    ui = new Actions(driver);
}

public void clickOnDiagram(int x, int y) {
    ui.moveToElement(canvas, x, y).perform();
    clickOnCanvas(x, y);
}
private void clickOnCanvas(int x, int y) {
//    js.executeScript("$('#upperSchematic').click();");
    js.executeScript("var evt = $.Event('click', { pageX: "+
                     x+", pageY: "+(y+55)+" } );\n" +
            "$('#diagramCanvas').trigger(evt);");
}
public void waitForPageOne() {
    waitForPage(page1);
}
public void waitForPageTwo() {
    waitForPage(page2);
}
private void waitForPage(WebElement page) {
    wait.until(ExpectedConditions.visibilityOf(page));
}
public void clickOnCheckbox(String id) {
    // In JQuery Mobile you need to click on the label
    WebElement element = (WebElement)js.executeScript(
                         "return $(\"label[for='"+id+"']\")[0];");
    ui.click(element).perform();
}
public void clickOnElement(String id) {
    WebElement element = page.findElement(By.id(id));
    if(!element.isDisplayed())
        throw new RuntimeException(
            "Cannot click on '#"+id+"' because it is not displayed!");
    if(!element.isEnabled())
        throw new RuntimeException(
            "Cannot click on '#"+id+"' because it is not enabled!");
    ui.click(element).perform();
}
public void replaceTextIn(String id, String text) {
    WebElement element = page.findElement(By.id(id));
    String old = (String) js.executeScript(
                                   "return $('#"+id+"').val();");
    ui.click(element);
    for(int i=0; i<old.length(); i++) {
        ui.sendKeys(Keys.BACK_SPACE);
        ui.sendKeys(Keys.DELETE);
    }
    ui.sendKeys(text);
    ui.sendKeys(Keys.TAB);
    ui.perform();
}
A little copy and paste and I end up with a test method that looks like this:
helper.mainMenu();
helper.clickOnElement("MainMenuNew");
helper.wait(100);
helper.clickOnDiagram(393, 412);
helper.waitForPageXYZ();
helper.replaceTextIn("SomeTextFieldID", "MyNewText");
helper.clickOnElement("SomeEditWidgetID");
confirm(helper.getElementText("SomeSpanID").startsWith("MyNewText"));
...

Running Tests:

I wasn't too interested in writing a dozen tests as command-line executables and running them one after the other. I wanted to run one command and have it run all the tests, counting successes and failures, logging stack traces for any failures, and so on. I really wanted a little GUI to show red and green bars and so on, but it wasn't worth the time to recreate that. I ended up writing a little wrapper to run and capture the results of all my individual tests, but it didn't seem like something I should have to be doing.

The bottom line is that I can issue a command, watch a browser flash through interactions with my app, and get a list of failures and stack traces at the end. When I want to write a new test I include my extra JavaScript file, click the obscure corner of the Canvas to turn on recording, and then interact with my app as usual. At the end, I copy and paste the script from the Web Console into a new source file and add any additional validation I want. Then I can comment out the QA JavaScript and run more tests.

It's what I wanted, but less flashy, and I had to write a lot of setup code to get there.

Here's the code from the master class that runs the tests:
public static void main(String[] args) throws InterruptedException {
    FirefoxProfile profile = new FirefoxProfile();
    profile.setEnableNativeEvents(true);
    DesiredCapabilities capabilities = new DesiredCapabilities();
    capabilities.setCapability("nativeEvents", true);
//    WebDriver driver = new SafariDriver(capabilities);
//    WebDriver driver = new ChromeDriver(capabilities);
    WebDriver driver = new FirefoxDriver(profile);
    driver.manage().window().setSize(new Dimension(1040, 850));
    driver.get("http://localhost:9393/");
    TestHelper helper = new TestHelper(driver);
    helper.waitForDefaultPage();
    List<Throwable> exceptions = new ArrayList<Throwable>();
    // NOT SHOWN: Introspect the test class and run the test methods
    if(exceptions.size() == 0)
        System.out.println("All tests passed!");
    else
        for (Throwable exception : exceptions) {
            exception.printStackTrace();
        }
    driver.quit();
}

Areas for Further Investigation:

The next thing I'd like to try is Geb as a wrapper for WebDriver (Update: see results here.)

It looks like a standard pattern for wrapping all the interesting application actions and state is PageObjects. My gut reaction is that it would have been way too much work to do that for all the "pages" in my application, but it's a much more organized approach and better than just scattering various IDs and coordinates through a bunch of individual tests. I feel like the approach has merit, I just can't put my finger on the line where the provided value overcomes the necessary work. Probably if you're writing all the tests by hand, but I'm not willing to go there anyway.

Tuesday, January 3, 2012

Introduction to Backbone.js with jQuery Mobile

If you are working on a JavaScript heavy application (think jQuery Mobile, etc.), you probably will want to look at some JavaScript libraries to help add structure, consistency and convenience to your applications. One of the JavaScript libraries I've used lately is Backbone.js. To quote Backbone themselves, it provides "models with key-value binding and custom events, collections with a rich API of enumerable functions, views with declarative event handling, and connects it all to your existing application over a RESTful JSON interface.". This is really a great summary of what Backbone can provide to your application.

As an example of how to apply Backbone, I'll create a small jQuery Mobile app and use a Backbone model, collection and view to render the user interface. Through the use of Backbone's event model, the user interface will update as the model changes or new models are created.

I decided to use jQuery Mobile as the UI framework mainly because I am a mobile guy and it will provide the HTML with some reasonable style with no additional effort on my part. I know, lazy...but it works for this blog post :) Continuing on the lazy path, I will refer to jQuery Mobile as JQM from this point forward.

The following assumes you are at least a little familiar with jQuery and JQM. If not, take a look at the JQM website (http://jquerymobile.com/). They have some great documentation and even a template for creating a basic app. All of the code in this post will be available in my github repository (https://github.com/stevenpsmith/Exercise/tree/BackboneIntro).

The application we are going to make will allow a user to log exercise they have done on any given day. Please remember this is a very basic app for the sole purpose of demonstrating Backbone.js. The piece that will be developed for this post will involve pulling exercise data from a server and presenting it in a list view, a very common paradigm for mobile apps. In the end, our app will look like this:

The JSON we will be dealing with will look like this:

I like to organize my application structure a little differently than the JQM introduction shows, mainly for organizational reasons. My application structure looks like this:

All of our work will be done in index.html and app.js. In real applications I usually break down my JS files even further, but it really isn't necessary here. The list page of the application is defined within index.html. It should look like a typical JQM page definition with the only the content portion defined. We will use the Backbone View to create the list view:

If we view this in the browser, we get an empty page with an add button in the header. Since we want to populate our list view with data, we should define a Backbone model and collection in order to manipulate and store the data via JavaScript. A basic model and its collection are easily defined:

The great thing about the collection is it provides a RESTful access back to your server for data retrieval and persistence. To keep this simple, our data will be served from a static JSON file with the structure referenced earlier. Since we have no real server side in this example, we can't really save our changes anywhere but in the client's memory. But the collection will fetch the data from the server and can be used to help render our user interface. The following code will create an instance of the collection in memory, retrieve the data from the server, and parse it into a collection of models.

The exercise.initData() function can be called during initialization of the application (or wherever else might be appropriate) to load the data. Now it is time to render the view. Backbone views depend upon Underscore.js. Underscore is a utility library that includes a lot of great features, one of which is view templates. Backbone can be used with many view template frameworks (e.g. Mustache, Handlbars, etc.), but here we will stick with Underscore templates as they meet our needs. For our exercise activity items, we need a very basic template like this:

Giving the template a unique id will let us access it using typical jQuery selectors. The data driven parts of the template are surrounded by

<%= %>
As you can see in line 2, we are using the id, date, and type attributes of whatever model is supplied to the template to fill in the HTML accordingly. While the "identifier" attribute is not a real HTML attribute and not required by any of the frameworks, it would be used to determine what activity the user selected in the list so the appropriate data could be retrieved and displayed on a subsequent page.

The next step is to define the Backbone View that will render the list view. Backbone views can be defined and set up numerous ways, and the documentation does a great job explaining the options, etc. I like to define my views so that they are as self contained as possible. So, rather than allowing Backbone to create the HTML within a DIV (the default behavior), I define the appropriate properties to create a JQM list view as shown in lines 2-4 below:

From the Backbone site: "The default implementation of render is a no-op. Override this function with your code that renders the view template from model data, and updates this.el with the new HTML. A good convention is to return this at the end of render to enable chained calls". The $(this.el) refers to the containing element, in this case the ul element. There are standard attributes that are accessible from within the Backbone View including collection and model. Lines 17-20 iterate over the collection of activities, fills in the template placeholders with the data from the model, and appends the resulting HTML to the containing element.

If you need to pass arbitrary data to your view, it can be accessed via the options object as shown on line 11 of the code snippet. As mentioned earlier, I like my Backbone Views to be as self contained as possible, so I provide the containing HTML element for this view (as a jQuery object) when constructing the View instance. Line 20 sets the HTML of the containing element to the Backbone View and line 21 tells JQM to style things appropriately.

The next step is to instantiate and render the view with the appropriate collection and HTML container. Since this is the "home" page in our application, this is done after JQM initializes the page.

Now we have a list view that renders a JQM list based on data in a Backbone collection. To take advantage of the real power of Backbone models/collections, we can add listeners for changes to the collection that will update the view based on those changes. The beauty of this is the additional code to accomplish this is minimal. By adding

this.collection.bind('add', this.add, this);
to the initialize method of our Backbone View and providing an add() method.

Here we are just grabbing the list view and appending the new model to the view (and refreshing it so JQM does its magic). By adding a click handler to the "Add" button, we can test that adding a model to the collection updates the view.

There are other methods on which to bind, but this provides a quick example on how Backbone can be used to assist with view rendering and data handling in JavaScript based client applications.

The entire code base is available here. It can be deployed on any web server. For OS X, I use the built in web server, but feel free to use the server of your choice.

Related Posts:

Tuesday, March 9, 2010

Rapid Mobile Application Prototyping with PhoneGap

There is no doubt that carving out a mobile presence is a priority for many companies offering any type of product or service, or for start ups looking to gain rapid exposure. The Apple App Store has over 100,000 apps on it, and the Android platform is gaining ground. Those are just the cool kids – let's not forget the grandfather of mobile technology, Blackberry. Others in the class include Palm, Symbian and of course, Windows.

Multiple Platforms
With so many platforms available there is a risk of losing focus on developing and producing applications as resources are expended on supporting multiple platforms – a necessary evil as each presents different development environments and eccentricities.
This is why PhoneGap really stands out at the moment as four [1] of the aforementioned platforms share a common factor – a WebKit-based browser. This doesn't include BlackBerry's announcement [2] of their new WebKit based browser to come in late 2010, which takes us to five out of the six platforms.

Enter PhoneGap
Where does PhoneGap sit in this picture? Well, it jumps off the canvas and sits in your lap! PhoneGap allows you to develop applications using HTML, CSS and JavaScript and provides a JavaScript API which enables you to interact with a device's native functionality [3]. In addition to this simple application architecture, PhoneGap comes with build scripts/plugins (PhoneGap extension for XCode) which bundle up your pages into a deployable application.
The packaging and deployment to device for iPhone, Android and Palm was trivial; I can only speculate that the remaining platforms are equally easy to deploy. In light of recent Apple 'crackdowns', as of October 7th [4], 2009, it is worth noting that PhoneGap (version 0.8.0 and higher) is Apple approved.

The JavaScript advantage?
Anyone who has used a JavaScript library, Prototype, jQuery, MooTools etc, knows how powerful they can be. Anyone who has not used them can easily pick up JavaScript basics. The guys over at PhoneGap recommend using XUIJS[5] with PhoneGap as it does not contain a lot of cross-browser compatibility overhead and weighs in at < 10kb compared with jQuery at 24kb – although for simple functionality you may want to avoid using a extra library at all.
Due to the fact that we are ultimately aiming to work inside one browser implementation (WebKit), we are not held hostage by the browser compatibility issues that have haunted JavaScript for so many years. That's not to say that fragmentation in the mobile world is not an issue [6].

Now, I am not going to try to argue that these HTML/JavaScript apps offer greater performance or a better user experience over their native counterparts. However, I will say that PhoneGap will allow the rapid development of an application for multiple platforms, using a single code base [7]. This is ideal for prototyping or for apps that will have a limited life span and do not warrant the time and investment of producing native apps.Considering all of this in combination with developments coming down the pipeline, most specifically HTML5 (which includes WebSockets and Client-side database storage), the potential for these types of mobile applications is vast and genuinely exciting.[1] Android, iPhone (safari), Palm (WebOS) Symbian/S60(WRT) http://en.wikipedia.org/wiki/List_of_web_browsers#WebKit-based_browsers

[2]http://devblog.blackberry.com/2010/02/blackberry-webkit-browser-preview-at-mobile-world-congress/

[3]http://nachbaur.com/blog/phonegap-officially-permitted-on-the-app-store

[4]Checkout the road map for specifics: http://phonegap.pbworks.com/Roadmap

[5]http://code.google.com/p/xui-js/

[6]http://techcrunch.com/2010/03/04/mobile-fragmentation-forever/

[7]ok, you are going to have to tweak it for BlackBerry.

Sunday, February 7, 2010

Interview on JQuery

Two weeks ago, Chariot Solutions had a day long event for our consulting team. With the original name of Chariot Day, our team has a series of talks on all different subjects. Some of the talks are on topics which may never see the light of day in the enterprise, some are about more accepted tools. Some of those talks and presentations will be the beginnings for seminars we will host for the public.

I thought I would take advantage of having so many topics to choose from for interviews. I will be posting them as I have the time to edit them.

This first one, with Lyle Anderson, is on JQuery. Lyle gave a two-part presentation which was well attended by our architects. When I see that much interest from the team, I know this is a technology worth watching.




We are still waiting to confirm, but it looks like we will be having a talk at ETE 2010 (www.phillyemergingtech.com) by Yehuda Katz.