Monday, February 4, 2013
A Spring summary: The Spring framework is still relevant
Wednesday, January 30, 2013
HTTPS with Client Certificates on Android
Many Android applications use REST or another HTTP based protocol to communicate with a server.
Working with HTTP and HTTPS on Android is generally fairly straightforward and well documented. Depending on the version of the Android OS, either HTTPClient or HttpURLConnection “just work”. Either one can be used, but the official recommendation is to use HttpURLConnection for Gingerbread or later, and HTTPClient for Froyo and earlier.
Things get interesting, however, once you go past “plain vanilla” HTTPS. On a recent project, we needed to communicate with an HTTPS server that required client certificates, and which used a self-signed server certificate. This presented some interesting challenges. The rest of this post describes how we met those challenges.
If you want to follow along by trying out the code, it’s available on GitHub. You will, of course, have to set up your own HTTPS server and your own server and client certificates. One way that you can do this is by using the Apache mod_ssl module.
Before starting out with using HTTP or HTTPS on Android, make sure that your application has permission to access the network by adding the following to your AndroidManifest.xml:
<uses-permission android:name="android.permission.INTERNET"/>
Challenge #1: Using a Client Certificate
Our client certificate was issued in the PKCS 12 format, as a .p12 file.To give our application access to the certificate, we used the DDMS utility to copy the certificate file to the root directory of the phone’s sdcard. If you are using an emulator, you can do the same thing to copy the certificate to the emulator’s sdcard.
You can also choose to mount the phone as a disk drive when you plug in the USB cable, and copy the file that way.
In our final application, we “imported” the certificate by copying it to application-specific storage, and then deleting the original from the sdcard root directory.
The secret to using our client certificate is setting up a custom KeyStore containing the certificate, and then using it to create a custom SSLContext.
Once we have a reference to a File containing the client certificate and the password for the certificate, we load it into an appropriate KeyStore (see the sample SSLContextFactory.java):
keyStore = KeyStore.getInstance("PKCS12");
fis = new FileInputStream(certificateFile);
keyStore.load(fis, clientCertPassword.toCharArray());
Now that we have the KeyStore containing the client certificate, we can use it to build an SSLContext:
KeyManagerFactory kmf = KeyManagerFactory.getInstance("X509");
kmf.init(keyStore, clientCertPassword.toCharArray());
KeyManager[] keyManagers = kmf.getKeyManagers();
SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(keyManagers, null, null);
The SSLContext can then be used with an HTTPUrlConnection to connect to the server:
String result = null;
HttpURLConnection urlConnection = null;
try {
URL requestedUrl = new URL(url);
urlConnection = (HttpURLConnection) requestedUrl.openConnection();
if(urlConnection instanceof HttpsURLConnection) {
((HttpsURLConnection)urlConnection).setSSLSocketFactory(sslContext.getSocketFactory());
}
urlConnection.setRequestMethod("GET");
urlConnection.setConnectTimeout(1500);
urlConnection.setReadTimeout(1500);
lastResponseCode = urlConnection.getResponseCode();
result = IOUtil.readFully(urlConnection.getInputStream());
lastContentType = urlConnection.getContentType();
} catch(Exception ex) {
result = ex.toString();
} finally {
if(urlConnection != null) {
urlConnection.disconnect();
}
}
Challenge #2: Trusting a Self-Signed Server Certificate
We now have Android client code that can connect to an HTTPS server and present a client certificate.
However, this only works if the server’s certificate is trusted. In practice, this means that the server certificate must be signed by one of the major certificate authorities, such as VeriSign, Thawte, Geotrust, Comodo, etc. CA certificates for these well-known providers are pre-loaded into the phone’s default trust store.
In our case, the certificate was self-signed. This means that the default TrustManager in our SSLContext will not trust the server’s certificate, and the SSL connection will fail.
If you search the web for ways to trust self-signed server certificates, you’ll likely see a lot of really bad advice. Most responses to this sort of question involve setting up a custom TrustManager that simply trusts everything. This approach is both pointless and obviously insecure.
What we really want to do is to set up a custom TrustManager that trusts our self-signed certificate, and provide that TrustManager to our custom SSLContext. To do this, we need a copy of the server’s certificate chain, which will have to include at least the self-signed CA Certificate and the Intermediate Certificate that is signed by the CA. If you simply export the server’s certificate, you will actually get a file with three certificates - the CA certificate, the Intermediate Certificate, and the server’s certificate. This is fine - the extra certificate is not needed, but won’t hurt, either (usually - see Challenge #3 below).
The method for trusting the self-signed server certificate is very similar to the method for providing a client certificate - we load the certificate into a KeyStore, use that KeyStore to produce an array of TrustManagers, and then use those TrustManagers to create the SSLContext.
In our app, we included the server certificate file in the application resources (since it doesn’t change from user to user, and doesn’t change very often), but you could put this in an external file as well.
Here’s the code for loading server certificate chain into a KeyStore that will be used as a “trust store”, based on having the base-64-encoded (.pem) certificate as a String:
byte[] der = loadPemCertificate(new ByteArrayInputStream(certificateString.getBytes()));
ByteArrayInputStream derInputStream = new ByteArrayInputStream(der);
CertificateFactory certificateFactory = CertificateFactory.getInstance("X.509");
X509Certificate cert = (X509Certificate) certificateFactory.generateCertificate(derInputStream);
String alias = cert.getSubjectX500Principal().getName();
KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
trustStore.load(null);
trustStore.setCertificateEntry(alias, cert);
Now that we have the “trustStore” KeyStore with the server’s certificate, we use it to initialize the SSLContext. Adding to the code that we had before, production of the SSLContext now becomes:
KeyManagerFactory kmf = KeyManagerFactory.getInstance("X509");
kmf.init(keyStore, clientCertPassword.toCharArray());
KeyManager[] keyManagers = kmf.getKeyManagers();
TrustManagerFactory tmf = TrustManagerFactory.getInstance("X509");
tmf.init(trustStore);
TrustManager[] trustManagers = tmf.getTrustManagers();
SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(keyManagers, trustManagers, null);
Now, we can securely connect to our server, trust its certificate (but not others), and present our client certificate. If all goes well, you’re done, and can safely stop reading.
Challenge #3: Handling an Improperly Ordered Server Certificate Chain
As it turns out, some Apache mod_ssl installations (and possibly other SSL providers), whether due to a bug or mis-configuration, provide the server certificate chain in the wrong order.
To work properly, the certificates in the server’s certificate chain must start with the “root”, or CA certificate, followed by any intermediate certificates. If the server’s certificate is included, it must come last. Each certificate in the chain (other than the root) must be preceded by the certificate that was used to sign it.
In our case, a mis-configured server was producing a certificate chain that started with the root CA certificate, but which had the server’s certificate next, followed by the intermediate certificate. This caused the SSL handshake to fail.
To work properly, the certificates in the server’s certificate chain must start with the server certificare, followed by any intermediate certificates. If the “root”, or CA certificate, is included, it must come last. Each certificate in the chain (other than the root) must be followed by the certificate that was used to sign it.
In our case, a mis-configured server was producing a certificate chain that started with the server certificate, but which had the root CA certificate next, followed by the intermediate certificate. This caused the SSL handshake to fail.
The most straightforward thing to do in this situation, if you can, is to correct the server’s certificate chain.
If you can’t, then the out-of-order certificate chain must be handled by the client.
In our Android app, we handled the out-of-order certificate chain by implementing a custom X509TrustManager that re-orders the chain.
The code for the custom X509TrustManager is too large to include here (see the full source on GitHub), but once we have it implemented, we use it in the creation of the SSLContext by replacing the ‘trustStores’ array with one containing the custom implementation:
KeyManagerFactory kmf = KeyManagerFactory.getInstance("X509");
kmf.init(keyStore, clientCertPassword.toCharArray());
KeyManager[] keyManagers = kmf.getKeyManagers();
TrustManager[] trustManagers = {new CustomTrustManager(trustStore)};
SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(keyManagers, trustManagers, null);
By loading our own certificates into our keystore and trust store, and re-ordering the server’s certificate chain, we were able to provide an SSLContext that let us connect to an SSL server with client a certificate and a badly-ordered, self-signed server certificate chain.
Wednesday, January 16, 2013
List of technologies we're watching in 2013
From cloud computing to Javascript, from micro services to repeatable environments, from Android and HTML 5 to Big Data and concurrent programming, Chariot's engineers are keeping watch.
The Internet of Things
The “Internet of Things” is all around us.. or, well, we’d like it to be. From “Just in Time” delivery of bus or train schedules as you leave your house, to sensors tracking air quality in the city, from phones to smart clothing, the Internet of Things is about integrating ourselves with this data available all around us. Free of ‘apps’ and ‘services,’ the Internet of Things will make data available ‘just in time’ wherever and whenever you need it. Whether you get the data from WiFi, RFID or QR codes it’s out there, and we think that’s exciting!
Cloud Host Almost Everything!
If you're a savvy IT person, it's may have been a while since you provisioned your own Linux host on a Virtual Private Server or hosting plan. Cloud hosting isn't emerging, it's emerged and taking over. You're likely to use a cloud-based service such as Heroku, Amazon EC2, SliceHost, or even Microsoft's Windows Azure. There are private cloud partitioning services to protect your data from theft in co-location. Companies such as CloudBees have been shifting so many services to the cloud that now they even provide development and QA platforms that provide your SVN or Git repositories, run Hudson or Jenkins, and provide Continuous Integration services.
Read the rest of this article, subscribe to our TechCast podcast or view all of our Philly Emerging Tech 2012 sessions at our emerging technology resources site, emergingtech.chariotsolutions.com.
Thursday, January 10, 2013
Science of Big Data - sessions coming online
Currently we have two - Basho's Tom Santero discusses "Eventual Consistency" and outlines the challenge of keeping data highly available and consistent enough without impacting performance.
The second screencast is the combination of Chariot's Andrew Oswald and Comcast's Josh Angotti's talks. Andy reviews the work that our team did for Comcast using Hadoop in analyzing set-top box users' click selections, and reducing the timeframe of finding answers from days to hours or minutes. Josh Angotti tells the story from Comcast's perspective.
More screencasts are expected soon - we had some video syncing difficulties and audio challenges so it'll take a bit to get them all online. Check the page soon for talks by Cliff Moon of Boundary, Stuart Sierra of Relevance, and Mark J. Headd, the Philadelphia Chief Data Officer.
Wednesday, December 5, 2012
The "Pivotal Initiative" - VMware and EMC spin off a new organization to focus on big data in the coud
On December 12th, EMC and VMware announced a spin-off that will merge units from both VMware and their parent company into a single, focused organization named the Pivotal Initiative.
Thursday, November 15, 2012
Installing PhoneGap Plugins with pluginstall
Phonegap applications can be extended with plugins to add additional functionality. Historically, installing plugins meant copying a bunch of files into the correct places in your project.
Andrew Lunny wrote the Cordova Plugin Specification and pluginstall. Pluginstall is a tool that automates the installation of plugins which conform to the plugin specification.
In addition to demonstrating pluginstall, we’ll also use the PhoneGap create script to make a new project. The cordova build scripts will be used for compiling and installing the binary. Using these tools it’s possible to develop PhoneGap apps with a text editor, instead of an IDE.
Prerequisites
I’ll assume that you have Phonegap 2.2.0 and the Android SDK installed.
Pluginstall requires node.js and is installed through npm. If you’re on a mac and use homebrew, install node and npm with brew install node.
Setup
Install pluginstall
$ npm install -g pluginstall
Create a new Android project using the PhoneGap create script. Create takes the new project path, package name and activity name as arguments.
$ cd /usr/local/phonegap-2.2.0/lib/android
$ ./bin/create ~/foo com.example.foo Foo
The create script built a new PhoneGap project in directory ~/foo. Inside the project there is a cordova directory with some command line tools.
Let’s ensure the generated project works before we modify anything. Start the emulator.
$ cd ~/foo
$ corodova/emulate
Patiently wait for the emulator to start. Unlock the screen and keep the emulator running. Compile, build, install, and launch your app.
$ cordova/BOOM
The application should launch and say “Device is ready”.
Install the plugin
Get a local copy of the Child Browser Plugin.
$ cd /tmp
$ git clone git://github.com/alunny/ChildBrowser.git
Install the plugin using pluginstall. Pluginstall takes the platform, project directory and plugin directory as arguments.
$ pluginstall android ~/foo /tmp/ChildBrowser
The Child Browser plugin is now installed. Pluginstall used to the information in the plugin.xml manifest to update the project. It added an Activity to the Android manifest, listed the plugin in config.xml, added java source code, and added javascript code and images to our project.
Write some code to use the plugin
Edit ~/foo/assets/www/index.html
Include the javascript after cordova.js
<script type="text/javascript" src="cordova-2.2.0.js"></script> <script type="text/javascript" src="childbrowser.js"></script> <script type="text/javascript" src="js/index.js"></script>
Edit ~/foo/assets/www/js/index.js
In onDeviceReady, add a call that will open a web page in the Child Browser after 3 seconds
onDeviceReady: function() {
app.receivedEvent('deviceready');
setTimeout( function() {
window.plugins.childBrowser.showWebPage(
'http://phonegap.com', { showLocationBar: true }
);
}, 3000 );
},
Compile, build, install, and launch your app into the running emulator.
$ cordova/BOOM
The sample application should run and open http://phonegap.com in the Child Browser Plugin.
Monday, October 22, 2012
Top Questions To Ask When Choosing an Android Development Firm or Developer
You are ultimately responsible for the application - even if you go the outsourcing route. Things happen, disagreements happen, and turn over happens. The likelihood is high that you will need to maintain the application at some point in its lifecycle. The better the code, the easier the maintenance will be.
There is a wide range of abilities, experience, and knowledge out there. These questions are designed to help you make the best choice about how to invest your Android budget. You need to ensure that anybody you onboard to build the application will advance your organization's goals and not cause unnecessary rework.
Interview Questions
- What applications have you published in the market? Do you have references we can contact? Describe application features and your role in each application. Do they speak positively of their clients and applications? What do their references have to say? How high are the app ratings? Take note of positive and negative comments. If possible, download and try the application(s). This will give you some insight into the quality of the implementation and design.
- How do fragments fit into your development process? Any firm or developer you hire should be knowledgeable of fragments and should be using them. Otherwise, they are writing obsolete code. This technology is nearing two years old and is the way to write Android apps going forward. Frankly, if they haven't taken the time to learn them by now, how can you trust that they will help you innovate in your application?
- What strategies do you use to ensure the user interface will scale to the multitude of Android devices? There are many strategies and best practices for making interfaces that scale. Some of the things you’ll want to hear from the developer or firm are:
- Using fragments
- Using dimensions, styles, and qualifiers
- Using 9-patches
- Using images in different densities
- Wrapping elements and using relative layouts
- Not using fixed pixels
- How do you minimize battery life drain and bandwidth usage? If they don't have ideas around this, beware. This is one of user’s main complaints! There is typically a cap on data plans, so users want to limit bandwidth so they don’t get hit with overages. Battery is also a major concern for users. A major factor in battery life is the amount of data downloaded and processed on the device.
- Do you unit test your applications? Do you write integration or other automated tests? Test coverage is an important judge of quality for the end product. Although it seems like it's just more code to write, the investment up front will reduce bugs and rework. It may or may not speed time to market, but it will definitely increase the quality of your application.
- How often will we discuss and decide priorities? Priorities change - often quickly and without warning. You will encounter challenges even with the best thought out designs. Business requirements will change and it's essential to keep in close contact with your developers to make sure critical features are implemented according to the latest specifications.
- How will I access the code base? It's always a good idea to have access to the code. A good practice is to create an organization on github and invite the developer(s) to join your team. They create a repository for your application and save any changes in the cloud. This gives you access to the most updated code as well as the commit history. You can browse code commits to see what they are working on and get an idea of where the project stands.
- Will I get regular builds of the application? It's also possible to get frequent builds of the app - either manually or as part of an automatic process. Giving stakeholders access to regular builds is a good idea so that the business can ensure that the developers are creating the features as expected.
Do they use similar platform features to what you plan to use? For example, have they only written map applications when you want to create a music streaming application? Keep in mind there is a plethora of platform features, so unfamiliarity with a particular feature isn't a deal breaker. Great engineers can pick up any new feature very quickly. If something is business critical and you are uncomfortable with them learning this for your application, you may want to look for someone with experience with these features.
If they don't use them, you will need to completely renovate the application when you expand to the Android tablet market. On the other hand, if they write it with fragments from the start it will be much easier to transition to tablets.
There is a large base of Android developers who remain resistant to learning this new and more flexible way to build applications. I've gone on a conference crusade this year to help businesses and developers see the benefits of switching to fragments before it's too late.
Some techniques are using services to complete background work, using alarms to kick of background tasks, detecting when on WiFi and downloading large amounts of data, designing APIs that give incremental data, or detecting when plugged into the wall to do a large amount of processing.
Developers that don't use unit testing spend a lot of time building to devices or emulators. This uses significant amounts of time that could be use for writing tests to verify behavior. Plus they won't be able to find all the bugs by poking around in the application.
Although many Android developers eschew using testing frameworks like Robolectric, setting up new projects with Robolectric, IntelliJ, and maven (or maven + Eclipse) avoids the headaches that come from integrating later on in the project. There are many things you could test, but there are limits and tradeoffs. Check out my article about best practices for testing with Robolectric to help you structure your testing strategy.
Although unit testing in great, there are other types of testing. The more types of tests that are written, the bigger safety net you have when the architecture needs to change. It will help ensure that large changes don't break existing functionality and find subtle bugs before they get into the wild.
Although we always embark on new contracts with good will on both sides, disputes with an external firm or developer can happen. Make sure that you have access to and retain the rights of your application's code. You can't develop or maintain what you don't have!