5 Tips for Big Software Projects

Software development isn't easy.  And the bigger the software gets, the harder it is to build right from the ground up.  A string of high-profile failures has given us a timely reminder of this.  But it's not hopeless -- here are five things to consider to bump the odds in your favor:

1. Great software is never built on a poor architecture

The only way to build a large project is to divide and conquer.  But ten teams working in ten different ways will just produce ten times the mess.  The bigger a project gets, the more important it is to build on the best architecture and tools, modularizing where it makes sense without simply introducing more discrepancies and more complexity.  And the more people working on top of an architecture, the more important that it doesn't introduce obstacles that every developer may find different ways to avoid.

2. Generated code is terrible

Many situations land programmers with large amounts of tedious boilerplate code.  The gut reaction is to code-generate your way past it.  But if you think a junior programmer writes bad code, you should see a programmer try to program a computer to program.  It's just as convoluted as it sounds.  Worse, it has to be done over and over again, and it's one way to guarantee that some large chunk of code is rarely up to date.  Whether it's persistence or XML or other boilerplate code, there's a different set of tools that will eliminate the tedium with better code instead of worse code.

3. Best practices aren't always best

If you want to be really cruel, you can write terrible code that adheres to multiple best practices, and just watch an architect splutter.  The problem is that many individual best practices don't make sense when used outside of their associated tools and frameworks, or when used in combination, or when otherwise taken out of context.  Part of what makes experienced programmers better is that they can review all the tools and practices at their disposal, and use their judgement to select the right combination.

4. Software without automated tests is just a pile of code

A large project is built, never adequately tested, and rushed into production.  It doesn't work well.  Why is this a surprise?  But testing is also tedious, and requires a whole additional staff.  That is, if you don't automate it from the beginning.  Your computer is perfectly capable of driving an application with simulated load, an actual Web browser, or real-world integration requests.  There's no reason not to build a comprehensive automated test suite to ensure the software operates as intended.  And the more people who have their hands in the code, the more important it gets.

5. Integration begins with testing

Projects with many integration points tend to have a large number of interested parties, many of whom may be developing their parts of the system in parallel.  Then everybody finishes, and guess what?  It doesn't all work together.  But there's an easy way to avoid that.  If everybody starts by providing simulated requests and replies then testing can begin immediately, and each group can phase in real implementations over time.  Each system gets immediate feedback as to whether anything's breaking, and there's much less debugging required to fix a small change from a working state.

Top Questions To Ask When Choosing an Android Development Firm or Developer

Now that we understand the implications and challenges of embarking on an Android project, let's discuss how to find the right firm or developer for the job!

You are ultimately responsible for the application - even if you go the outsourcing route. Things happen, disagreements happen, and turn over happens. The likelihood is high that you will need to maintain the application at some point in its lifecycle. The better the code, the easier the maintenance will be.

There is a wide range of abilities, experience, and knowledge out there. These questions are designed to help you make the best choice about how to invest your Android budget. You need to ensure that anybody you onboard to build the application will advance your organization's goals and not cause unnecessary rework.

Interview Questions

1. What applications have you published in the market? Do you have references we can contact? Describe application features and your role in each application.
Do they speak positively of their clients and applications? What do their references have to say? How high are the app ratings? Take note of positive and negative comments. If possible, download and try the application(s). This will give you some insight into the quality of the implementation and design.

Do they use similar platform features to what you plan to use? For example, have they only written map applications when you want to create a music streaming application? Keep in mind there is a plethora of platform features, so unfamiliarity with a particular feature isn't a deal breaker. Great engineers can pick up any new feature very quickly. If something is business critical and you are uncomfortable with them learning this for your application, you may want to look for someone with experience with these features.


2. How do fragments fit into your development process?
Any firm or developer you hire should be knowledgeable of fragments and should be using them. Otherwise, they are writing obsolete code. This technology is nearing two years old and is the way to write Android apps going forward. Frankly, if they haven't taken the time to learn them by now, how can you trust that they will help you innovate in your application?

If they don't use them, you will need to completely renovate the application when you expand to the Android tablet market. On the other hand, if they write it with fragments from the start it will be much easier to transition to tablets.

There is a large base of Android developers who remain resistant to learning this new and more flexible way to build applications. I've gone on a conference crusade this year to help businesses and developers see the benefits of switching to fragments before it's too late.


3. What strategies do you use to ensure the user interface will scale to the multitude of Android devices?
There are many strategies and best practices for making interfaces that scale. Some of the things you’ll want to hear from the developer or firm are:
• Using fragments
• Using dimensions, styles, and qualifiers
• Using 9-patches
• Using images in different densities
• Wrapping elements and using relative layouts
• Not using fixed pixels


4. How do you minimize battery life drain and bandwidth usage?
If they don't have ideas around this, beware. This is one of user’s main complaints! There is typically a cap on data plans, so users want to limit bandwidth so they don’t get hit with overages. Battery is also a major concern for users. A major factor in battery life is the amount of data downloaded and processed on the device.

Some techniques are using services to complete background work, using alarms to kick of background tasks, detecting when on WiFi and downloading large amounts of data, designing APIs that give incremental data, or detecting when plugged into the wall to do a large amount of processing.


5. Do you unit test your applications? Do you write integration or other automated tests?
Test coverage is an important judge of quality for the end product. Although it seems like it's just more code to write, the investment up front will reduce bugs and rework. It may or may not speed time to market, but it will definitely increase the quality of your application.

Developers that don't use unit testing spend a lot of time building to devices or emulators. This uses significant amounts of time that could be use for writing tests to verify behavior. Plus they won't be able to find all the bugs by poking around in the application.

Although many Android developers eschew using testing frameworks like Robolectric, setting up new projects with Robolectric, IntelliJ, and maven (or maven + Eclipse) avoids the headaches that come from integrating later on in the project. There are many things you could test, but there are limits and tradeoffs. Check out my article about best practices for testing with Robolectric to help you structure your testing strategy.

Although unit testing in great, there are other types of testing. The more types of tests that are written, the bigger safety net you have when the architecture needs to change. It will help ensure that large changes don't break existing functionality and find subtle bugs before they get into the wild.


6. How often will we discuss and decide priorities?
Priorities change - often quickly and without warning. You will encounter challenges even with the best thought out designs. Business requirements will change and it's essential to keep in close contact with your developers to make sure critical features are implemented according to the latest specifications.


7. How will I access the code base?
It's always a good idea to have access to the code. A good practice is to create an organization on github and invite the developer(s) to join your team. They create a repository for your application and save any changes in the cloud. This gives you access to the most updated code as well as the commit history. You can browse code commits to see what they are working on and get an idea of where the project stands.

Although we always embark on new contracts with good will on both sides, disputes with an external firm or developer can happen. Make sure that you have access to and retain the rights of your application's code. You can't develop or maintain what you don't have!


8. Will I get regular builds of the application?
It's also possible to get frequent builds of the app - either manually or as part of an automatic process. Giving stakeholders access to regular builds is a good idea so that the business can ensure that the developers are creating the features as expected.

What has your experience been with hiring Android developers or firms? How did you determine their skill level? We'd love to hear about it in the comments.

Web Application Security

The recent news that the popular dating site Plenty Of Fish was hacked and that passwords and other user information was stolen truly disheartened me.

It was just the latest in a seemingly endless list of such hacks over the years, recently including Gawker Media (Lifehacker, Gizmodo), McDonald's, Walgreen's and Pizza Hut.

Apparently, Little Bobby Tables is alive and well.

What is disheartening to me is not so much the security breaches themselves. No site is completely secure. The operators of these and other sites may have been able to do more than they did to prevent the breaches, or maybe they had really good (but not good enough) security. But security breaches will happen. I think that every site has to assume that it is possible that their site will be hacked and their database stolen.

I'm not suggesting that they just accept the inevitable, and give up. I'm suggesting that everyone that builds a website that collects information from their users (that's every site that has user accounts) should care enough to do their best to ensure that when the site is compromised, the damage is as minimal as possible. That's why the Plenty Of Fish hack was so disturbing. Passwords were apparently stored in the database in clear-text. Once the database was stolen, passwords were readily available, with no further work on the hacker's part.

I had thought that clear-text passwords were such a well-known evil that no-one was doing it any more, but apparently I was mistaken. Web developers who don't know any better apparently still store plain-text passwords so that they can implement the even less-secure feature of emailing it to you if you forget it (or even sometimes even if you don't).

So, what's the right thing to do? Obviously, the most important prerequisites are being aware of the problem, and caring enough about your users to do something about it. If you've read this far, I'll assume that you have those two covered.

The next thing is to not store passwords. Let me repeat that - do not store your user's passwords. By that, I do not mean that you should encrypt the passwords before you store them, though that would be a huge improvement over plaintext. The problem with storing encrypted passwords is twofold. The first, as previously mentioned, is that it may tempt you to decrypt it for some reason (like sending it to the user in an email). The second problem is that unless you are really careful with your encryption, the data thief may very well have the resources to decrypt the passwords.

Instead of storing a user's password, store a cryptographic hash of their password. A cryptographic hash is also known as a one-way hash, and cannot be "decrypted". For authentication, take the user-entered password, run it through the same hash function, and then compare the result to the hash stored in the database. Provide users with the means to securely re-set their password (choose a new one) in the event that they forget it, rather than attempting to send them the old one.

There are still some issues to be aware of when using cryptographic hashes. If there are collisions (two different passwords hash to the same value), and the thief possesses one of the clear-text passwords by some other means, then they can substitute the known password for the one in the other account. Of course, there's always the possibility that two or more users will pick the same password. For example, in the Gawker Media case, the password data was published on the web, and it was revealed that thousands of users had chosen the same few passwords, with '123456' and 'password' leading the list. If you use a simple hash function, then it seems likely that you will have lots of collisions. This makes several attack vectors, like a dictionary attack, or other brute-force attack much more likely to be successful.

The answer to collisions is random salt. By appending random bits to the password before hashing it, you can make the hashes much more likely to be unique. A common method is to generate a new, random salt value for each user, and to store the salt in an additional column in the password table. Making the salt available to a hacker does not make the hashes more susceptible to a brute-force attack if no two of the salt values are the same. One thing to be careful of is using the username for salt - unless you intend to never let a user change their username.

All hash functions are not created equal. As time goes on, and more and more computing power becomes available at relatively low cost, once-unbreakable hash functions become breakable. For instance, SHA-1, apparently still the most widely used cryptographic hash function, was broken in 2005, and the National Institute of Standards and Technology has recommended that federal agencies stop using it. SHA-2, the recommended replacement for SHA-1, while not yet broken, has already been recognized by NIST as potentially insecure, and there is already an effort under way to develop SHA-3.

One-way, randomly-salted SHA-2 (or eventually SHA-3) hashes should solve your password security issues, but what about other user data? What about email addresses, credit card numbers, and the like? I'd like to suggest that the first line of defense is collecting as little sensitive information as possible. Storing credit card information (where legal) is generally unnecessary and provides the potential attacker with substantial incentive. In short, unless you are a credit card company, you probably shouldn't store credit card information.

Other user information should be treated just as carefully - stored only if absolutely necessary, and stored with encryption as secure as possible if it must be stored. Assume from the start that your database will be compromised, and make it as difficult as possible for the data to be used.

There are many other security issues to be concerned about, and I encourage you to research them, but if you care about your users and your reputation, then the least you can do is stop storing sensitive user information in the clear.

David Black at Philly ETE

David Black was kind enough to give two talks at the conference this year.

The first is entitled, Opinionated is relative: choice and modularity in Rails applications.


The second, Barbara, Demeter, and Don: notes on some CS precepts from a non-scientist programmer, was rated as one of the best talks by one of Chariot's consultants. Very high marks indeed.



I hope you are finding these recaps valuable.