The standalone metrics of past business generations are not relevant to most results oriented companies these days. End users do not concern themselves with scripts executed, errors discovered or total code lines written. These traditional metrics may mean something to the dev team, but it has no direct effect on customer facing quality.
In an age where continuous integration is becoming par for the course, upgrading the sophistication of your software metrics is just as important as changing the process itself. As a matter of fact, changing metrics that you prioritize is a huge part of a process upgrade. Here are three of the most relevant software metrics to consider moving forward into 2017.
Number of Commitments to User Stories
Has the company really worked out what kind of customer its product attracts? Does the company have a buyer profile for each of its main constituent types? Matching different user stories to buyer profiles is a good way to focus the agile development process in a specific direction. Even if product iterations theoretically never end, they must still be directed in a productive fashion.
If you have committed yourself to too many user stories, it is likely that you are also committing yourself to a bloated development process that deserves a more concise vision before you commit money and manpower to a Build. Although virtualization and multivariate testing can give you many concurrent build lines, why devote more resources than is necessary towards building out approximations of a usable product? Your customers will only send you back to the drawing board, complaining of lower usability. Take the time to match your user stories to your buyer profiles from the beginning of the process and reduce the number of stories that you commit to as much as possible.
End User Delivery
How many of the stories that earned your commitment were delivered at standard to the buyer profiles they matched to? Is your quality standard specific and driven by budget metrics that will keep the company profitable and set-aside enough resources to continue development at the same pace moving forward? This is the development metric that your end-user most closely associates your brand with, and it is also the metric that will drive continuous improvement efforts.
One of the cultural shifts that a newly agile company will need to make is learning to snapshot your progress while aiming at a moving target. The entire idea of continuous integration is built upon the ideal that the second you deliver, improvements can immediately be made. The delivery point is more like a checkmark in a never-ending race.
For the first few days (or maybe shorter), the end user compares the quality of your feature delivery to the last iteration of your product. However, this quality standard will soon shift forward to expectation for the next iteration, using your current deliverable as the new standard. Creating an ongoing metric for a feature quality standard as a satisfaction percentage or something similar will give you the ability to keep up with the moving target.
Time to Market
Just as your end users will continuously compare feature sets from your last iteration to this one, they will also become gradually more impatient with future improvements. However, as your product moves forward, the depth of resources and manpower that is required to implement changes should become smaller. In short, continuous improvement should move your product closer to perfection. Although you may never achieve perfection, the distance should close at every checkpoint.
The metric most closely associated with this general end-user story is time to market. How much time did your company take to fulfill a product delivery from concept to end of production? Was your delivery time in keeping with distribution deadlines and market expectations? Unless your end-users are told differently, smaller iterations should take less time. In many cases, companies will forget to forewarn their customer base before build overhauls, leading to overbearing expectations. Having the time-to-market metric at the forefront also serves as a great reminder to market big changes beforehand, because larger overhauls tend to cause weird spikes in the metric. (Hopefully you have created a culture in which you actively investigate weird spikes in your metric graphs or spreadsheets!)
Although you may need to tweak these metrics depending on your audience or industry, they will work generally across all types of companies. As you learn to implement these as your overarching benchmarks, you should be able to identify more process specific, smaller metrics in keeping with these themes.
Keep in mind that the end-user should always be at the center of your software metrics. The final question that you should always ask yourself is if your end-user is getting what they asked for. If they are, then you know that your metrics are helping your company move in the direction that people are expecting.
