Software Quality and Software Costs by Capers Jones

"An occupation where failures and disasters are the top cost drivers is not a true engineering discipline.  To become a true engineering discipline, software engineering needs better quality control, better quality measures, and better economic analysis than current norms." - Capers Jones in ASQ/SQP June 2011.

This month's (June/2011) issue of the American Society for Quality's Software Quality Journal, has a 'Must Read' article, "Software Quality and Software Costs" by Capers Jones where he explores the application of two metrics frameworks - software cost of quality and software defect containment. Both to model and manage the cost and quality consequences of poor requirements and spending time on debugging, instead of not putting the bugs in the first place.


Some ASQ/SQP articles are member only, and a few are available with a free registration.  Some articles such as the one we are discussing here are made available as PDF's under 'Open Access'.  The system may ask you to register with a name and email address, then you may be directed to a page saying you must purchase the article.  However if you click the link again, the article should open due to it being an Open Access article.


Jone's metrics are based on the International Function Point Users Group (IFPUG).  Using Function Points allows for measuring defects across different languages, and in requirement documents where measurements of "Lines of Code" are useless.


To summarize: The Bad, and alas the typical development process today:

• Either inadequate estimation or the arbitrary rejection of accurate estimates by clients who then imposed unachievable schedules; [Clients refusal to listen to accurate estimates on development time, usually being driven by an unmovable trade show deadline].
• Inadequate status tracking by management that concealed problems until too late to recover.
• Poor change control combined with creeping requirements in excess of 1 percent growth per calendar month; [The Creeping Feature Creature is a powerful task master, as late addition requirements contain more bugs].
• Poor quality control.
• Testing alone is not very efficient in finding bugs.  Less that 35% effective.

The good, that is proper project planing and management (Always make a case from the point of the bottom line to get the attention of the Bean Counters):

• The high-quality project schedules will be shorter by about 15 percent.
• Software Quality and Software Costs in defect removal efficiency will cost about 20 percent less to develop than identical projects with poor quality.
• Cumulative Total Cost of Ownership of high-quality applications from the start of the first release through five years of maintenance and enhancement will be about 30 percent lower than identical projects with poor quality.
• Annual maintenance costs will be lower by about 40 percent.
• For large applications, high quality levels will minimize the chances of failure.
• High-quality applications tend to have quicker testing schedules and hence quicker overall schedules.
• The economic value of excellent quality is directly proportional to application size. The larger the software application, the more valuable quality becomes.

"No true engineering discipline should have defect repairs and canceled projects as the two top cost drivers.  For software engineering to become a true engineering discipline, quality control will have to be much better than it is in 2011."