While Ford announced that “Quality is Job One,” it is Toyota and Honda that continue to build the most reliable cars year after year. Many, if not all, software companies want to build a quality product, few actually do build products that meet their own internal management quality expectation and external customer’s satisfaction. The lack of quality is a symptom of many issues. What can be done to improve quality?
At ReplayTV, I was responsible for getting the world’s first digital video recorder (DVR) accepted for Panasonic branding, the premier brand name owned by Matsushita Electronics Inc. (MEI), which also owns JVC and Magnavox. I worked closely with the QA team from the manufacturing division that pumped out 3 million VCRs a year. In my visits to Japan, I experienced first hand how they lived and breathed quality day in and day out. Panasonic employed several hundred engineers to build the automated VCR manufacturing line to exact specifications, and there were only a handful of people to watch over the production line in operation. With respect to ReplayTV, they put the hardware through torture tests (shock, extreme heat over extended time) and the software through thousands of test cases. The end result: my original Panasonic branded ReplayTV is still working perfectly after almost a decade.
Quality is no accident. It requires a lot of work from the conception phase, through design and implementation, and continues through the maintenance phase. In the case of software, maintenance was, is, and will always be the longest portion of the software life cycle, unless the software is quickly withdrawn from the market. The approach to solving the quality problem must be approached from all directions. Here’s a quick run down of what I recommend as the key ingredients for a solution to the software quality challenge:
- A requirement review process that includes customers or those who are as closed to customers as possible. Do this early and often as requirements change. Many of these requirements would be stated as “bugs” and viewed as quality issues from the customers’ view. As discussed in my previous blog, requirements management is critical to a project.
- Employ test driven development and test automation as much as it is practical to do so. Try out different approaches and measure the ROI. It’s a sliding scale and not all or nothing.
- A peer review process for design review, code review, and documentation review. Why peer? While it’s good to have a key leader to facilitate and spot check, it’s too difficult for anyone person to have the time or the detail knowledge of the entire product. A small group of senior engineers and technical leads can be assigned for specific areas of expertise. Non-engineers can also review design docs and user docs.
- A basic set of metrics to measure quality. Some of the obvious ones include the number of bugs of a given severity (easy to collect), and time to fix a bug (hard to collect). Whatever the metrics are, it’s important to agree up front what’s expected, and also adjust accordingly when the actual results come in.
- Formal classes and informal on-the-job training for engineering to ensure that the skills and knowledge are shared within the organization. One common reason for poor quality is that there isn’t enough people knowing about the product to be able to fix the problems at the root.
- Formal classes and informal on-the-job training for technical support to be able to troubleshoot the product.
- Plan a tour-of-duty for developers and QA engineers to work as technical support. While it’s often impractical to dedicate the best developers on customer support, periodic direct exposure to customers is highly valuable.
The list above is by no means comprehensive, but it points out that quality is a multi-prong challenge, and requires a systematic solution. Most software products don’t have to be built like a Sherman tank (or a ReplayTV 🙂 ) and they certainly aren’t.
Decide on how much quality is enough, plan for it, and be willing to pay for it.
Because quality costs.