Reasoning Study Concludes Tomcat Quality on Par with Commercial Code

8/4/2003 - Reasoning, the leading provider of automated software inspection (ASI) services, announced the results of a study inspecting mature Tomcat version 4.1.24 code. Based on the findings, the Tomcat program showed a defect density similar to proprietary code at a similar point in the development lifecycle. Tomcat is a Java-based application server and part of the Apache Jakarta project. Like Apache, Tomcat is developed in an open and participatory environment.

A key quality measurement indicator is defect density, which is defined as the number of defects found per thousand lines of source code. Reasoning found 17 software defects in 70,988 lines of Tomcat source code. The defect density of the Tomcat code inspected was .24 per thousand lines of source code. Reasoning’s inspection study shows that a majority of the defects found were null pointer dereference (NPD) errors, which are well-known, crash-causing defects. The proportion of these errors was very similar to what Reasoning has seen in many C/C++ applications.

“This finding is important because some believe that Java automatically provides protection against these defects and, therefore, do not expect them to occur,” explained Jeff Klagenberg, Reasoning director of product management. “In actuality, Java can only identify these conditions — not provide a remedy. If the conditions go unresolved, they can lead to reliability or other quality issues.”

Dick Heiman, Research Director of Application Development & Deployment, at IDC Research stated, “Although Java provides better protection against some categories of coding errors than other languages, it is not a cure-all. Defects can show up in any language and independent of the language one is using, code inspections are a very valuable process for detecting errors early in the application design cycle.”

“Ultimately, this study underscores why it is important for companies to adopt regular software inspection in order to avoid serious crash-causing defects. We’re pleased that we are able to share these findings with our customers and the market so they can be more sensitive to potential defect vulnerabilities,” said Bill Payne, President & CEO of Reasoning.

Reasoning’s announcement is the third published comparison of Open Source software to equivalent commercial software applications. The defect report of Reasoning’s Tomcat inspection, as well as the other Open Source inspection results, can be obtained free-of-charge by visiting

Java and Automated Software Inspection
While Java is an impressive language and has introduced significant advances in its ability to build cleaner quality code, Reasoning’s studies indicate that its defect densities are similar to other coding languages. By applying automated software inspection, one can quickly locate and remedy crash-causing defects.

ASI does not totally replace testing; however, it can significantly reduce the time and expense invested in testing and improving overall quality. It is also a cost-effective practice that enhances traditional QA efforts, expedites time-to-market, eliminates defects and increases reliability. In a research study conducted by the Standish Group, most QA organizations are only 30-40 percent effective at identifying software defects. Studies have found by applying ASI early and at critical intervals in the development process saves development time, resources, and significantly reduces costs.

About Reasoning
Reasoning Inc. is the leading provider of automated software inspection services that help development organizations reduce the time and cost involved in finding software defects. The company's business is focused on organizations that develop Java, C, and C++ applications. Reasoning is headquartered in Mountain View, CA. Contact Reasoning at 650.316.4400 or at

Reasoning and the Reasoning logo are trademarks and/or service marks of Reasoning, Inc.

Previous Page | News by Category | News Search

If you found this page useful, bookmark and share it on: