Effective Software Testing: 50 Ways to Improve Your Software Testing

• Authors: Elfriede Dustin
• Venue: Effective Software Testing: 50 Ways to Improve Your Software Testing
• Year: 2002

From the Book:In most organizations, the testing program functions as the final "quality gate" for an application, allowing or denying the move from the comfort of the software engineering environment into the real world. With this role comes a large responsibility: the success of an application, and possibly the organization, can rest on the quality of the software product. There are a multitude of small tasks that must be performed and managed by the testing team. So many, in fact, that it is tempting to focus purely on the mechanics of testing a software application, and pay little attention to the surrounding tasks that are required of a testing program. Issues such as the acquisition of proper test data, testability of the application's requirements and architecture, appropriate test procedure standards and documentation, and hardware and facilities, are often addressed very late, if at all, in the lifecycle. For projects of any significant size, test scripts and tools alone will not suffice, a fact that most experienced software testers will attest to.The knowledge of what constitutes a successful, end-to-end testing effort is typically gained through experience. The realization that the testing program could have been much more effective, if certain tasks had been performed earlier in the project lifecycle, is a valuable lesson. Of course, at that point it's usually too late to benefit from the experience.Effective Software Testing provides experienced-based practices and key concepts that can be used by an organization to implement a successful and efficient testing program. The goal is to provide a distilled collection of techniques and discussions that canbe directly applied by software personnel to improve their products, and to avoid making costly mistakes and oversights. This book details 50 specific software testing points, which are contained in ten parts that roughly follow the software lifecycle. This structure in itself is a key concept in software testing: the testing effort must be integrated into the software development process as a whole to be most effective. Isolating the testing effort into one box in a "workflow" is a common mistake that must be avoided.The material in the book varies from process and management related topics, such as managing changing requirements and the makeup of the testing team, to technical aspects such as ways to improve the testability of the system and the integration of unit testing into the development process. Although some pseudo-code is given where necessary, the content is not tied to any particular technology or application platform.It is important to note that there are factors outside of the scope of the testing program that will heavily factor into the success or failure of the project. Although a complete software development process and testing program will ensure a successful engineering effort, any project must also deal with issues relating to the business case, budgets, schedules, and the culture of the organization. In some cases, these issues will be at odds with the needs of an effective engineering environment. The recommendations in this book assume that the organization is capable of adapting, and providing the support of the testing program necessary for its success. OrganizationThis book is organized into 50 separate chapters that span 10 important coverage areas. The selected best practices are broken down in a sequence that parallels the phases of the system development life cycle. The reader can approach the material sequentially, meaning chapter-by-chapter and part-by-part, or simply refer to specific chapters when necessary to gain information and understanding of a related problem. For the most part, each chapter stands on its own, although there are references to other chapters, and other books, where necessary to provide the reader with additional information.Part I describes requirement phase considerations for the testing effort. It is important in the requirements phase for all stakeholders, including the testing team, to be involved and informed of all requirements and changes. In addition, basing test cases on requirements is an important concept for any large project. The importance of the testing team's involvement during this phase cannot be understated; it is in this phase that a thorough understanding of the system and its requirements can be obtained. Part II covers test-planning activities, including the strategy for creating test cases, the goal of the testing effort, and considerations related to data, environments, and the software itself. Planning must take place as early as possible in the software life cycle, as lead times must be considered for implementing the test program successfully. Early planning allows for testing schedules and budgets to be estimated, approved, and then incorporated into the overall software development plan. Part III focuses on the makeup of the testing team. A successful testing team will have a mixture of technical and domain knowledge, as well as a structured and concise division of roles and responsibilities. Part IV discusses the architectural considerations for the system under test. Often overlooked, it is necessary to ensure that the system itself is testable, and provides information and data to enable gray-box testing and effective defect diagnosis.Part V details the effective design and development of test procedures, including considerations for the creation and documentation of tests, as well as a discussion of testing techniques. As requirements and system design are further refined over time and system development iterations, so are the test procedures refined to include the new or modified requirements and system functions.Part VI examines the use of developer unit testing in the overall testing strategy. Unit testing in the implementation phase can result in significant gains in software quality. If unit testing is done properly, later testing phases will be more successful. There is a difference, however, between casual, ad-hoc unit testing based on knowledge of the problem, and structured, repeatable unit testing based on the requirements of the system.Part VII explains automated testing tool issues, including the proper types of tools to use on a project, the build vs. buy decision, and concerns for deciding on the right tool for the organization. The numerous types of testing tools available for use throughout the development life cycle phases are described here. In addition, custom tool development is also covered. Part VIII discusses selected best practices for automated testing. The proper use of capture/playback, test harnesses, and regression tests are described.Part IX provides information on the non-functional test considerations for a software application. Items discussed include performance, security, usability, compatibility, and concurrency testing.Part X provides a strategy for managing the execution of tests, including the appropriate way to track test procedures, defects, and gather metrics.AudienceThe target audience of this book is Quality Assurance professionals, software testers and test managers. Much of the information presented can also be of value to project managers and software developers looking to increase the quality of a software project.