Software Testing

Ideal for programmers and project managers who want to better understand what their software test teams do and how they operate.

Most large software companies are so committed to quality that they have one or more testers for each programmer.

Contents

Introduction

Part I - The Big Picture

Chapter 1 - Software Testing Background

• Infamous Software Error Case Studies
  • Disney's Lion King, 1994–1995
  • Intel Pentium Floating-Point Division Bug, 1994
  • NASA Mars Polar Lander, 1999
  • Patriot Missile Defense System, 1991
  • The Y2K (Year 2000) Bug, circa 1974
• What Is a Bug?
  • Terms for Software Failures
  • Software Bug: A Formal Definition
• Why Do Bugs Occur?
• The Cost of Bugs
• What Exactly Does a Software Tester Do?
• What Makes a Good Software Tester?
• Summary
• Quiz

Chapter 2 - The Software Development Process

• Product Components
  • What Effort Goes Into a Software Product?
  • What Parts Make Up a Software Product?
• Software Project Staff
• Software Development Lifecycle Models
  • Big-Bang Model
  • Code-and-Fix Model
  • Waterfall Model
  • Spiral Model
• Summary
• Quiz

Chapter 3 - The Realities of Software Testing

• Testing Axioms
  • It's Impossible to Test a Program Completely
  • Software Testing Is a Risk-Based Exercise
  • Testing Can't Show That Bugs Don't Exist
  • The More Bugs You Find, the More Bugs There Are
  • The Pesticide Paradox
  • Not All the Bugs You Find Will Be Fixed
  • When a Bug's a Bug Is Difficult to Say
  • Product Specifications Are Never Final
  • Software Testers Aren't the Most Popular Members of a Project Team
  • Software Testing Is a Disciplined Technical Profession
• Software Testing Terms and Definitions
  • Precision and Accuracy
  • Verification and Validation
  • Quality and Reliability
  • Testing and Quality Assurance (QA)
• Summary
• Quiz

Part II - Testing Fundamentals

Chapter 4 - Examining the Specification

• Getting Started
  • Black-Box and White-Box Testing
  • Static and Dynamic Testing
  • Static Black-Box Testing: Testing the Specification
• Performing a High-Level Review of the Specification
  • Pretend to Be the Customer
  • Research Existing Standards and Guidelines
  • Review and Test Similar Software
• Low-Level Specification Test Techniques
  • Specification Attributes Checklist
  • Specification Terminology Checklist
• Summary
• Quiz

Chapter 5 - Testing the Software with Blinders On

• Dynamic Black-Box Testing: Testing the Software While Blindfolded
• Test-to-Pass and Test-to-Fail
• Equivalence Partitioning
• Data Testing
  • Boundary Conditions
  • Sub-Boundary Conditions
  • Default, Empty, Blank, Null, Zero, and None
  • Invalid, Wrong, Incorrect, and Garbage Data
• State Testing
  • Testing the Software's Logic Flow
  • Testing States to Fail
• Other Black-Box Test Techniques
  • Behave Like a Dumb User
  • Look for Bugs Where You've Already Found Them
  • Follow Experience, Intuition, and Hunches
• Summary
• Quiz

Chapter 6 - Examining the Code

• Static White-Box Testing: Examining the Design and Code
• Formal Reviews
  • Peer Reviews
  • Walkthroughs
  • Inspections
• Coding Standards and Guidelines
  • Examples of Programming Standards and Guidelines
  • Obtaining Standards
• Generic Code Review Checklist
  • Data Reference Errors
  • Data Declaration Errors
  • Computation Errors
  • Comparison Errors
  • Control Flow Errors
  • Subroutine Parameter Errors
  • Input/Output Errors
  • Other Checks
• Summary
• Quiz

Chapter 7 - Testing the Software with X-Ray Glasses

• Dynamic White-Box Testing
• Dynamic White-Box Testing versus Debugging
• Testing the Pieces
  • Unit and Integration Testing
  • An Example of Module Testing
• Data Coverage
  • Data Flow
  • Sub-Boundaries
  • Formulas and Equations
  • Error Forcing
• Code Coverage
  • Program Statement and Line Coverage
  • Branch Coverage
  • Condition Coverage
• Summary
• Quiz

Part III - Applying Your Testing Skills

Chapter 8 - Configuration Testing

• An Overview of Configuration Testing
  • Isolating Configuration Bugs
  • Sizing Up the Job
• Approaching the Task
  • Decide the Types of Hardware You'll Need
  • Decide What Hardware Brands, Models, and Device Drivers Are Available
  • Decide Which Hardware Features, Modes, and Options Are Possible
  • Pare Down the Identified Hardware Configurations to a Manageable Set
  • Identify Your Software's Unique Features That Work with the Hardware Configurations
  • Design the Test Cases to Run on Each Configuration
  • Execute the Tests on Each Configuration
  • Rerun the Tests Until the Results Satisfy Your Team
• Obtaining the Hardware
• Identifying Hardware Standards
• Configuration Testing Other Hardware
• Summary
• Quiz

Chapter 9 - Compatibility Testing

• Compatibility Testing Overview
• Platform and Application Versions
  • Backward and Forward Compatibility
  • The Impact of Testing Multiple Versions
• Standards and Guidelines
  • High-Level Standards and Guidelines
  • Low-Level Standards and Guidelines
• Data Sharing Compatibility
• Summary
• Quiz

Chapter 10 - Foreign-Language Testing

• Making the Words and Pictures Make Sense
• Translation Issues
  • Text Expansion
  • ASCII, DBCS, and Unicode
  • Hot Keys and Shortcuts
  • Extended Characters
  • Computations on Characters
  • Reading Left to Right and Right to Left
  • Text in Graphics
  • Keep the Text out of the Code
• Localization Issues
  • Content
  • Data Formats
• Configuration and Compatibility Issues
  • Foreign Platform Configurations
  • Data Compatibility
• How Much Should You Test?
• Summary
• Quiz

Chapter 11 - Usability Testing

• User Interface Testing
• What Makes a Good UI?
  • Follows Standards or Guidelines
  • Intuitive
  • Consistent
  • Flexible
  • Comfortable
  • Correct
  • Useful
• Testing for the Disabled: Accessibility Testing
  • It's the Law
  • Accessibility Features in Software
• Summary
• Quiz

Chapter 12 - Testing the Documentation

• Types of Software Documentation
• The Importance of Documentation Testing
• What to Look for When Reviewing Documentation
• The Realities of Documentation Testing
• Summary
• Quiz

Chapter 13 - Web Site Testing

• Web Page Fundamentals
• Black-Box Testing
  • Text
  • Hyperlinks
  • Graphics
  • Forms
  • Objects and Other Simple Miscellaneous Functionality
• Gray-Box Testing
• White-Box Testing
• Configuration and Compatibility Testing
• Usability Testing
• Introducing Automation
• Summary
• Quiz

Part IV - Supplementing Your Testing

Chapter 14 - Automated Testing and Test Tools

• The Benefits of Automation and Tools
• Test Tools
  • Viewers and Monitors
  • Drivers
  • Stubs
  • Stress and Load Tools
  • Interference Injectors and Noise Generators
  • Analysis Tools
• Software Test Automation
  • Macro Recording and Playback
  • Programmed Macros
  • Fully Programmable Automated Testing Tools
• Random Testing: Monkeys and Gorillas
  • Dumb Monkeys
  • Semi-Smart Monkeys
  • Smart Monkeys
• Realities of Using Test Tools and Automation
• Summary
• Quiz

Chapter 15 - Bug Bashes and Beta Testing

• Only As Far As the Eye Can See
• Test Sharing
• Beta Testing
• Outsourcing Your Testing
• Summary
• Quiz

Part V - Working with Test Documentation

Chapter 16 - Planning Your Test Effort

• The Goal of Test Planning
• Test Planning Topics
  • High-Level Expectations
  • People, Places, and Things
  • Definitions
  • Inter-Group Responsibilities
  • What Will and Won't Be Tested
  • Test Phases
  • Test Strategy
  • Resource Requirements
  • Tester Assignments
  • Test Schedule
  • Test Cases
  • Bug Reporting
  • Metrics and Statistics
  • Risks and Issues
• Summary
• Quiz

Chapter 17 - Writing and Tracking Test Cases

• The Goals of Test Case Planning
• Test Case Planning Overview
  • Test Design
  • Test Cases
  • Test Procedures
• Test Case Organization and Tracking
• Summary
• Quiz

Chapter 18 - Reporting What You Find

• Getting Your Bugs Fixed
• Isolating and Reproducing Bugs
• Not All Bugs Are Created Equal
• A Bug's Life Cycle
• Bug-Tracking Systems
  • The Standard: The Test Incident Report
  • Manual Bug Reporting and Tracking
  • Automated Bug Reporting and Tracking
• Summary
• Quiz

Chapter 19 - Measuring Your Success

• Using the Information in the Bug Tracking Database
• Metrics That You'll Use in Your Daily Testing
• Common Project-Level Metrics
• Summary
• Quiz

Part VI - The Future

Chapter 20 - Software Quality Assurance

• Quality Is Free
• Testing and Quality Assurance in the Workplace
  • Software Testing
  • Quality Assurance
  • Other Names for Software Testing Groups
• Test Management and Organizational Structures
• Capability Maturity Model (CMM)
• ISO 9000
• Summary
• Quiz

Chapter 21 - Your Career as a Software Tester

• Your Job as a Software Tester
• Finding a Software Testing Position
• Gaining Hands-On Experience
• Formal Training Opportunities
• Internet Links
• Professional Organizations
• Further Reading
• Summary
• Quiz

Apendix A - Answers to Quiz Questions

Index