Skip to content
Study Hall
Lesson 3 of 7Software Development & the CPT
Unit 7 · Lesson 3 of 7

Testing, Debugging and Documentation

Choosing test cases that find bugs, reading what Java tells you when it crashes, and writing code the next person can maintain.

6 min read12 practice questionsAsk AI about this →

Key things to remember

  1. 1Every test set needs normal, boundary and invalid cases; boundary cases find the most bugs.
  2. 2Test the boundary value, one below it and one above it — that is where < and <= disagree.
  3. 3Syntax errors stop compilation, runtime errors stop execution, logic errors produce a wrong answer in silence.
  4. 4A stack trace gives the exception type, the message, and the exact file and line where it stopped.
  5. 5A trace table finds logic errors by executing the code by hand exactly as written.
  6. 6Comments should explain why, not what; the code already says what.

Running a program once with the numbers you had in mind while writing it proves almost nothing — of course it works on those. Testing is the deliberate search for inputs that break the program, and it is a planned stage with written cases and expected results decided in advance. Almost every logic bug lives at a boundary, because a boundary is exactly where < and <= disagree, and they disagree on precisely one value. Test with 73 and both versions pass; test with 100 and only the correct one does.

KindWhat it checksFor a mark from 0 to 100
NormalThe ordinary case works50, 73, 88
BoundaryThe exact edges of every valid range0, 1, 99, 100
InvalidBad input is rejected instead of accepted−1, 101, 3.5, "abc", nothing at all
A test set with no boundary and no invalid cases is not a test set.
Test on the boundary, one below and one above
For a limit of 100, test 99, 100 and 101. For an array, test the first element, the last element, and an empty array. Three values per boundary will find more bugs than fifty values chosen from the middle of the range, because the middle cannot distinguish a correct comparison from an incorrect one.
Debugging is a routine, not a talent. Follow the same five steps every time.1Reproducefind the exact input that makes it fail, every single time2Read the errorthe message names the line number and the kind of fault3Isolatetrace table or println at each step — find the first wrong value4Change one thingthe smallest edit that could explain what you actually saw5Re-testrun the failing case and the cases that used to workStill broken? Go back to step 3 with what you just learned — and never change two things at once, or you cannot tell which one helped.
Fig 1Reproduce, locate, understand, fix, then re-test everything — not just the part you changed.
TypeWhen it appearsJava exampleHow you find it
SyntaxAt compile time — the program never runsMissing semicolon, unmatched braceThe compiler names the file and line
RuntimeWhile running — the program stopsDivide by zero, ArrayIndexOutOfBoundsException, NullPointerExceptionA stack trace naming the exception and the line
LogicNever announced — the program runs and is wrong> written where >= was meantOnly testing against expected results reveals it
The later an error shows itself, the more it costs you.you type itit runsit shipsSYNTAXBefore the program runs at allprint("hi" ← missing bracketPython refuses to startRUNTIMEPart-way through, at the failing linemarks[6] on a 6-item listCrashes with a tracebackLOGICNever — it runs and looks finetotal = total * m, meant +Wrong answer, no warningOnly testing against answers you worked out yourself will find a logic error — the computer will never tell you.
Fig 2Compile-time, run-time and logic errors, and the point in the process at which each one shows itself.

Treat the three types as a diagnostic routine rather than a list to memorize. Did it refuse to compile? Syntax — read the first error message only, because later ones are often knock-on effects of the first. Did it start and then stop? Runtime — read the stack trace, which names the fault and the exact line. Did it run cleanly and produce the wrong answer? Logic, and now you need a trace table: one column per variable, one row per pass, filled in by hand, executing the code exactly as written rather than as intended. The moment a row disagrees with the value you expected, the bug is in the statement you just executed.

Tracing by hand is how you find a logic error the computer will not report.total = 0i = 1while i <= 4: total = total + i i = i + 1itotalpasscondition—00—1111 ≤ 4 true2322 ≤ 4 true3633 ≤ 4 true41044 ≤ 4 true51055 ≤ 4 FALSE — exitFinal total = 10One row per pass.Write down everyvariable, every time.Forget the i = i + 1 line andthe table never ends: infinite loop.
Fig 3One row per pass, one column per variable, filled in by hand from the code as written.
int[] data = {3, 7, 2};
int total = 0;
for (int i = 0; i < data.length; i++) {
    total += data[i];
}
System.out.println(total);
Passii < 3?data[i]total after
before———0
10true33
21true710
32true212
exit3false—12
The trace proves the output is 12, and would have exposed a wrong loop bound immediately.

When a Java program throws, it prints a stack trace. It looks intimidating and it is actually a gift: it tells you the exception type, a message, and the exact line of the exact file where the program stopped. Read it top down — the first line is what went wrong, the second is where, and each line after that is the call that led there, so the last line is usually main. The message itself carries both numbers you need: "Index 5 out of bounds for length 5" says what you asked for and what was actually there, and when those two numbers are equal the cause is nearly always a loop condition using less-than-or-equal instead of less-than.

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: Index 5 out of bounds for length 5
    at MarkReport.average(MarkReport.java:18)
    at MarkReport.main(MarkReport.java:7)
Open line 18 of MarkReport.java, inside the method average. The line below it is only the route that got there.
APrint-statement debugging
  • Add System.out.println to show a variable's value at a point
  • Always available, needs no setup, works anywhere
  • You must guess in advance which values matter
  • It changes the very code you are diagnosing, and forgotten ones become bugs of their own
BA real debugger
  • Set a breakpoint and the program pauses on that line
  • Inspect every variable at once without changing the code
  • Step line by line, or step into a method call, and watch values change
  • Wins on any bug that takes more than two guesses

Maintenance is the longest stage of the life cycle, and the person doing it is usually reading the code cold. Four conventions do most of the work. Name variables for what they hold — passCount, not x or temp2. Indent four spaces per level every time, because inconsistent indentation hides the structure the logic depends on. Use comments to explain why a decision was made, never to restate what the code obviously does. And put a short description above every method saying what it takes, what it returns, and what happens when it fails. Keeping methods short enough to fit on one screen makes all four easier.

// Hard to maintain: nothing here says what any of it is for
int x=0;
for(int i=0;i<a.length;i++){if(a[i]>=50){x++;}}

// Maintainable: the names carry the meaning and the comment explains WHY
int passCount = 0;
for (int mark : marks) {
    // 50 is the reporting cut-off set by the specification, not an arbitrary number
    if (mark >= 50) {
        passCount++;
    }
}

/**
 * Returns the index of the first occurrence of target in arr,
 * or -1 if target does not appear in the array.
 */
public static int indexOf(int[] arr, int target) {
    for (int i = 0; i < arr.length; i++) {
        if (arr[i] == target) {
            return i;
        }
    }
    return -1;
}
Both loops compile to the same thing. Only one of them can be changed safely in six months.
A comment that repeats the code is worse than no comment
i++; // add one to i tells the reader nothing they could not see, and it becomes a lie the moment the line changes. Comment the reasoning instead: why this cut-off, why this order, why this special case exists. If a comment is needed to explain what a line does, the better fix is usually a clearer variable name.

Worked examples

Reveal one step at a time and try to beat it to the next line.

Example 1
Design a full set of test cases for a method that accepts an age from 13 to 19 inclusive and rejects anything else.
Example 2
A program crashes with: ArrayIndexOutOfBoundsException: Index 5 out of bounds for length 5, at MarkReport.average(MarkReport.java:18). Diagnose it.
Example 3
This is supposed to count marks of 50 or more but reports 2 for {50, 62, 49, 88}: for (int m : marks) { if (m > 50) { count++; } } Find the error type and fix it.

Where marks get lost

  • Testing only with data the program is expected to handle, so the boundaries and invalid inputs are never exercised.
  • Running a test without writing down the expected result first, which makes it impossible for the test to fail.
  • Reading the last compiler error instead of the first — later errors are usually knock-on effects of the first one.
  • Ignoring the stack trace and guessing at the cause, when it already names the exception, the file and the line.
  • Writing comments that restate the code, which add nothing and become false as soon as the line changes.

Flashcards 10

Name the three kinds of test case.
Why do bugs cluster at boundaries?
Which three values should you test around a limit?
Syntax error
+ 6 more

Practice

1
A method accepts values from 1 to 100 inclusive. Which set of test values is most likely to find a bug?
2
A program compiles and runs without stopping, but every average it reports is slightly too low. What type of error is this?
3
Trace: int[] data = {3, 7, 2}; int total = 0; for (int i = 0; i < data.length; i++) { total += data[i]; } What is printed?
4
In the stack trace line at MarkReport.average(MarkReport.java:18), which line number should you open first?
5
Match each error type to how you find it.
Syntax error
Runtime error
Logic error
Off-by-one
6
For a valid range of 1 to 100 inclusive, how many test values does the boundary rule require in total (each boundary, plus one value on either side of it)?
values
7
Which comment is worth writing?
8
Order the steps of a disciplined debugging process.
  1. 1Re-run the whole test set, not just the failing case
  2. 2Make the fix
  3. 3Understand why that line is wrong before changing anything
  4. 4Reproduce the fault reliably with a specific input
  5. 5Locate the line using the stack trace or a trace table
9
What is the main advantage of a breakpoint over adding print statements?
10
A test set contains only inputs the program is supposed to accept. What is the weakness?
11
What debugging tool has one column per variable and one row per pass of a loop?
12
A compiler reports 14 errors. What should you do first?

Spot a mistake on this page? Let us know