Make exploratory testing session-based test management accountable

Why exploratory testing finds what scripted passes miss#

Exploratory testing runs learning, test design, and execution in the same moment. Because the tester shapes the next check from what the last one revealed, the path bends toward risk. A scripted pass cannot do that. It only runs the cases someone wrote in advance. Therefore it never exercises the defect nobody anticipated.

The coupon-stacking bug in our session is the plain example. No written case said "stack two percentage coupons until the total goes negative". Yet a curious tester found it in minutes. In practice, that is the class of defect a scripted suite structurally misses. However, finding more is only half the story.

The accountability gap: who tested what, when, with what oracle#

Exploratory testing catches more, and managers still distrust it. Their reason is fair. A scripted suite leaves a record: the case list, the pass or fail, the run date. Freeform exploration often leaves nothing but a verbal "I looked at checkout". Consequently the work is real but invisible. That invisibility is the accountability gap. Moreover it is not a testing problem. It is a documentation problem.

One instrument, one method: how a session, a sheet, and typed notes fit together#

Session-based test management is the method. Rapid Reporter is the instrument. Most articles treat them as two topics. In practice they are one motion. First a charter defines a timeboxed mission. Then the tester works that mission and types one-line notes as they go. Because the tool timestamps each note, the session produces a sheet with no extra effort.

Next that sheet exports to CSV, rolls up into metrics, and feeds a debrief. Finally the open questions become the next charters. This is exploratory testing session-based test management as a single loop, not two subjects. For the deeper definition of charters and the SBTM method itself, read our companion on how manual testing strengthens automation coverage. The diagram traces the whole cycle.

The note taxonomy, and where each note goes next#

Every note carries a type prefix. Because the prefix is literal text, the log stays readable and the meaning never depends on color. The six prefixes below are the ones the session used. In particular, watch the last column. Each note type feeds a specific downstream artifact, which is what turns a log into a work queue.

Read the BUG row as the key handoff. Because a bug note is already timestamped and stateful, it drops into triage with its repro anchor attached. That is where why bug triage discussions matter takes over the story. Meanwhile a TEST note that found a real risk is the raw material for a durable automated check.

One real 90-minute session, shown in full#

Here is exploratory testing session-based test management as a single receipt. The mission was narrow. Rapid Reporter was the instrument. What it produced is a 42-note sheet you can read, replay, and audit. First comes the charter. Then the live note stream. Finally the exported artifact a manager can grep.

The charter we ran#

A charter is a mission with a time budget, nothing more. Because it is written before the session, it makes scope a decision instead of a mood. This is the exact charter we ran.

The live note stream: add your own, watch the sheet build#

Below is the actual session, seeded into a working Rapid Reporter simulator. The color-coded, timestamped stream is the receipt, and it is fully present with JavaScript disabled. Because the controls sit above the stream, you can pick a note type, type a one-liner, and append it to the sheet. Then export the whole thing as CSV, exactly as the real tool does.

Notice what the timestamps buy you. Because every line is pinned in time, the debrief can point to the exact moment a defect surfaced. For example, at 47:00 the note reads "stacking two coupons yields a 120% discount, a negative order total". That single row is the difference between a claim and a receipt.

The exported session report a manager can grep#

The export is the point. Rapid Reporter writes each note to a CSV row with its sequence, its elapsed timestamp, its type, and its text. Because the format is plain, the artifact is indexable, greppable, and auditable long after the session closes. Here is the raw sheet the session produced.

seq,elapsed,type,note
1,00:20,SETUP,"Env is staging build 4.18.2, seeded catalog, account checkout_qa@example.test"
2,01:10,SETUP,"Coupons on file: SAVE10, FLAT15, FREESHIP, WELCOME20 (new-customer), BULK5 (per-item)"
3,02:40,SETUP,"Cart baseline is 3 items, subtotal 120.00, standard shipping 8.00"
4,04:15,SETUP,"Oracle is the pricing-rules doc v3 plus the coupon admin config"
5,05:30,SETUP,"DevTools network open on /cart/apply-coupon to watch each response"
6,07:05,TEST,"Apply SAVE10 to 120.00, expect 12.00 off"
7,08:12,CHECK,"Total is 116.00 including 8.00 shipping, matches the rules doc"
8,09:40,TEST,"Remove SAVE10, expect the total back to 128.00"
9,11:15,TEST,"Apply FLAT15, expect a flat 15 off the order"
10,12:00,CHECK,"Total is 113.00, correct"
11,14:30,TEST,"Apply EXPIRED1, expect a rejection"
12,15:10,CHECK,"Rejected with ""coupon has expired"", a clear message"
13,17:20,TEST,"Apply WELCOME20 on an existing account, expect a new-customer rejection"
14,19:05,BUG,"WELCOME20 accepted for an existing account; the new-customer gate is not enforced"
15,21:00,TEST,"Enter lowercase save10, expect the same result as SAVE10"
16,21:50,CHECK,"Accepted, 12.00 off, case-insensitive as intended"
17,23:30,TEST,"Enter a code with surrounding spaces, expect a trim and accept"
18,25:10,BUG,"Surrounding spaces are rejected as an invalid code; the input is not trimmed"
19,26:40,QUESTION,"Should codes be trimmed on paste, or is an exact match intended?"
20,29:15,TEST,"Apply SAVE10 then FLAT15, expect the single-coupon rule to block the second"
21,31:40,BUG,"SAVE10 and FLAT15 both apply; the rules doc says one coupon per order"
22,34:05,TEST,"Apply BULK5 per-item across 3 items, expect 15 off"
23,35:30,CHECK,"15.00 off, the per-item math is correct"
24,38:10,TEST,"Change the quantity after a coupon is applied, expect a recalculation"
25,40:25,BUG,"The discount is not recalculated after a quantity change; a stale 15.00 is held"
26,44:30,TEST,"Stack two percentage coupons on a high-value cart, expect a block"
27,47:00,BUG,"Stacking two coupons yields a 120% discount, a negative order total, and checkout stays enabled"
28,50:15,TEST,"Proceed to the test gateway with the negative total"
29,52:40,BUG,"Payment accepts the negative total, amount due is 0.00, and the order is creatable"
30,55:30,TEST,"Apply FREESHIP, expect shipping 0.00"
31,56:20,CHECK,"Shipping zeroed, subtotal unchanged, correct"
32,58:40,TEST,"Apply FREESHIP with SAVE10, expect free shipping and 10% off goods"
33,61:15,TEST,"Apply a coupon on an already-reduced sale item, expect a precedence rule"
34,63:30,BUG,"The coupon stacks on the sale price with no precedence; a double discount results"
35,66:10,TEST,"Apply a coupon, abandon, return to the cart, expect defined persistence"
36,68:00,QUESTION,"Is an applied coupon meant to survive a session timeout?"
37,71:20,TEST,"Submit a 300-character code, expect a graceful rejection"
38,74:30,TEST,"Apply a coupon through the API, bypassing form validation"
39,77:30,QUESTION,"Can the API apply a coupon the form would reject?"
40,80:10,QUESTION,"Should the payment step floor the order total at zero before authorization?"
41,84:30,NEXT-TIME,"Dedicated security charter for coupon API auth and rate limiting"
42,87:40,NEXT-TIME,"Regression charter for the negative-total and stacking defects once fixes land"

The accountability comes from this trail, not from a script. A reviewer can filter the type column to read every BUG in seconds. Furthermore they can jump to 47:00 and see the state that produced the worst defect. In short, the CSV is the object that converts unscripted testing into an auditable record.

What exploratory testing session-based test management proves: coverage and metrics#

A sheet is only useful if it rolls up. Because every note is typed and timestamped, the rollup is arithmetic, not opinion. The session logged 42 notes across the six types. Moreover it produced 7 filed bugs and 4 open questions, and those questions became 2 charters for the next session.

The time split matters as much as the note count. Session-based test management reports time in three buckets: Test, Bug, and Setup. Applying that breakdown to the 90 minutes gives a clear shape. In addition, the on-charter versus on-opportunity split ran roughly 80/20, so most of the time served the stated mission.

These numbers are the accountability payload. Because they derive from a timestamped log, a manager can trust them without watching the tester work. Contrast this with an equivalent scripted pass. That pass had no case for coupon stacking, so it never exercised the defect at all.

The debrief: turning the sheet into an answer managers trust#

The debrief is a short conversation the sheet makes possible. Session-based test management frames it as PROOF: Past, Results, Obstacles, Outlook, and Feelings. Because the log is the shared reference, the debrief is a walkthrough, not a memory test. Read the five stages in order.

That walkthrough is where exploratory testing session-based test management earns the word accountable. Consider the exchange a debrief actually produces.

When to explore and when to automate, the honest limit#

A session is not a regression net, and it is not a substitute for automated tests. It complements automated regression; it never replaces it. Because the two answer different questions, the honest move is to name which one a situation calls for. The matrix below scores that decision across five factors.

Use the two together, in sequence. First a session finds the defect nobody scripted, as the coupon-stacking bug showed. Then the durable ones become automated checks so they never return. The same instinct governs structured accessibility testing sessions, where a charter drives an exploratory pass before automated audits, and it shows up again in the testing techniques used in safety-critical automotive software.

Authoritative sources#

This method and this tool both have primary sources worth reading. Rapid Reporter is Shmuel Gershon's free note-taker for exploratory sessions, documented on his Rapid Reporter project page. Session-based test management was defined by Jonathan and James Bach in the original SBTM paper. For the wider practice, see James Bach on exploratory testing and Cem Kaner's exploratory testing reference. Read them for the theory; use the session above for the practice.