The psychology of bugs: why defects happen and which checks catch them
Most escaped defects are not bad luck. They follow a few habits of thought that research has counted, and each habit has a check that would have stopped it.
What is the psychology of bugs, and why do bugs happen?#
Bugs happen because people make errors, and a field study in Communications of the ACM (April 2022) found a cognitive bias in 45.72% of developer actions. The 2022 study's researchers watched 10 developers at a US startup do their normal work while thinking aloud. They coded 2084 separate actions. In the April 2022 paper's words, "approximately half of developer actions (45.72%) were associated with some form of bias".
Show data table
| Item | Value |
|---|---|
| Actions that carried a bias | 45.72% |
| Reversals that involved a bias | 68.75% |
| Biased actions that were reversed | 79.64% |
| Observed time spent reversing biased actions | 34.51% |
Almost half of the observed actions carried a bias, and most biased actions were later undone.
So the psychology of bugs is not a theory about careless people. Instead, it is the study of the mental shortcuts every software engineer takes, and of the defects those shortcuts leave behind. A cognitive bias is a steady, one-way lean in judgement. It is the same lean, in the same kind of situation, every time. Because the lean is steady, the defect it writes is predictable. And a predictable defect can be caught on purpose.
That answer also settles the question "why do defects happen?" The defect is the trace in the code, and the bias is the push that put it there. The error is the act that joined them. Therefore the useful question after an escaped defect is never "who was careless?" Instead, ask which shortcut it followed, and which check forces the step that shortcut skipped.
What is a defect in software testing, and how is it different from an error or a failure?#
An error is a human action that produces an incorrect result, a defect is that result left in the code, and a failure is the defect running. The ISTQB, the International Software Testing Qualifications Board, sets these terms in its standard glossary, version 2.2 of 19 October 2012. It defines an error as "A human action that produces an incorrect result."
The same 2012 glossary calls a defect "a flaw in a component or system" that "can cause the component or system to fail". Then a failure is the visible event: the system's "deviation" from its "expected delivery, service or result". In short, the error happens in a head, the defect sits in the code, and the failure happens in front of a user.
The chain matters because each practice acts on a different link. For example, a design review acts on the error, before any code exists. Next, a test acts on the defect, before it ships. Finally, monitoring acts on the failure, after users meet it.
Defect leakage measures how many defects slip past the phase meant to catch them. The ISTQB glossary of 2012 defines the mirror figure, the Defect Detection Percentage. It is the number of defects a test phase found, divided by those it found plus any found afterwards. So defect leakage is the rest: the share found later, often by users. When a phase leaks a lot, it is rarely just weak. More often, it shares a blind spot with the people who wrote the code.
What does prone to human error mean for a software engineer?#
Being prone to human error means a software engineer will slip, lapse or make a mistake at some steady rate, so the risk is managed rather than removed. The risk of human error never reaches zero, because the same mind that plans the work also does it.
Hu, Carver, Walia, Anu and Bradshaw built a Human Error Taxonomy (HET) for software requirements from psychological research. Their online survey in CEUR Workshop Proceedings, 2017, splits human error into three kinds, with a drive to the store as the example. "A slip occurs when you perform the planned steps incorrectly or in the wrong sequence," like putting the wrong key in the car. Next, a lapse is when you forget the goal midway or "omit a step that is part of a routine sequence". Finally, a mistake is when you "design an incorrect plan to begin with".
Each kind leaves a different defect, and each needs a different check:
| Kind of error | What happens | Typical defect | Check that catches it |
|---|---|---|---|
| A slip | The planned steps are performed incorrectly or in the wrong sequence | A typo in a constant or two lines in the wrong order | Linters, type checkers and a careful diff review |
| A lapse | A step that is part of a routine sequence is omitted | The migration you meant to write and forgot, or the null check you always add and skipped once | Checklists and required tests |
| A mistake | An incorrect plan is designed to begin with | A correct build of the wrong plan, such as a cache that was never safe to share | Only a review of the plan, before the code, catches it cheaply |
So biases mostly drive mistakes. While a slip is a fumble, a biased plan is often carried out perfectly. That is why the rest of this guide spends its time on biases.
Which cognitive biases has software engineering research actually studied?#
A 2018 systematic mapping study in IEEE Transactions on Software Engineering found 65 studies covering 37 biases, led by anchoring in 26, confirmation in 23 and overconfidence in 16. Mohanani, Salman, Turhan, Rodriguez and Ralph published the review in 2018. It covers papers published between 1990 and 2016.
Show data table
| Item | Value |
|---|---|
| Anchoring bias | 26 |
| Confirmation bias | 23 |
| Overconfidence bias | 16 |
Anchoring, confirmation and overconfidence lead the 65 studies the 2018 mapping study found.
The mapping study defines each of the three in one sentence, and the definitions are worth keeping:
- Anchoring bias is "the tendency to stick too closely to the anchor", the first value or idea you met.
- Confirmation bias is "the tendency to pay undue attention to sources that confirm our existing beliefs while ignoring sources that challenge our beliefs".
- Overconfidence bias is "the tendency to overestimate one's skills and abilities".
The same review also found a gap. Its abstract reports "a scarcity of research on mitigation techniques". In other words, researchers know far more about which biases occur than about what stops them. So treat any confident list of fixes, including the one later in this guide, as reasoned practice rather than proven cure.
How often do biases shape a developer's real working day?#
Observing 10 developers at work, the April 2022 Communications of the ACM field study counted 2084 actions, 953 of them biased, and 759 of those biased actions were later reversed. Most earlier research ran in labs. By contrast, this 2022 study watched real tasks on a normal workday.
Show data table
| Item | Value |
|---|---|
| Developer actions observed | 2,084 |
| Actions carrying a bias | 953 |
| Reversal actions | 1,104 |
| Biased actions later reversed | 759 |
Of the 953 biased actions, 759 were later reversed.
The key result is about undoing. In the April 2022 study, biased actions were reversed 79.64% of the time, and 68.75% of all reversals involved a bias. A reversal is the moment a developer backs out work: deletes a fix, rewrites a query, or goes back to an old version. So each one is a defect caught late, by the person who wrote it, after the time is already spent.
The study's own example shows how small the first step is. One participant needed part of a hashmap and did not know its query interface. So he chose to collect the data by hand. As he put it, "Easiest thing to do would be to collect all input statements" by hand. After 18 minutes, the 2022 paper records, he found the manual route far harder than he had expected. Then he went back to learn the query. The paper files this under two of its categories: convenience and preconception.
He needed part of a hashmap.
He did not know its query interface.
He chose to collect the data by hand.
"Easiest thing to do would be to collect all input statements" by hand.
After 18 minutes, the manual route was far harder.
Harder than he had expected, the 2022 paper records.
He went back to learn the query.
The paper files this under two of its categories: convenience and preconception.
How many hours a week does undoing biased work cost?#
At the 34.51% share of observed time spent reversing biased actions in the April 2022 field study, a 40-hour week in this worked example loses about 13.8 hours to undoing them. The arithmetic is plain: 40 hours times 0.3451 is 13.8 hours, rounded to one decimal.
Hours a week spent reversing biased actions
Set the hours you work in a week. It applies the field study's 34.51% share of observed time spent reversing biased actions, and the paper's own summary of about 25% beside it.
Hours reversing biased actions, at the observed 34.51%
13.8
- Hours at the paper's summary of approximately 25%
- 10
Arithmetic on a published share from hour-long sessions with 10 developers; modelled, not measured.
But read the number with its limits. The Communications of the ACM paper of April 2022 calls its sessions "an hour-long observational study" of 10 people at one startup. Also, the 2022 paper's authors summarise the loss as "approximately 25%" of working time. So 13.8 hours is a ceiling drawn from short, watched sessions, not a promise about your team.
Even so, the figure turns bias from a soft topic into a number a lead can check. For instance, a team can count how many commits in a sprint undo or rewrite work from the same sprint. If that share is near a quarter, the study's picture fits. If it is near zero, the team is either very good or not looking.
How does confirmation bias write a defect, and which test catches it?#
Confirmation bias is the habit of testing to prove code works rather than to break it, and testers in a 2018 Springer experiment wrote more confirming than disconfirming test cases. Salman and colleagues published the controlled experiment in Empirical Software Engineering on 18 December 2018. They report that "participants designed significantly more confirmatory test cases as compared to disconfirmatory ones".
Here is the defect it writes. A developer builds a coupon check for a checkout. The tests try a valid code, a second valid code and a valid code in lowercase. All three pass, and the check ships. However, no test tries an expired code, so the check never compares the expiry date. Then the first customer with an old coupon gets a discount the shop never meant to give.
Yet every test in that suite was correct. Still, the defect lives in the tests that were never written, because each test was chosen to show the code working. That is confirmation bias in a test suite. In fact, the mapping study notes it shows up in testing as "positive test bias".
| Test case | Written? | What it shows |
|---|---|---|
| A valid code | Written | Passes: the code works |
| A second valid code | Written | Passes: the code works |
| A valid code in lowercase | Written | Passes: the code works |
| An expired code | Never written | Would fail: the check never compares the expiry date |
| Used, empty or wrong currency | Never written | The inputs that must fail, listed before any happy path |
Two practices catch it:
- Write the negative cases first. Before any happy path, list the inputs that must fail: expired, used, empty, wrong case, wrong currency. Our guide to writing clear and effective test cases shows how to structure them.
- Have someone who did not write the code design some tests. That person has no belief to confirm. How developers and testers collaborate covers how to set that up without friction.
Skill helps more than seniority here. In a 2010 paper for the Psychology of Programming Interest Group, Calikli, Arslan and Bener tested 88 subjects. They were developers, testers and graduate students. Their results showed "regardless of experience and company size, abilities such as logical reasoning and strategic hypotheses testing are differentiating factors in low confirmation bias levels".
How does anchoring bias lead to wrong fixes and wrong estimates?#
Anchoring is sticking to the first number or first hypothesis, and in a 2018 experiment with 410 developers a workshop cut the anchor's effect from a Cohen's d of 1.19 to 0.72. Cohen's d is a standard measure of effect size, and the authors rate an anchor effect of this size "between large and very large". In their 2018 paper on arXiv, Shepperd, Mair and Jørgensen report that "The anchors had a large effect" on how developers estimated their own productivity.
1.19
Without the workshop
0.72
After a bias awareness workshop
Awareness weakened the anchor without removing it: the authors still read 0.72 as a medium to large effect.
| Option | robust Cohen's d |
|---|---|
| Without the workshop | 1.19 |
| After a bias awareness workshop | 0.72 |
In the psychology of bugs, anchoring writes defects as well as bad estimates. Say a bug report reads "checkout times out since Tuesday's deploy". The developer anchors on the deploy and rolls back a config change. Then the timeouts stop for a day, because traffic dropped, and come back. Meanwhile, the real cause stays in the code: a slow query on a table that grew past its index. The April 2022 field study calls this pattern fixation. It means "anchoring problem-solving efforts on initial assumptions" and not adjusting "in light of added information or contradictory evidence".
The practices that catch it all force a second idea onto the page:
- Write down two rival causes before touching the code. Then say what evidence would rule each one out.
- Estimate alone, then compare. Independent estimates stop the first number spoken from setting the range.
- Let a fresh pair of eyes read the bug report. The field study lists different perspectives, such as pair programming, as a practice that helps with fixation.
The workshop result answers whether training helps. It does, but it does not finish the job. In the 2018 paper, the authors still read 0.72 as "a medium to large effect", so awareness weakened the anchor without removing it. That is why the practices above change the process rather than rely on people remembering a lesson.
Which review or test practice catches each bias?#
Each bias leaves a typical defect, so the catching practice is the one that forces the missing step: a second reviewer, a negative test, a written hypothesis or a checklist. The April 2022 field study grouped 28 observed biases into ten categories. It also sorted the practices developers suggested in interviews into five themes. The table maps each category to a defect it tends to write and the practice the study links to it.
| Bias category | One-sentence definition | Typical defect | Practice that catches it |
|---|---|---|---|
| Preconceptions | Acting on the mental model you arrived with | A solution that never fit the task | Read the docs first; systematic review of options |
| Ownership | Giving extra weight to code you wrote | A duplicate helper where shared code existed | Coding standards and standard libraries |
| Fixation | Holding the first assumption against new evidence | A fix for the symptom, not the cause | Rival hypotheses; a different perspective |
| Resort to default | Choosing an option because it is the default | A setting that does not fit the context | Coding standards; review of defaults |
| Optimism | Assuming a chosen solution is correct or fast | An untested change that breaks later | Tests that run on every change |
| Convenience | Taking the quick route and assuming a simple cause | A quick fix that becomes debt | Clean code training; review of quick fixes |
| Subconscious action | Trusting a tool's hint without judging it | Following an error message to the wrong place | Problem-solving routines; marked flaky tests |
| Blissful ignorance | Assuming all is well despite warnings | A warning ignored until it fails in production | Documented error severities |
| Superficial selection | Picking by surface features | A library or answer chosen without reasoning | Pair programming |
| Memory bias | Using what you remember first or last | A call to a deprecated API | Docs and team playbooks |
Source: categories, definitions paraphrased and practices from Cognitive Biases in Software Development, Communications of the ACM, April 2022; typical defects are illustrative.
Overconfidence, the third most studied bias in the 2018 mapping study, fits the optimism row. It skips the same step: checking that the work does what you believe it does. Therefore the same counter applies, a test that runs whether or not you feel sure.
To apply the psychology of bugs with this table, start from a defect rather than a bias. First, ask which step was skipped. Then find the row, and add that practice where the defect slipped through. Triage meetings are the natural place for this, and our piece on bug triage discussions shows how to run them. Also, structured reviews such as a heuristic evaluation or cognitive walkthrough add the different perspective several rows call for.
When is cognitive bias the wrong tool for explaining a defect?#
Bias is the wrong explanation when the defect comes from a gap between the team and its users, a missing requirement, or a deadline that the evidence does not blame. The psychology of bugs explains a lot, but a bias story is easy to tell after the fact, so test it before you accept it.
First, some defects come from users the team does not resemble. Chauhan, Arora, Khalajzadeh and Grundy call these human-centric defects, or HCDs, in a paper posted to arXiv on 3 August 2024. They arise from end-user "differences, such as their genders, ages, cultures, languages, disabilities". In that 2024 paper, the authors identified 176 HCDs among 1,100 issues drawn from open-source projects. Debiasing a developer will not find these. Instead, user research and testing with real, varied users is the better tool.
Second, some defects trace to a requirement that was never written. When the plan was missing a case, the fault sits with the plan. So a requirements review, with the HET survey's kinds of error in hand, is the better tool there.
Third, deadlines are a common scapegoat. However, the Springer experiment of December 2018 found that time pressure did not raise confirmatory testing. In the paper's words, "we did not observe time pressure as an antecedent to an increased rate of confirmatory testing behaviour". If a defect is blamed on rushing, check whether the same gap appears in work done without a deadline.
Where should a team go from here?#
Start with the last three escaped defects, name the error type and the bias behind each, and add the one practice that would have caught it. That small retrospective uses everything above: the error chain, the split into slips, lapses and mistakes, and the table.
Then go deeper on the practice you picked. Developer and tester collaboration puts a second mind on a decision before it ships. AI and machine learning in software testing covers automated checks that do not share a developer's beliefs about the code. And the pieces on bug triage and test case design turn two rows of the table into habits.
If a team wants a tester whose job is to look for the disconfirming case, our QA software engineers page describes that role. If escaped defects are already piling up in production, the maintenance and support page covers that work. But the studies linked above are enough to start on your own. They are free to read, and the field study alone will change how you read your next bug report.