Chapter 4. Building hypotheses — running induction, deduction and abduction with AI

Key points

• The quality of an analysis is decided by the question and hypothesis, not the method
• There are three routes to a hypothesis: induction, deduction, abduction
• AI is a sounding board that runs all three at speed

The quality of an analysis is decided before the analysis

Learn a statistics package and you can analyse data. But answering the wrong question correctly means nothing. The quality of an analysis is decided far more by the question and hypothesis in front of it than by the accuracy of the calculation.

Coding and Building Systems said the AI writes the code and the human's work is putting into words what they want to build. Data analysis is exactly the same. The AI does the calculating. The human's work is turning what you want to confirm into a testable hypothesis.

So how is a hypothesis built? Research knows three routes.

Route 1: induction — finding a rule from observation

Inductive reasoning draws a shared tendency or rule out of individual observations and data.

Example: users of several brain-training apps are observed to "work at it at a fixed time each day" → the hypothesis that habit formation is contributing to the effect.

Useful when you want ideas from abundant data or from observation in the field. For a therapist, this is the feeling of noticing a pattern shared by the patients you have handled.

Route 2: deduction — deriving a prediction from theory

Deductive reasoning starts from an existing theory or principle and predicts what will happen in a particular case.

Example: the theory that price lowers the barrier to purchase → predicting that lowering the price of a cognitive-support service will increase uptake among older adults.

Useful when applying academic theory or a marketing framework (behavioural economics, STP, the four Ps) to the field. Close to the clinical judgement of applying a guideline recommendation to the patient in front of you.

Route 3: abduction — leaping from surprise

Abductive reasoning takes an event that existing knowledge cannot explain and leaps to the most plausible hypothesis: "probably, it is this."

Example: one particular mailing gets an unusually high response rate → guessing that it contained a story with high emotional resonance.

It shows its strength when reaching for meaning and feeling that data alone cannot show. Innovation and new concepts usually come from this leap. The something a veteran finds in "the patient whose test results explain nothing, but who somehow gets better" — that is abduction.

AI is a sounding board for all three

Here is where this guide really earns its keep. AI is a partner that runs all three at speed.

• Helping induction: hand over observation notes or a data summary and ask, "list the shared patterns you can see here and five possible hypotheses."
• Helping deduction: "derive hypotheses for raising uptake of this service from behavioural economics."
• Helping abduction: "give three surprising hypotheses that could explain this unexpected result; do not be afraid to leap."

Abduction especially is hard to do alone — you stay inside your usual thinking. Have the AI produce a large number of leaping hypotheses, and let the human pick out the interesting ones. Generation by AI, selection by human. That combination enriches hypothesis-building enormously.

A prompt to try (abduction booster)

You are a researcher who is good at ideas that cross fields. I made the following unexpected observation.

[Observation] (e.g. adherence to a home exercise programme was higher in the group given a deliberately simplified explanation than in the group given a careful one)

Give seven hypotheses that could explain this, ranging from the conventional to the far-fetched. For each, add one line on what data would test it if it were true. Surprising hypotheses worth testing are more welcome than obvious ones.

Chapter 4 summary

• Analysis quality is decided by the question and hypothesis, not the method
• Three routes: induction, deduction, abduction
• AI is a sounding board for all three; generation by AI, selection by human