A glossary of evidence words: primary source, sample, selection, case study

Arguing about business advice is hard partly because the vocabulary for talking about evidence is missing. People say "study", "data", "case study" and "proof" interchangeably, and the conversation collapses into who sounds more confident.

So this page defines the evidence words. Each entry gives the class and the one difference from its nearest neighbor, and says what the thing can and cannot support.

What to take away

Words about where the material came from

Primary source. A record made by a participant or an instrument at the time.

Its nearest neighbor is the secondary source, which describes or interprets one. The difference matters because interpretation accumulates: a claim repeated through three secondary sources usually differs from what the first one said. The general distinction is set out in the entry on the primary source.

Account. A description given afterward by someone involved. It is a primary source about the person's recollection and a weak source about what happened, because it was formed knowing the outcome.

Record. Something created as part of doing the thing rather than as part of describing it. Records are usually the strongest available material and the least interesting to read, which is why they are underused.

Words about how the cases were chosen

This is where most business writing fails, and the vocabulary is worth having

Look first at the fourth row. A book about companies that succeeded, with no account of companies that did the same things and failed, describes a set rather than explains it. The formal name is selection bias; it is not a technicality and can reverse a conclusion entirely.

A recommended books case study shows how selection bias works in a single business book.

Words about what a result shows

Association. Two things occurring together. It supports a place to look and

nothing more.

Mechanism. A stated route from cause to effect. Its value is that you can check whether the route exists in your situation, which is the only way general advice becomes specific.

Replication. A result found again by someone else under the same conditions. Its neighbor is repetition, which is the same people finding the same thing again, and which shows much less.

Generalization. A move from studied cases to unstudied ones. Legitimate when the rule for choosing cases is stated and the new case resembles them. Illegitimate, and extremely common, when neither condition is checked.

Words about strength

• Evidence is anything that changes how likely a claim is. It comes in degrees.

That last entry is worth remembering. "Consistent with" is a real and useful phrase in careful writing and a way of implying support in careless writing.

Using the vocabulary

The point of having these words is to be able to say what is missing rather than to say you are unconvinced. "This is a set of accounts with no comparison group" is a specific, checkable objection. "I am not sure I buy it" is not.

The twelve claim checks turn this vocabulary into a reading procedure, and the recommended books guide covers how the different types of book fail. Where a claim is about people's abilities rather than about businesses, the assessment guide covers the additional properties an instrument has to have before a result means anything.

What this vocabulary cannot do

It cannot make a weak claim strong, and it cannot settle a question that turns on your own circumstances. Two people with the same vocabulary can still disagree about whether a mechanism applies to their case, and that disagreement is legitimate and is resolved by trying something small rather than by arguing.

It also cannot substitute for expertise in a regulated area. Where a claim concerns money, employment or law, the vocabulary tells you what to ask; the answer comes from someone qualified where you are.

Common questions

Is one well-documented case worth anything?

Yes, for showing that something is possible and for exposing a mechanism. It is worth nothing for saying how often it works, and the two uses are constantly confused.

How do I tell an account from a record?

Ask when it was made and why. Something written to do the work is a record; something written to explain the work afterward is an account.

Should I distrust everything without a comparison group?

Distrust the general conclusion, not the material. A set of interesting cases is useful; the sentence at the end of the chapter that turns them into a rule is where the caution belongs.

Where can I see this vocabulary used properly?

In any field that publishes its methods. Public bodies that test their own material with readers, as described in federal guidance on testing for understanding, state what they did and what it showed, which is the habit worth copying. The decision record fields are the small-scale version.