THE ART OF DETECTION
a k a
SCIENTIFIC METHOD
Explained Through
QUESTIONS AND ANSWERS
by
Tyrone Lai
©
2006
[I]t has always been my habit to hide none of my
methods, either from my friend Watson or from any one who might take an
intelligent interest in them.
——Sherlock
Holmes in The
Of
late I have been tempted to look into the problems furnished by nature rather
than those more superficial ones for which our artificial state of society is
responsible.
——Sherlock Holmes in The Final Problem
QUESTIONS:
The
Art of Detection—does it have to do with Sherlock Holmes?
Where
does the name come from?
What
does the Art of Detection aim at?
What
do you mean … ‘hidden from them’?
Is
the Art useful only for solving crimes?
Scientists
follow the scientific method; do they also practise the Art of Detection?
You
prefer the name ‘The Art of Detection’ over ‘Scientific Method’; why?
Is
Scientific Method just a matter of making observations?
Is
the Art of Detection a branch of Logic?
When
Sherlock Holmes solves a crime he arrives at his results by inference; does he
not?
Reconstruction,
inference—it is all the same thing; isn't it?
After
I have deciphered a simple sentence, have I not inferred it from some premises?
Scientific
discoveries are conveyed by propositions; are these propositions inferred?
I
am acquainted with this Art; am I not?
Does
the Art of Detection lead to perfect knowledge?
Is
the knowledge obtained by following clues worthy of the name?
Why
do we need to have this Art explained?
How
is it possible that all knowledge is obtained by following clues?
How
did people catch criminals before they paid attention to the Art of Detection?
What
alternative is there besides confessions—even if confessions are sometimes
false?
On
those occasions when criminals confess voluntarily do we still need clues?
Do
we need to follow clues when there are eyewitnesses?
Should
detectives trust direct experience?
Perhaps
direct experience can be trusted when the situation is ideal?
If
even direct experience cannot be trusted how can detectives know anything?
What
are clues? How come they can lead us to knowledge of things hidden?
How
did you come to this idea that clues are the characteristics of structures?
Why
are some clues more significant than others?
Should
we discount insignificant clues?
Why
is it that an ordinarily insignificant clue can sometimes become significant?
Why
are clues so hard to find?
So
coherence is the criterion of truth?
If
coherence is not the criterion of truth, what is?
How
do you know your reconstruction correspond to the real thing?
Is
it not possible for two different reconstructions to produce the same clues?
Is
it possible for two wildly different reconstructions to produce the same large
number of clues?
What
is the Small Steps Principle?
Can
we take just anything to be a clue?
How
do we know what a clue means? Where does this knowledge come from?
Are
there unbreakable ciphers?
Why
are unbreakable ciphers not used more often?
Are
there unbreakable ciphers that can be used to encrypt long messages?
What
would a cryptogram look like, one that is generated by an unbreakable cipher?
Are
there things we cannot know?
How
can we ever have enough clues?
How
do we develop new clues from old?
Why
is it that new clues can appear?
What
if the hypotheses are wrong?
What
does it mean if new clues keep appearing?
What
is the Right Direction Catechism?
What
happens in an investigation after serious mistakes have been made?
What
is the Brick Wall Catechism?
Can
we avoid making assumptions when carrying out investigations?
What
if the assumptions are wrong?
Why
will the interpretation of clues be wrong after we have made wrong assumptions?
Do
assumptions have to be exactly right?
We
can be misled by an investigation based on wrong assumptions; no?
How
do we know what assumptions to make?
Do
we have to be sure our assumptions are right before moving on?
There
is a common warning about hidden assumptions; is there not?
How
do we know an assumption is wrong, hidden or otherwise?
What
is the ideal way for an investigation to begin?
Can
we be sure we are not looking for
shadows in an investigation?
Can
we begin an investigation just with suspicions?
Do
we have to be absolutely certain of some things before we can begin an
investigation?
When
an investigation begins with suspicions, are we not prejudging its outcome?
Why
is it better that we collect as much evidence as we can as early as we can?
Why
is it that we can sometimes pin-point the evidence we need but not always?
Why
do we so often fail to recognize evidence?
Is
there anything we can do to prevent evidence from escaping us?
Does
evidence have to be error-free?
Do
scientists and detectives believe the senses are infallible?
Why
do we look for empirical evidence?
How
do we look for additional evidence?
Does
the search for additional evidence depend on feedback?
Are
positive feedback loops the same as circular reasoning?
The Art of Detection—does it have to do with Sherlock Holmes?
It does indeed since it is simply the art of finding and following clues.
Where does the name come from?
The name comes from Sherlock Holmes himself. In The Adventure of the Abbey Grange Sherlock Holmes was complaining that Watson, in his attempt to please readers, was not paying sufficient attention to the finer points of his art, that is, the art that Sherlock Holmes practises as a detective. As remedy he proposes to spend his retirement years 'in the composition of a textbook which shall focus the whole Art of Detection into one volume.'
What does the Art of Detection aim at?
The Art of Detection aims at knowledge of things hidden from those practising this Art, called detectives (or investigators). This is to say, detectives practise the Art of Detection in order that they can come to know things hidden from them.
What do you mean … ‘hidden from them’?
When a crime is taking place Sherlock Holmes is not there to witness it. He could be miles and miles away, not having the faintest idea this particular crime is occurring. But people want to know what happened during the crime; they want to know who is responsible; so they hire Sherlock Holmes. They hire him to find out something about which neither they nor Sherlock Holmes have any direct knowledge. This is what I mean by ‘hidden’. Detectives want to find out things hidden, things of which they have no direct knowledge.
Is the Art useful only for solving crimes?
No; we follow clues on all kinds of occasions, not just in solving crimes. Watson was out doing errands one morning. Without accompanying him Sherlock Holmes was nevertheless able to determine where Watson had been. How does Sherlock Holmes do it? By practising the Art of Detection. Millions of years ago there were dinosaurs roaming this earth. Who told us this? Scientists. How do they know? By following clues, that is, by practising the Art of Detection.
Scientists follow the scientific method; do they also practise the Art of Detection?
The scientific method and the Art of Detection are one and the same. Scientists are detectives and detectives are scientists. Sherlock Holmes switches between the two:
Of late I have been tempted to look into the problems
furnished by nature rather than those more superficial ones for which our
artificial state of society is responsible.
——Sherlock
Holmes in The Final Problem
Writers on scientific method do not usually think of scientific method as the art of finding and following clues; why?
Writers
on scientific method usually do not have much to say about clues at all. This
is because, like most people, they think clues can only suggest theories but
cannot help in determining their truth. In reality, clues not only suggest,
they can show us whether the theories suggested are true. This will surprise a
lot of people but in fact it is implicit in what most of us already know. Most
of us already know that after Sherlock
Holmes has solved a crime he will have no trouble showing that the crime as he
has reconstructed it leaves behind the same clues as the ones he actually
finds.
Now this is the common thing we do not only in crime detection but in science; in fact, this is the common thing we do whenever we follow clues. Whenever we follow clues, we are engaged in reconstruction, the reconstruction of those things we want to know whatever these things are. When we succeed, our reconstructions always leave behind the same clues as those we actually find. Why? Simply because this is how we measure success. If we have reconstructed a crime but it leaves behind a totally different set of clues, obviously we have failed.
The Art of Detection is the art of finding and following clues. Sherlock Holmes sometimes calls this Art his method. It is his method for arriving at true theories, or theories close to the truth, those very things the scientific method is supposed to bring about. In the past writers on scientific method had not paid much attention to the Art of Detection; they should from now on.
You prefer the name ‘The Art of Detection’ over ‘Scientific Method’; why?
For two main reasons:
1. The name ‘The Art of Detection’ brings to mind Sherlock Holmes and Sherlock Holmes reminds us of the importance of following clues. Most people do not think of scientific method as having much to do with clues. The ‘Art of Detection’ is a better name because clues are really important.
2. With the word ‘art’ in it ‘The Art of Detection’ alerts people to the fact that this Art is complex; which it is. For historical reasons ‘Scientific Method’ leads to the expectation that there is a simple, even fool-proof, method for arriving at the truth. There is no such method.
Is Scientific Method just a matter of making observations?
In exercising Scientific Method or the Art of Detection we have to make observations but Scientific Method is much more complex than just making observations.
Is the Art of Detection a branch of Logic?
No, it is not; it is an art in its own right. Photography is not a branch of Logic; neither is the Art of Detection. Logic, photography, the Art of Detection——these are separate arts.
When Sherlock Holmes solves a crime he arrives at his results by inference; does he not?
No; Sherlock Holmes solves crimes by reconstructing them——in such a way that the crimes reconstructed leave behind the same clues as those he actually finds.
Reconstruction, inference—it is all the same thing; isn't it?
No. If you succeed in reconstructing a vintage airplane you will have an airplane that resembles the original. To infer, you start with premises and end up with conclusions. Both premises and conclusions are propositions. But an airplane is not a proposition or a set of propositions.
After I have deciphered a simple sentence, have I not inferred it from some premises?
Clearly not. Suppose the sentence is, 'Zoro will be at the west gate tonight at eleven.' You are likely to have reconstructed it bit by bit, piece by piece. The fact that you have a sentence at the end does not mean that the sentence is inferred. We can reconstruct airplanes and we can reconstruct sentences. It is true that inference leads to conclusions which are sentences but not all sentences are produced by inference.
After Sherlock Holmes has solved a crime he conveys his solution to his audience in a set of propositions; are these propositions inferred?
In solving a crime Sherlock Holmes has reconstructed it, at least in its essentials. The fact that his solution can be conveyed in a set of propositions does not mean that his solution is inferred. The propositions merely describe the crime as he has reconstructed it.
Scientific discoveries are conveyed by propositions; are these propositions inferred?
Scientific discoveries are about structures present in the universe. To understand these structures scientists try to reconstruct them. Propositions conveying scientific discoveries describe some of these structures. These propositions are not inferred; they merely describe those structures scientists have succeeded, in varying degrees, in reconstructing.
Sherlock Holmes excels at reasoning; does this not show that the Art of Detection is a branch of Logic?
No; it merely shows Sherlock Holmes practises one art within another: the art of reasoning (Logic) within the Art of Detection. Einstein also excels at reasoning but that does not make theoretical physics into a branch of Logic.
I am acquainted with this Art; am I not?
In practice, yes. In practice we all know how to follow clues. Some are better at it, some not so good, but we all do it. We all follow clues. This is to say we all practise the Art of Detection. In this sense, yes we are all acquainted with the Art of Detection.
We all follow clues?
Yes, we all do. Some clues are simple, so simple that we all know how to make sense of them. Who for example is not able to correct a typographical error? But what are we doing when correcting typographical mistakes? We are relying on context for clues; context tells us how the mistake should be corrected. Now we depend on context not just in correcting typographical errors but in filling in missing knowledge in other kinds of instances. When context is rich enough we can figure out what is in a person’s mind without that person telling us; we have all done this frequently enough with people we know.
Does the Art of Detection lead to perfect knowledge?
No, the Art of Detection does not lead to perfect knowledge; it does not lead to the kind of knowledge philosophers traditionally demand but which most of them admit only God can have. We follow clues to uncover knowledge of things hidden. When things are hidden knowledge about them is difficult even when possible. Indeed, left to ourselves, we are likely to think knowledge of things hidden is impossible. If they are hidden, how can they be known?! For which reason, people often marvel at what Sherlock Holmes does; indeed, some simply would not believe anyone can do what Sherlock Holmes does. The idea that things hidden can be known is counterintuitive. When this is the case it is too much to expect that the knowledge we derive by practising the Art of Detection will be perfect.
When Sherlock Holmes has solved a crime he would not claim he knows everything that has happened. As he himself says, he only knows 'in essentials'. We live in a complicated world. When the world is complicated and the things we want to know are hidden, how is it possible that we can know everything, even if it is everything in a crime?
To help a jury understand what happened during a crime detectives sometimes re-enact the crime. Will anyone expect the re-enactment to duplicate the crime to the last detail? No; the recreation will be regarded as successful if it bears out the important points the detectives want to prove.
When we carry out investigations by following clues there will always be things relevant to the investigation which we want to know but which for the moment we do not. Some of these we may know later on, as we continue with the investigation; others we may never know. And among the things we know, some we will know with greater certainty, some with less. But there will be none that we will know with absolute certainty.
When we are following clues; when the things we want to know are hidden; it is easy to make mistakes. Now it is possible to correct some of these mistakes sometimes but there is no assurance that we can correct them all. Most of the time we do not even know they are there. This is to say, when carrying out investigations by following clues we can never rule out the possibility of hidden mistakes. We can reduce the probability of their occurring but we can never decrease this probability to zero.
Is the knowledge obtained by following clues worthy of the name?
The knowledge we obtain by following clues will never be perfect; however, it is not so imperfect that most of us will not regard it as worthy of the name. When conditions are favourable the quality of the knowledge we obtain by following clues sometimes surpass our expectations. More than one person has been surprised by the fact that some people can crack ciphers. Now ciphers are kept secret by those who use them. A common practice is to keep them in a safe under lock and key. But without breaking into the safe cryptanalysts (codebreakers) can sometimes tell you how these ciphers work. And if you take these ciphers out from the safe and compare them to the results obtained by these cryptanalysts, you will find they correspond. Would it be an exaggeration to say in these cases that the cryptanalysts know?
Ciphers are broken by following clues (Sherlock Holmes has broken a few in his career). Once upon a time, if it is known that you can crack ciphers you could be hauled before the Inquisition. Your charge? Communicating with the Devil! For, how else can you produce a message that corresponds exactly to the real one, the one that its author has taken so much trouble to hide, and which no other unauthorized person could decipher? In those days, there was no question that you knew; the only question was whether your knowledge was obtained with the help of the Devil.
We do obtain a kind of knowledge by following clues even though this knowledge is not perfect. To indicate that it is not ; to indicate that it is not the kind that will satisfy a Descartes or a Plato; I call it approximate knowledge. Approximate knowledge will never be absolutely certain. Sometimes it corresponds exactly to the real thing; at other times it only comes close. Approximations can sometimes be improved upon; this is true of approximate knowledge also. Sometimes we can bring approximate knowledge closer to the truth and/or make it more certain by continuing or expanding the investigation.
Why do we need to have this Art explained?
There are a number of benefits to be derived from an explanation of the Art of Detection:
1. The better our understanding of the Art of Detection, the better we will be when we come to practise this art.
2. Being better at practising the Art of Detection includes not making the more common kinds of mistakes. A better understanding of the Art of Detection will tell us what these common mistakes are.
3. Whether we practise the Art of Detection or not, we often have to evaluate results obtained by others who practise this Art, such as scientists or detectives. If we do not understand the Art of Detection we may not be able to provide a proper evaluation. For example, if we have no understanding that the Art of Detection can only lead to approximate knowledge, we could easily dismiss all results obtained by this Art on the ground that they are imperfect, as many do with scientific knowledge.
4. It is counterintuitive that we could have knowledge of things hidden but if we understand the Art of Detection we understand why in some cases this kind of knowledge is possible.
5. Many things we do while practising the Art of Detection can mystify laypeople. If we understand the Art of Detection we can explain these things to them.
6. In all likelihood all the knowledge human beings have is obtained by following clues. Philosophers have been asking for ages how human beings could know. If we understand the Art of Detection we have answered this question.
7. As educators have known for a long time, the best way for students to learn is to discover for themselves. Now we discover by following clues. A better understanding of the Art of Detection therefore should help both students and teachers. Teachers then can guide and students, discover for themselves.
How is it possible that all knowledge is obtained by following clues?
All claims to knowledge can be challenged. To answer these challenges we have to carry out investigations, which means we will have to follow clues (see also the answers to the next few questions).
How did people catch criminals before they paid attention to the Art of Detection?
Once upon a time to catch criminals the common practice was not to follow clues, but to round up likely and not-so-likely suspects and torture them until they confess. However, since torture often produces false confessions this practice is nowadays illegal in all civilized countries.
What alternative is there besides confessions—even if confessions are sometimes false?
According to Descartes——who is sometimes called the Father of Modern Philosophy, there is no alternative besides confessions even if they are sometimes false. If we are to have any knowledge at all (he thinks) ultimately we have to rely on confessions. Not, however, the confessions of just anybody——but God’s. What God lets us apprehend clearly and distinctly, Descartes says, must be true. This is the way God has chosen. Of course, He could have chosen otherwise but Faith requires us to believe He has not.
Perhaps we can add this to our list of benefits to be gained by having the Art of Detection explained. If we understand the Art of Detection we will then be able to see there is an alternative to relying ultimately on confessions (see also answer to the next question).
On those occasions when criminals confess voluntarily do we still need clues?
Because of the possibility of false confessions detectives have learnt to be cautious even with voluntary confessions. Some people desire notoriety and if confessing to a crime would give them notoriety, they would gladly do it even when they are not at all responsible. Or they may think they have actually committed the crime when in fact they have not: we all must have heard of the story of a person admitting to murder when in fact all that he was doing was shooting at a body already dead.
How do we find out whether a confession is false? We follow clues, that is, we practise the Art of Detection. A confession that contradicts all the clues cannot be true.
Do we need to follow clues when there are eyewitnesses?
Just as they have learnt to be cautious with confessions detectives also have learnt to be careful with eye-witnesses. Eye-witnesses to the same event often contradict each other. How do detectives find out who is correct, if any? They find out by following clues, that is, by practising the Art of Detection.
Detectives have to be suspicious if they are to be good at their job. It is not just that people sometimes lie but even well-intentioned people sometimes make mistakes. From experience detectives have learnt it is best to take everything with a grain of salt, at least initially. Listen to what people have to say; see what 'evidence' you can find; but decide whom to trust and what evidence to take as genuine only after the investigation, not before.
Should detectives trust direct experience?
To a lot of people direct experience is more certain than conclusions arrived at by following clues. To them inferring that the butler is innocent can never be more certain than actually seeing the butler shoot the victim. To detectives the reverse is the case: direct experience is open to doubt, which can only be resolved or reduced by following clues. Yes, the butler was shooting at the victim but clues tell us the victim was dead before he was shot.
Perhaps direct experience can be trusted when the situation is ideal?
Maybe——but, surprisingly, we do not know. We have never been in an ideal situation! In an ideal situation we should be able to prevent all mistakes from arising. But we do not know all the ways in which mistakes could occur, so we cannot prevent all mistakes from arising. Perhaps God knows directly and unerringly but human beings are never in a situation in which their knowledge is such that it cannot possibly contain any mistakes. If we are to rule out all possibilities of mistakes, we will have to know all the ways in which they could be made. But how can we do this? Human knowledge is limited, including knowledge of the ways in which mistakes could occur.
Sometimes people say, I have taken all precautions to prevent mistakes from arising; I cannot imagine what else I could have done. But we all know the answer to this: imagination cannot be used as a yardstick. For a long, long time people could not imagine how human beings could talk to each other over great distances; this nowadays is common occurrence.
If even direct experience cannot be trusted how can detectives know anything?
If even direct experience cannot be trusted, many will draw the conclusion that neither detectives nor anyone else can know anything. For in such a situation, they will ask, whatever we say we know we will have no way of confirming it. We will have no way because we cannot compare what we say to the thing itself. If I say the tower is round but can never compare what I say to the tower itself (because my experience of the tower cannot be trusted), I will never know whether the tower is round. Not only this, but it should follow from the same eternal lack of comparison that I should not be able to tell whether my claim is close to the truth. If I say the tower is round but the tower is only approximately round and I can make the comparison, I will be able to say I am close to being right even though not exactly so. But if I cannot make the comparison, I will not be able to tell.
Now what I have just explained is the common view. According to this common view, if direct experience cannot be trusted we should not be able to know anything. According to this common view also, if direct experience cannot be trusted we should not be able to tell if we are coming closer to the truth or moving farther away from it. Now the question we want to ask is, should detectives agree with this common view?
Here we come to an interesting point. Obviously detectives cannot agree with this common view. They cannot agree with this common view and remain detectives. Detectives follow clues because they want to know. They have learnt from experience that direct experience cannot be trusted but obviously they believe it is nevertheless possible to uncover the truth, if not completely then at least come closer and closer to it. If they do not even believe they can approximate to the truth they should not remain detectives.
But how can this be? How can detectives believe they can approximate to the truth if they can never compare what they say with the real thing? Suppose they say the crime occurred in such and such a fashion. Suppose they have rejected the confessions from the criminals on the ground that they are false. Suppose also that the eyewitnesses contradict each other. Now in such circumstances how do the detectives know what they say is true, or even close to the truth? How can they know when they cannot compare what they say to the real thing? that is, to what actually happened.
Here we see one of the reasons why we need to have the Art of Detection explained. For, if detectives know what happened, and they do, they must know without having made the kind of comparison people usually demand. They can know without comparing what they say to the real thing. This may sound mysterious but once the explanation is provided everyone will say it is common sense. Needless to say, the reason detectives can know is that they follow clues ...
What are clues? How come they can lead us to knowledge of things hidden?
If you have a chance to look into them you will find that clues are simply the characteristics of structures. That is, the characteristics of those structures that we want to know. This is why when we have enough of them we can reconstruct these structures. Knowing by following clues is a kind of reconstruction. As is well known, when we have solved a crime we can reconstruct what happened. The reconstruction will be successful if it, and it alone, leaves behind the same clues as the real crime.
From what has just been said we see why when we follow clues we do not have to compare what we say to the real thing. The real thing is hidden. Because it is hidden we cannot compare our reconstruction to it. But we can compare the clues of the reconstruction to the clues of the real thing. If we have a large number of clues from both and they correspond, we know then our reconstruction is successful. In practice it is hard to have a large number of clues, so we will have to settle for sufficient. How many is sufficient? It depends on the degree of accuracy we want. The higher the accuracy, the larger the number of clues we need.
Clues are the characteristics of structures. It is because they are, that when we have enough clues we can reconstruct the structures of which they are the clues. And this is how we find out things hidden. We find out by engaging in reconstruction. Detectives in solving a crime will try to reconstruct the crime in such a way that the reconstruction and the real crime will leave behind the same clues. For this purpose they need many clues. For if they have only a few, there will be many possible reconstructions that will leave behind the same few clues so that you will not know which corresponds to the real crime. But the more clues you have the fewer the number of possible reconstructions. In solving crimes we may not be able to rule out all possible reconstructions except one, but if all the reconstructions achieved point to the same individuals being responsible, that will usually be regarded as sufficient.
We
follow clues to reconstruct a crime; we follow clues also in re-constructing
those laws governing the workings of nature. Why did
How did you come to this idea that clues are the characteristics of structures?
It came to me while I was examining a simple example in cryptanalysis.
What is cryptanalysis?
Cryptanalysis is simply the technical term for cracking ciphers. Ordinarily, if you have been given a message in cipher, to read it you need the cipher. With the cipher you can decipher the message. But sometimes you can find out what a cipher message says without having been given the cipher. This requires a lot of hard work but it can be done, at least sometimes. And this we call cracking a cipher, or cryptanalysis.
In trying to crack a cipher we of course will have to follow clues. Let us illustrate this with a simple example.
Example in cryptanalysis:
Here is a secret message (also called ciphertext or cryptogram):
SBR SBCTU DBCKERVS FCGG WTTCXR SFH FRRJD YTHE SHUWI
In this example we are asked to find out what the message says without having been given the cipher.
Since the cipher is not given we will have to engage in cryptanalysis. And of course one of the first things we do is to look for clues.
Where are the clues?
There are a number as we can easily see. For the moment let us focus on the simplest: SB, which occurs at the beginning of the first word and also at the beginning of the second.
SB, everyone will say, is a clue; why? Why do we regard SB as a clue? A clue stands for something; it has a meaning. What does SB stand for?
One possibility is that SB stands for TH.
Why? Why would we think this way? Why would we say SB could mean TH?
The answer to this question is simple, as anyone can see. It is a characteristic of the English language that many words start with TH ...
So this is where I find the answer to the question what are clues. The English language has a structure. It is a characteristic of the English language that many words begin with TH. Because the English language has a structure SB is a clue: SB could stand for TH. Now there could be other interpretations of SB; this is the way clues are; they are not always definite as to their meaning. But although SB could have other meanings SB cannot mean just anything; and whatever meaning SB actually has, its meaning is a characteristic of the English language. The English language has a structure. Because it has a structure this English message we are analysing leaves behind clues. Clues therefore are the characteristics of structures. English has a structure, a crime has a structure; so not only English messages leave behind clues, but also crimes. Things that happen during a crime do not happen in just any old way. When a gun fires there will be gunpowder residues. These residues can tell us who was holding the gun even though we were not there to witness the shooting.
Clues are the characteristics of structures. I arrived at this conclusion by looking at SB in our cryptanalytic example. In our cryptanalytic example, if we had finished cracking the cipher we would have re-constructed an English sentence that would leave behind the same clues as those we actually find.
Why are some clues more significant than others?
Clues are the characteristics of structures. Some characteristics of a structure can be found in a large number of other structures, some in a few. Those common to a few are the more significant clues. They give us more useful information as to what structures give rise to them. Knowing that a word contains the letter E does not tell us much as to what this word is since a large number of English words contain the letter E. But if we know that the word begins with the letter X, this is a much more significant clue since very English words begin with the letter X.
Some clues are unique to a particular structure. These are the best to have since they identify the structure unambiguously. Detectives look for fingerprints because no two individuals have the same set of fingerprints (DNA samples nowadays serve the same purpose). Chemists perform flame tests because some chemical elements produce each its own characteristic colour when heated in a flame.
Should we discount insignificant clues?
Most people have two arms and two legs. Knowing that the person who committed a murder has two arms and two legs does not help us much in catching him/her. But this is a characteristic of the crime. The crime is committed by a person with two arms and two legs. So that the murderer has two arms and two legs is a clue but an insignificant one.
Is it so insignificant that we can discount it altogether?
I think the proper advice is that we should not discount even insignificant clues. An ordinarily insignificant clue can become significant in special circumstances. Consider the following. A crime has been committed. After an extensive investigation the suspect has been narrowed down to one twin out of an identical pair. Now obviously this is a good development: there are no other suspects; only one person out of two is the guilty party. But now we face the difficult question, how are we to recognize the guilty twin?
Fortunately for us in this case, this difficult question happens not to be difficult at all. The guilty twin is the one with two arms and two legs.
Two arms and two legs! How could such a clue be useful? Most people have two arms and two legs. What use is it telling us that the suspect in this case has two arms and two legs?
Ordinarily, having two arms and two legs is an insignificant clue but in this case it has become highly significant. Why? Because the innocent twin has lost one arm and one leg in a car accident some years ago.
Even insignificant clues should not be discounted. We may not want to go out of our way to collect them but if they are already ‘on file’, we should keep them on file. You never know, they may be useful one day.
Why is it that an ordinarily insignificant clue can sometimes become significant?
This comes about because the clue-following process is a narrowing-down process, as is well known. In catching a criminal, for example, we first cast the net wide to make sure the culprit falls within the net. Then we gradually tighten the noose until eventually the criminal is caught. In the earlier stages of an investigation an insignificant clue is of little use because it will not help us much in tightening the noose. Knowing that the murderer has two arms and two legs still leaves us with too large a number of suspects. Instead of using this clue, therefore, we will look for some other, better, clues. But in the later stages, when there are very few candidates left, an ordinarily insignificant clue can sometimes become significant by helping us sift out a large proportion of the remaining candidates. When there are only two suspects left and we know that the innocent person has only one arm and one leg, then the other suspect——with two arms and two legs——must be the guilty party.
Why are clues so hard to find?
Clues are the characteristics of structures, not as they normally appear, but to different degrees disguised. In our cryptanalytic example TH does not appear as TH, but as SB. TH has been disguised, which is why we do not recognize it readily. In this case the disguise is somewhat light so that with a bit of effort we can have it removed. In other cases the disguise can be heavy and removing it could take a great deal of ingenuity.
One way in which clues can disguise themselves is worth remarking upon. Sometimes a criminal, after committing the crime, pretends to be one of the by-standers milling around at the crime scene. By putting on this particular disguise they sometimes succeed in melting away undetected. Now clues can be like that too. Sometimes they are right before our eyes but by mixing in with the crowd they escape detection.
When we are looking for clues we are looking for clues relevant to the investigation we are conducting at the time. But it is not always easy to tell to which investigation a clue is relevant. It was known for a long time that the brightness of the planet Venus varies. Few people before Copernicus suspected this was a clue to the motion of Venus in relation to the sun (and thus in relation to the earth).
Clues have a reputation of being hard to find. This comes about not because they are all hard to find but because we tend to forget the easier cases. When we are correcting typographically mistakes we are in fact depending on context for clues——to tell us where the mistakes are and how to correct them. But because the task is relatively easy no one would make a big show out of it. Indeed, in cases such as this we often are not conscious that we are following clues at all; we simply do what comes naturally.
When you have broken a cipher do you have to check with your opponents to determine whether your solution is correct?
No. Why should they help you?
So coherence is the criterion of truth?
No. In following clues we do not use a criterion of truth. This will surprise a lot of people, so I should explain.
When we have broken a cipher we can tell on our own that our solution is correct; we do not have to check with our opponents (if we had to, there would be no point in solving he cipher). Similarly when we are solving crimes: if we are successful, we do not have to ask the criminals to confess. In the case of science, we do not ask God. However, that we can do all this does not mean we are using coherence as a criterion of truth. In following clues, we engage in reconstruction based on clues. To determine success we do not look to the reconstruction to see whether it is coherent, but whether it is based on sufficient clues. Because in simple cases messages deciphered through cryptanalysis are coherent, it is easy to be lulled into thinking that in searching for the truth, when we arrive at results that are coherent, these results must be right (true or close to being true). This is a mistake. As illustration consider the following partial decipherment.
MEET ME AT THE T?P
In this message all the letters have been deciphered except one, represented by the question mark. There are at least two ways to decipher this last letter. The question mark can be translated into I or O, producing the message MEET ME AT THE TIP or MEET ME AT THE TOP. Both messages are coherent. Are we to say they are both true?
No; no one will say they are both correct; rather, they will say in this case we do not have enough clues. We need more clues to determine which of the two decipherments is correct, if either. It is possible that additional clues will tell us that both decipherments are wrong: the last word in this message may not be a word at all but an abbreviation.
In following clues, we do not use coherence as a criterion of truth. It is not that coherence is not important but we do not determine whether our results are correct simply by seeing whether they are coherent. A coherent set of results could be wrong. In following clues it is the clues themselves which tell us whether our results are correct. Clues are the characteristics of structures. In following clues the more clues we have the more accurate our reconstruction. Success in following clues is not determined by anything as simple as a criterion of truth. If we want to find out whether our reconstruction is successful we have to examine how it is done.
If coherence is not the criterion of truth, what is?
We do not need a criterion of truth when following clues. In following clues coherence is not the criterion of truth, neither is anything else. In following clues we are trying to reconstruct those things we want to know but which are hidden from us. Success in this enterprise depends on whether we have enough clues. Generally speaking, the larger the number of clues, the more accurate the reconstruction (provided, of course, we are doing things properly).
How do you know your reconstruction correspond to the real thing?
You know when your reconstruction and the real thing produce the same clues, that is, have the same characteristics. Reconstructing a cipher or a message is no different from reconstructing an ancient sailing ship based on clues and hints gathered from different sources. If your reconstruction has all the many characteristics as they are described in these sources, you are likely to have succeeded.
Is it not possible for two different reconstructions to produce the same clues?
If by 'the same clues' you mean the same clues that have been found so far, then yes, it is possible. But this only means you need more clues. In our example earlier the clues we have, produce two messages: MEET ME AT THE TIP and MEET ME AT THE TOP. Both are compatible with the clues so far available. Which of these two decipherments is correct, if either? We can only determine if we have more clues.
Is it possible for two wildly different reconstructions to produce the same large number of clues?
In following clues we distinguish between those that are significant and those that are less so. A significant clue is a characteristic very few structures possess; it is special to those few structures and, because of this, helps us in identifying them. An insignificant clue is a characteristic found in many structures. Because the characteristic is common, it is not of much use in helping us identify structures. When we look for clues we try to look for significant clues. In an investigation, by constantly looking for significant clues we narrow down the possibilities as to what kind of structure we should reconstruct. The larger the number of significant clues we find, the narrower the field of possibilities. This is the whole point in looking for clues. The ideal is that if we have a sufficiently large number of significant clues, the number of possibilities will be narrowed down to one.
In theory it is possible to have two wildly different structures with a large number of characteristics in common but these characteristics will not be translated into significant clues for the very reason that they can be found in these two wildly different structures. Since in an investigation we constantly look for significant clues we have precluded ourselves from mistaking one of these wildly different structures for the one we are trying to reconstruct. To make this point clearer let us use an analogy. If we knock on the wall of a skyscraper it will make a sound. If we knock on the hull of an ancient sailing ship the same thing will happen. But if we are trying to reconstruct an ancient sailing ship we will not pay attention to the large number of trivial characteristics like this one, that is, that it will make a sound when you knock on it. Instead we will want to know what those characteristics are that are special to this ancient sailing ship, characteristics which are not found in too many other things. And because this is what we will do we will not end up with a skyscraper instead of the ancient sailing ship.
What is the Small Steps Principle?
In an investigation to reconstruct the structures we want to know we need many clues. This means, proportionately, each clue tells us only a little. In an investigation our steps forward are guided by clues. Since each clue tells us only a little, our steps forward are necessarily small steps, never huge leaps. So this is the Small Steps Principle: In an investigation we always take small steps, never gigantic jumps.
The Small Steps Principle is useful for detecting counterfeit investigations. If a person claims she has carried out an investigation but in this so-called investigation she is taking huge leaps after huge leaps, the so-called investigation is a counterfeit; she is not following clues whatever she claims.
The Small Steps Principle is also useful for reminding all investigators, experienced and not-so-experienced, not to be carried away. An investigation is trying on our patience because it requires us to take small steps; it is therefore easy to depart from the straight and narrow. We meet with a few minor successes and then we let these successes get to our head: instead of taking small steps we start making gigantic leaps. At such times it is useful to remind ourselves, or have other people to remind us, of the Small Steps Principle.
The name, the Small Steps Principle, is my invention but the content of the principle is well known to most people who have had some experience with clues. A common objection we often hear when discussing clues is that so and so is taking too huge a leap. In our SBR example, it is legitimate to hypothesize that SB could stand for TH. But if we say from SB alone we can discover the rest of the message, it is too large a leap.
In an investigation it is an important question how much each clue can tell us. If we take a clue to mean more than it can legitimately mean we are taking an unjustifiable leap.
Can we take just anything to be a clue?
We cannot. When we take something to be a clue we have to explain why. Explaining what a clue means is an important part of an investigation. Moreover, if what we take to be a clue is in fact not a clue, we pay for our mistake. When following clues we don’t want to waste time and effort pursuing false clues. False peter out; they lead nowhere.
How do we know what a clue means? Where does this knowledge come from?
Knowledge about a crime, knowledge about some structure in the world——knowledge of this kind we can arrive at by following clues. But what about knowledge of what a clue means. If a clue is about things we already know, there is no mystery. We know English, so we can figure out that SB could stand for TH. But what if we don’t know English? What if the clue is about something we don’t know, something we have never come across? How could we ever figure out the meaning of such a clue?
To answer this question I would like to retell my story about coincidental co-invention. This story is about you, so don’t be surprised if you know it already.
The Story of Coincidental
Co-invention
You are a cryptanalyst, as you already know. But in addition to cracking ciphers you also make up ciphers; that is to say, you are also a cryptographer. For some time now, as a cryptographer you have been working on a new cipher you have invented, a cipher about which you are rather proud. This new cipher is desirable in a number of ways: it is to use, hard to crack, and totally unlike any other cipher known. This new cipher is hard to crack but not unbreakable. This is a trade-off you found you had to make; you could have designed a cipher that is unbreakable but it would not have been easy to use.
Your cipher is unlike any cipher already known. This is an advantage. People who want to crack your cipher will try out different kinds to see to which yours belongs. They cannot try out a kind they do not know. Known ciphers have known characteristics. Your cipher, being new, will have characteristics not familiar to those who want to crack your cipher; they therefore will have a hard time finding clues.
But you know what kind of clues your cipher will leave behind. This is one of the duties of a good cryptographer. All good cryptographers will study carefully the ciphers they have created in order to find out their weaknesses. They have to do this if they are to know to what extent they can trust their ciphers. If their cipher can be broken in one day, they do not want to make the mistake of thinking it needs two.
Tomorrow is the big day. Tomorrow you are going to put your cipher 'in the field'; you are actually going to use it. But a little while ago you have intercepted a message, encrypted of course, sent out by your enemies to their own people. You have tried to decipher this message (through cryptanalysis) but after many attempts you still have not succeeded. You have tried all the different kinds of ciphers your enemies are likely to use; you have found nothing; no clues have turned up. At this very minute you are still staring at this undeciphered message ...
And then, suddenly, to your horror, you start to notice in this message, in this intercepted message, first one, then a second, then a third, then more and more, characteristics of a kind that you would never dream of finding: characteristics of the kind that your own new cipher would generate!
What has happened?
What has happened is that, the coincidence of all coincidences, you and your enemies——independently of each other of course——have invented the same kind of cipher!
Will you use your new
cipher tomorrow?
Not likely. Instead, you will keep very, very quiet and break theirs.
End of Story
To make use of a clue we have to know what the clue means. This requires that we know in advance the structure from which the clue originates. What do we do when we not know in advance? What do we do when the clue comes from some structure we have never been acquainted with? This appears like a difficult question but our story about coincidental co-invention has provided us with the answer. Our story tells us that when a clue comes from a structure we don’t know we will have to re-invent this structure. Normally, this will involve a lot of trial-and-error. Even then success is not guaranteed. In our story we manage to avoid the trial-and-error because by the rarest of coincidences both parties have invented the same cipher.
What one man can invent, another can discover.
——Sherlock
Holmes in
The Adventure
of the Dancing Men
Are there unbreakable ciphers?
There are unbreakable ciphers. The idea behind them is quite simple. To crack a cipher you need clues, and enough of them. If there are no clues, or if there are not enough of them, you cannot crack the cipher. So, if you want your cipher to be unbreakable you can simply do the following: you keep your messages short and you never use the same cipher twice. By doing both you satisfy this requirement; that is, you can be sure there will never be enough clues. This is the idea behind the Onetime Pad, an unbreakable cipher that has been in use for centuries. By keeping your messages short, by changing your cipher after every short message, you prevent clues from arising. Without clues, your ciphers cannot be broken.
With the Onetime Pad, because you have to change cipher after every short message, you need a large number of ciphers in reserve to take care of future needs. In the old days, these ciphers are written on a pad, one cipher to each page. Once a cipher has been used, the page is torn off and destroyed to ensure the cipher will never be used again; hence the name. Needless to say, the ciphers on a Onetime Pad have to be chosen at random. If not, clues will arise in the long term.
What is the difference between theoretically unbreakable ciphers and practically unbreakable ciphers?
A theoretically unbreakable cipher leaves behind no clue whatsoever. Since there are no clues the cipher is unbreakable however hard you try.
A practically unbreakable cipher leaves behind clues. Theoretically, therefore, it is breakable. However, to actually break it will require so many steps that no one in practice will bother because they simply will not have the time. Many of the ciphers used on the Internet are theoretically breakable but practically unbreakable. They are practically unbreakable because to break them will take the fastest computer working continuously for thousands of years.
Why are unbreakable ciphers not used more often?
Unbreakable ciphers were not used very often in the past. This was because in the past they were invariably clumsy. In the first place they could not be committed to memory. In the second, both encipherment and decipherment took a long time. But nowadays in addition to human memory we have CDROMs and microchips. Encipherment and decipherment can be carried out using computers, which render these processes almost instantaneous. As a result, the use of unbreakable ciphers is becoming more and more common.
Are there unbreakable ciphers that can be used to encrypt long messages?
We can modify the Onetime Pad to produce a cipher that can be used to encrypt long messages. We carry out the modification in the following way. Instead of changing cipher after every short message, we change cipher after every letter (this means, to encipher THE we will need three different ciphers). This will increase the total number of ciphers we need but so long as we have as many ciphers as there are letters in the message, the message can be enciphered, however long.
What would a cryptogram look like, one that is generated by an unbreakable cipher?
When a long message has been encrypted by an unbreakable cipher it will appear as a random series of symbols. This comes about because the series of ciphers used to encrypt the message is itself random. If it is not, in the long terms clues will arise.
Are there things we cannot know?
It is possible for there to be things so well hidden from us that we cannot know. A cryptogram generated by an unbreakable cipher is no different from a random series of symbols. When we meet with a random series of symbols it is possible that it is just a random series of symbols or there is a message behind it; which it is, we cannot tell.5
How can we ever have enough clues?
This is an important question. We need clues to reconstruct those structures we want to know and to distinguish it from neighbouring structures. But clues are hard to find; how can we ever have enough to finish the reconstruction? At the beginning of an investigation there are usually very few clues. If we are confined to these few we will never find the things we want to know. Finding these few is hard enough; how can we find the rest?
This sounds like a difficult question but there is an answer to it, as those familiar with investigations will know. In an investigation we can develop new clues from old, at least sometimes. Because of this we can often, or often enough, bring an investigation to an acceptable conclusion even though we have only a very few clues to begin with.
How do we develop new clues from old?
To see how new clues can be developed from old let us turn again to our cryptanalytic example. Below I show the first three steps in cracking this cipher.
Notice that in Steps 1 and 2 we are relying on clues already present from the beginning. In Step 3, however, the two clues we use are new; they were not there at the beginning; they first appear in Step 2. In Step 3 itself more new clues appear, as readers can easily see.
In simple investigations developing new clues from old is almost automatic. In our cryptanalytic example once we have some idea what a letter might stand for we immediately translate all instances of that letter. We think S might stand for T, so we translate all the Ss to Ts. Now when we keep doing this, that is, when we keep making this kind of translation, from time to time new clues appear.
But while developing new clues from old is easy in simple investigations it may not be so simple in more complicated investigations. For example, how do we develop new clues from old when solving crimes? or when carrying out investigations in science?
To answer this question we look again to our example. There we have a cryptogram:
SBR SBCTU DBCKERVS FCGG WTTCXR SFH FRRJD YTHE SHUWI
We look for clues in this cryptogram. Basing ourselves on clues found we form hypotheses, such as S = T. Then we apply these hypotheses to the cryptogram and in doing so produce partial decipherments. In the process new clues appear. Now ask ourselves this question: instead of calling the cryptogram, cryptogram, what other term would we use if this had been a different kind of investigation? Let me put the question in a different way. To solve the cipher we need a cryptogram. If we were solving a crime, what do we need? In solving a crime what will perform the same function as the cryptogram in cryptanalysis?
Once the question is put this way it will not take us long to arrive at the answer that the term we are looking for is 'evidence'. To solve a crime; to carry out any kind of investigation; we need evidence. In solving a cipher the cryptogram is the evidence. This evidence leads us to the conclusion that the hidden message is such and such.
So in general this is what we do when we want to develop new clues from old: We apply the hypotheses we have formed to as much of the evidence as we can. In the process new clues might appear. When there are new clues we can form more hypotheses, thus gaining a better and better idea of those things we want to know. And these new hypotheses, we apply them to the evidence also, leading to even more new clues. We see here there are feedback loops when following clues. It is these feedback loops that produce the large number of clues we need if the investigation is to be successful. Notice that these feedback loops are positive: clues lead to more clues, the reason why by following clues knowledge can expand.
Why is it that new clues can appear?
New clues do not appear by fiat; we cannot command them to appear. If we want new clues to appear we have to apply the hypotheses we have formed to as much of the evidence as we can. Even then they only sometimes appear. Why should things happen this way? Why should new clues appear at all? This is the question we now want to answer.
The question sounds difficult but the answer is actually quite simple. Clues are the characteristics of structures. Structures are things that behave in more or less predictable ways. If you have an English word that contains only three letters and the first two are TW, the third has to be (is likely to be) O; it cannot be just any letter or just anything. Now when we apply hypotheses to the evidence and these hypotheses are correct, we will be find out more and more of the structures we want to know. The more we know, the easier it is to find out the rest. If we do not know that the first two letters are TW we will not know that the third is O. But if we do know that the first two letters are TW, we have a clue to the third. This is why by applying hypotheses to the evidence we can develop new clues. Clues can be looked upon as small gaps in our knowledge. At the beginning of an investigation when little is known, we have huge gaps all over the place. As we find out more; as we fill in some of these huge gaps by putting forward the right hypotheses; smaller gaps appear. When a gap is small enough we have a new clue.
What if the hypotheses are wrong?
This is an important question. To answer it, let us first retrace our steps.
In an investigation we put forward hypotheses in response to clues. We apply theses hypotheses to the evidence. If these hypotheses are correct we will know more of the structures we want to know. The more such partial knowledge we gain, the easier it is to develop new clues. Now the question is, what if the hypotheses are wrong?
Having retrace our steps, we see what the answer has to be. If the hypotheses are wrong we cannot develop new clues. For, new clues appear when our partial knowledge is increasing. But our partial knowledge cannot increase if our hypotheses are wrong. If the first two letters are TW but because of some wrong hypotheses we have translated them to CM, we will not have in CM a clue to the third letter.6
So the answer to the question is simple. If the hypotheses are wrong we cannot develop new clues.
But although the answer is simple, it has important consequences, as we can see in the answer to the next question.
What does it mean if new clues keep appearing?
In an investigation if new clues keep appearing it must mean we are on the right track. New clues do not appear for no reason. As we have explained above, they appear only when we find out more and more of the things we want to know. If our hypotheses had been wrong new clues could not appear. TW? is a clue but not CM?.
When new clues keep appearing investigators rejoice. They may yet have a lot to find out but the fact that new clues keep appearing tells them they are making progress. Even though they do not know what the final result will be, they know they are heading in the right direction.
What is the Right Direction Catechism?
This is the Right Direction Catechism:
How
do you know you are heading in the right direction?
When new clues keep appearing.
How can we know we are heading in the right direction when we do not yet know what our destination is?
In
an investigation we know we are heading in the right direction when new clues
keep appearing. This of course is no news to any one familiar with following
clues, But to someone who do not understand the Art of Detection it could be
puzzling. When we travel by car we have to know what our destination is if we are
to tell whether we are moving in the right direction. Suppose you have just
left
But when we are following clues we can tell whether we are moving in the right direction without knowing our destination. Sherlock Holmes can tell he is on the right track even when he does not know yet who the murderer is. In cracking a cipher we can tell we are moving in the right direction long before we know what the message says. Scientists can tell they are making progress in their investigations without knowing what they are progressing towards.
How do we tell we are moving in the right direction when following clues? We tell by whether we are developing new clues from old. When new clues appear after old clues have been interpreted we know we are moving in the right direction. If wrong hypotheses had been offered in response to old clues, new clues could not have appeared.
When travelling by car it is irrational to say we are moving in the right direction without knowing the destination. When following clues it is not irrational. We must therefore not compare following clues to travelling by car. The two kinds of activity are not just different, but very different.
What is the Meno Paradox?
The Meno Paradox is a simple argument purporting to show it is impossible to look for knowledge. It takes its name from Plato's dialogue Meno, in which it is reported. It goes as follows:
When we are looking for knowledge we look either for knowledge we already have or for knowledge we do not yet have. There is no point in looking for knowledge we already have, since we already have it. And it is impossible to look for knowledge we do not yet have. For knowledge we do not yet have is about things we do not know. But how can we look for things we do not know? In what direction should we turn? And if by chance we should have found them, how do we know we have?
The Meno Paradox says we cannot look for knowledge we do not yet have because we will not know what to do. And even if we should find it, we would not know that we have since we did not know what we were looking for in the first place. But this argument is fallacious (in the Meno Socrates calls it a trick argument). It has to be, since by practising the Art of Detection we do from time to time find out things we originally did not know.
Why is the argument fallacious? Where is the mistake? This we can answer now that we have some understanding of the Art of Detection. The Meno Paradox, we can now see, makes the mistake of thinking that looking for knowledge is like travelling by car. When travelling by car, if we do not know where we are going we will not know what to do——for example, in what direction we should drive. The same with looking for knowledge we do not yet have, the Meno Paradox says. Because the destination in this case is something we do not yet know, we again will not know what to do. Also when travelling by car, if we do not know where we are going, even if by chance we have reached our destination, we will not know that we have. The same with looking for knowledge we do not yet have according to the Meno Paradox. Even if we should have found this knowledge, we will not know that we have because we are ignorant of it right from the start. But this argument, we can now see, is a mistake. Looking for knowledge we do not yet have is not like travelling by car (see answer to this earlier question). To look for knowledge we do not yet have we have to follow clues. When we are following clues we can tell we are closing in on the things we want to know when we develop new clues. In following clues we can tell we are making progress and moving in the right direction without first knowing the things we will eventually find out.
Because the Art of Detection is not always well understood, confusions frequently arise. The Meno Paradox is one of these confusions. It derives its persuasive force from a common mistake that anyone can make, that of assuming that following clues is like travelling by car. If one makes this false assumption the conclusion follows that it is impossible to look for knowledge we do not yet have. If one does not make this false assumption; if instead, one pays attention to the Art of Detection; it is obvious (as obvious as anything can be obvious) that it is possible to look for knowledge we do not yet have.
What happens in an investigation after serious mistakes have been made?
In an investigation new clues can appear as our partial knowledge increases. But when we make serious mistakes our partial knowledge cannot increase. This is to say, when we make serious mistakes no new clues can appear. When there are no new clues, there can be no progress in the investigation after old clues have been used up. After we have made serious mistakes, therefore, the investigation will come to a halt (usually soon); or as we sometimes say, we will hit a brick wall.
What is the Brick Wall Catechism?
This is the Brick Wall Catechism:
What
do you do after you’ve hit a brick wall?
Try to find out what serious mistakes you have made.
What is the Open Secret?
An open secret is a valuable piece of information which everybody knows but nobody mentions. People used to following clues share an Open Secret: Follow clues and develop new clues from old.
This is an important piece of information because in an investigation, if we do not follow clues or if we follow clues for a while and then, out of impatience, stop and start making wild guesses, we will never find the things we want to know. But it is not sufficient that we follow clues, we also have to develop new clues from old. When new clues appear, and keep appearing, we know we are moving in the right direction. Without these new clues, we will not know where we are.
Can we avoid making assumptions when carrying out investigations?
No, we cannot. In an investigation we look for clues and we try to make sense of them, but whenever we are looking for clues——especially when we are trying to make sense of them——we have to make assumptions. In our cryptanalytic example, if we had not assumed that the plaintext was in English we would not have suspected that SB might stand for TH.
What do assumptions do?
The assumptions in an investigation tell us the general area in which we are to look for the things we want to know. In an investigation we are looking for things hidden. But there are many things hidden; we are not looking for all of them, only some of them; what they are exactly, we do not yet know; still, we cannot roam all over the place; we have to concentrate on a manageable area so that we can have some chance of success.
What if the assumptions are wrong?
If the assumptions are wrong; if they are very far from the truth; we will not find anything. When the assumptions in an investigation are very far from the truth, we may miss all the clues. Even if we do not; even if we notice a few; we will interpret them in the wrong way. Wrong assumptions cannot sustain an investigation; they will lead to a dead-end, usually soon.
Why will the interpretation of clues be wrong after we have made wrong assumptions?
Clues are the characteristics of structures. We cannot take the characteristics of one structure and impose it on another. The frequent occurrence of TH is a characteristic of the English language. If a message is in Chinese and we have mistaken it for English, however hard we try we will not find any TH in it. If we interpret any clue as meaning TH we will simply be wrong.
Do assumptions have to be exactly right?
The assumptions we make in an investigation do not have to be exactly right; they need only to be sufficiently close to being exactly right for the investigation to advance. As the investigation advances; as we know more; the assumptions can then be brought closer to the truth. Suppose in deciphering a message we have made the assumption it is in English. By this, of course, we mean ordinary English. But the message is not in ordinary English; it is in Newfoundland English. In a strict sense our assumption that it is in ordinary English is false. Will this prevent us from deciphering the message? Clearly, no. Because Newfoundland English is close to ordinary English we should be able to decipher large parts of the message under the false assumption. After this is done, context will enable us to decipher the rest in addition to telling us that we are not dealing with ordinary English here, but one of its variants.
In deductive reasoning any set of premises, true or false, can lead to an infinite number of conclusions; why can’t false assumptions in an investigation lead to an infinite number of false results?
In deductive reasoning there is no need to follow clues but in an investigation we have to. In an investigation, when the assumptions are far from the truth they will shield us away from the clues or lead us to interpret them in the wrong way. Wrong interpretations will not lead to new clues. Without clues, new or otherwise, the investigation cannot advance, which is to say, no further results can be obtained.
We can be misled by an investigation based on wrong assumptions; no?
If by ‘wrong assumptions’ we mean ‘assumptions far from the truth’, no; not if we are careful. In an investigation based on assumptions far from the truth we will not be able to detect many clues. The few we can detect, if we can detect any at all, will be interpreted in the wrong way. But an investigation cannot be completed with just one or two clues. An investigation based on assumptions far from the truth, therefore, will soon come to a halt and whatever results arrived at up to that point will be unreliable.
However, people often say they are following clues when they are not. They may even deceive themselves into thinking they are following clues when in fact they are simply making things up. If we take these people at their word then of course we can be misled. This is to say, if we do not pay attention to how an investigation is carried out, whether it is guided by clues and whether new clues have been developed, yes we can be misled by an investigation based on assumptions far from the truth. But if we are careful; if we can examine how the investigation was carried out; it will not be so easy.
How do we know what assumptions to make?
In stock situations making assumptions is relatively easy. If we have broken a cipher, next time we meet with a cipher of the same or similar kind, we know what assumptions to make. Our assumptions may not be completely right; there may have to be a lot of adjustments later on; but at least we have a start. However, when we are faced with a new kind of investigation, a kind unlike any which we have experienced, deciding on what assumptions to make will not be easy. Very often we are simply stumped as to how we should proceed. But since doing nothing will not produce results, normally there are a few things we can try. It will be seen all these are desperate measures:
1. We do not carry out investigations for no reason. Something must have provoked the investigation, something must have occurred which pique our curiosity. What is this something? Now we must have asked this question before; otherwise we would not have come to the conclusion that the investigation facing us is unlike those we are familiar with. But we should ask this question again in a more searching way, in the hope of finding some resemblance, however slight, between the current investigation and those we have come across in the past. If there is resemblance, it might provide us with some idea as to what assumptions to make. Because investigations are difficult there are frequent occasions for doing things a second or more time to make sure we have not missed out anything. This is one of those occasions.
2. Even in normal investigations a lot is done through intuition. When we are in a desperate situation as in the present case, we may have to put greater faith in intuition after we have tried (1) above and found it does not work.
3. Since intuition can vary from day to day, if we have tried (2) and it does not work on one day, we may want to wait for the next and try again.
4. Intuition also varies from person to person. If our intuition does not help in arriving at some useable assumptions in this difficult investigation, we may want to ask for help from other people and see if their intuition has better luck.
5. As the last of a set of desperate measures, we can try shooting in the dark, that is, throw all caution to the winds and start making guesses. When we do this the chance of success is very low but it is better than doing nothing.[1]
Do we have to be sure our assumptions are right before moving on?
We do not have to be sure our assumptions are right before moving on. In fact it is by moving on that we can find out whether our assumptions are likely to be right. Right assumptions will facilitate an investigation; wrong assumptions will hinder it. If we find new clue after new clue as a result of a particular set of assumptions, that set is likely to be right.
What are hidden assumptions?
In investigations we often form assumptions without ourselves knowing we have formed them. These assumptions are called hidden assumptions.
There is a common warning about hidden assumptions; is there not?
Yes there is. We should beware of hidden assumptions because they could be wrong. And since wrong assumptions could lead to impasse in an investigation, when we find an investigation particularly difficult to move ahead we should consider the possibility that some hidden assumptions are wrong.
How do we know an assumption is wrong, hidden or otherwise?
Wrong assumptions lead to impasse in an investigation. One way to find out whether an assumption is wrong, therefore, is to replace it. If after the replacement the investigation is able to move forward, the replaced assumption is wrong.
What is the ideal way for an investigation to begin?
The way we began our investigation in cracking the SBR cryptogram is a good candidate for the best way for an investigation to begin. There we are told the aim of the investigation (find the secret message), there we are given sufficient evidence to arrive at a satisfactory conclusion, so that all we have to do is make assumptions and follow the clues.
Can we be sure we are not looking for shadows in an investigation?
In some investigations it is possible that there is nothing to be found of the kind that we expect to find. We could have started the investigation with nothing more than suspicions. We experience certain things, we suspect there is something behind it, and we want to find out what it is. But is there really something behind it? Is this something of the kind that we expect? This we do not know at the beginning of the investigation and may never know if the investigation leads nowhere.
Can we begin an investigation just with suspicions?
If we do not even suspect, there is no reason to gather evidence and look for clues. We carry out investigations to uncover knowledge we do not yet possess, knowledge of things hidden. If there is nothing hidden; if we do not even suspect that there are things hidden; there will be no reason to carry out investigations.
But it is not sufficient that we suspect there is something hidden; our suspicion has to include some general idea as to what this something is even if this general idea is vague and imprecise. For example, are we looking for some action by the gods or are we looking for a set of natural laws in our attempt to quiet our suspicion? The search for evidence and the search for clues will depend on which we suspect to be the case.
It is permissible to suspect more than one thing before we gather evidence and search for clues. A person has been found dead. Is it murder or is it an arranged suicide? We may not want to rule out either. But we cannot suspect everything under the sun. That would be casting the net too wide, making the gathering of evidence and the search for clues impossible.
Do we have to be absolutely certain of some things before we can begin an investigation?
We do not; in an investigation certainty increases as we advance——that is, as old clues lead to more and more new clues. If as a result of a number of suspicions more and more new clues are found, it is (more and more) likely that the suspicions are well founded.
When an investigation begins with suspicions, are we not prejudging its outcome?
No, we are not prejudging the outcome. In an investigation we need a large number of clues if we are to reconstruct those structures we want to know. We look for evidence so that we can find in it the clues required. If we suspect the wrong things; if for example we suspect murder when in fact it is suicide; whatever we will find when looking for evidence will not contain enough clues to allow us to reconstruct a non-existent murder.
But over and over again people are sent to prison for crimes they did not commit!
This shows how much more careful we should be when carrying out investigations and how important it is that we should understand the Art of Detection. One wrongful conviction is one too many. To prevent wrongful convictions we all need a better understanding of this complex Art. People sometimes think it is a simple business making a decision based on evidence. The Art of Detection tells us it is not. In carrying out an investigation we do not just fit the evidence into a conclusion arrived at beforehand or afterwards. Instead, we have to find clues in the evidence and see where these clues lead. Suppose person P is seen running away from a crime scene. Suppose also that blood stain is found on this person matching the blood type of the victim. Around these two pieces of evidence we weave a story showing that person P is responsible for the crime. Does this mean P is really responsible? The story ‘explains’ why P was running away. The story also ‘explains’ how the blood stain got on P. Is this sufficient? The Art of Detection tells us it is not. The Art of Detection will ask, are there sufficient clues to allow us to reconstruct what happened? Are there sufficient clues to tell us things could not have been otherwise? Following clues is very different from spinning a story to explain some ‘evidence’ (see also answer to What is the Small Steps Principle?).
Why is it better that we collect as much evidence as we can as early as we can?
If resources permit, it is better to collect as much evidence as we can as early as we can. There are a number of reasons:
1. In an investigation we need clues and clues are usually hard to find at the beginning of an investigation. If we have a large body of evidence at this stage it makes the task of finding those clues easier which will set the investigation on its way. With a large body it is easier to notice patterns and unique occurrences, which are often clues. If the body of evidence is too small patterns will not appear and the determination of uniqueness will be impossible for lack of comparison.
2. The evidence we collect sometimes contains errors. For example, an intercepted cryptogram could contain transcription mistakes. To find clues in a body of evidence containing mistakes we need a larger body than normal. But since we do not know how much mistake there is, it is better that we collect more evidence than less.
3. In some investigations evidence not collected in time will deteriorate and even disappear for ever. This often is the case with evidence needed in solving crimes. Archaeological evidence provides us with another example of this sort.
Is it absolutely necessary to collect all the evidence needed in an investigation at the beginning of the investigation?
It is not absolutely necessary. So long as the evidence we have provides us with enough clues to set the investigation going, this for the moment could be sufficient. As we find out more, we will be clearer as to what additional evidence we need. We can look for this additional evidence then, assuming it is still available.
Why is it that in collecting evidence we so often include material irrelevant to the investigation and miss out on relevant material?
In an investigation we like to collect as much evidence as we can as early as we can. But in the early stages of an investigation we know very little. This is why in collecting evidence we so often include material irrelevant to the investigation and miss out on relevant material. The collection of evidence at the early stages of an investigation depends largely on our assumptions and suspicions. Since these tend to be vague and imprecise our judgment as to what is relevant and what is not will not always be as good as we would like it to be.
When evidence is incomplete and contaminated by irrelevant material to boot, as it so often happens at the beginning of an investigation, do we have any hope for a successful outcome in carrying on with the investigation?
Incomplete evidence will not present a major problem so long as we have enough to provide us with sufficient clues to set the investigation going. Clues can lead to new clues. As we know more we will know what additional evidence we will need.
A small amount of contamination by irrelevant material we can deal with. If the amount is small enough it will not cover up all the clues or dilute them to such an extent that they are not recognizable. The contamination will in all likelihood slow down the investigation but not necessarily make it impossible.
Contamination of evidence by irrelevant material is a well-known trick when sending secret messages. A common practice there is to sprinkle nulls randomly all over the message before enciphering it. These nulls are simply extra symbols that have no meaning whatsoever. But when they are added into the message they break up the usual letter sequences and thereby make clues harder to discern. It is well known that nulls slow down cryptanalysis but will not by itself completely prevent it.
However, as contamination increases it may reach a stage when clues can no longer be detected. Without clues, there cannot be progress in the investigation.
A well-known way in which evidence can be contaminated by irrelevant material is the red herring. A red herring is a clue in a different investigation, not the one we are interested in at the moment. A red herring causes confusion from which we may never recover. Even if we do, it will have slowed down our investigation.
Why is it that we can sometimes pin-point the evidence we need but not always?
In the later parts of an investigation when most of the structures we are looking for are known, it is relatively easy to pinpoint the evidence we need so that we can answer the remaining questions we have. But this kind of pinpointing is harder to do in the earlier stages of an investigation because at that time much less is known and the assumptions and suspicions we are working with tend to be vague and imprecise.
Why do we so often fail to recognize evidence?
Correct recognition of evidence depends on what we know. In the early stages of an investigation we know very little. This is when we often fail to recognize evidence that later on we find we need.
Is there anything we can do to prevent evidence from escaping us?
There are well-known measures we can take to minimize the chance of letting relevant material escape us when collecting evidence. These measures are usually taken at the beginning of an investigation since at that stage the chance of letting relevant evidence escape is especially high. The following are some of these measures:
1. The collection of evidence depends on our assumptions and suspicions. To ensure that we will catch most of the evidence we need we make as liberal an interpretation of these assumptions and suspicions as we can. For example, if we suspect murder, do not assume only one person is responsible; allow for the possibility that there could be more than one.
2. Work with more than one set of assumptions and suspicions if necessary. Suppose we suspect murder but there is some slight indication of an arranged suicide. In this case do not just look for evidence for murder but also evidence for suicide.
3. Allow duplication and even over-duplication. Take cracking a cipher for example. If we have some rough idea of the kind of cipher we are trying to crack we can estimate the minimum length of ciphertext we need. But do not collect the minimum length! Collect twice the amount or even many times the amount.
4. Pay attention even to bordering material. When we go to the library with the view of borrowing a certain book, we often look at other books on the same shelf and take home a few. Do the same when gathering evidence. Do not just gather evidence your assumptions and suspicions tell you to gather but pay attention to material bordering it.
5. Listen to your intuition. If intuition tells you a certain piece evidence is relevant, pick it up even if the assumptions and suspicions you have formed tell you otherwise.
If a theory can explain all the evidence available, is it then not the right theory? Why do we need to bother with clues at all?
When we follow clues we have ways of determining what is evidence and what, irrelevant material. If we do not follow clues; if we simply gather evidence and then fashion a theory to explain it we face the following questions (among others):
1. How do we know whether our ‘evidence’ may not also contain irrelevant material? And if it does, how do we separate the relevant from the irrelevant?
2. How do we know we have all the evidence? Is it not possible some has been left out?
3. Suppose more than one theory can be constructed to explain the same body of evidence, how do we determine which, if any, is the right theory?
Our aim in following clues is knowledge, the knowledge of those structures which give rise to the clues. We try to achieve this aim by reconstructing these structures. If we succeed, we should be able to explain what the evidence means——in the same way that we can explain what the cryptogram means in our SBR example. Our primary aim in following clues is knowledge, not the explanation of evidence. If we simply focus on explaining the evidence by some theory, paying no heed whatsoever to clues, we will not achieve our aim. Reconstruction by following clues is a complex process. We cannot replace this complex process by a simpler one that appears to have one or two things in common with it. An analogy may be useful here. A car is a box on four wheels but this does not mean if we succeed in putting a box on four wheels we will have a car. Theories resulting from reconstruction by following clues will explain the evidence but this does not mean that any theory that somehow ‘explains’ the evidence is the right theory.
Why do we need evidence?
We need evidence so that we can find clues and so that we can develop new clues. When we have evidence we can search it for clues. When we find them we propose hypotheses as to what they mean. If these hypotheses are correct, by applying them to the evidence we have a chance of developing new clues. Some of the hypotheses we advance in the course of an investigation we may want to test. For this we also need evidence. In our cryptanalytic example the cryptogram is the evidence. In it we find clues, leading to the proposal of certain hypotheses. By applying these hypotheses to the rest of the cryptogram we develop new clues. Some of the hypotheses we propose can be tested against what we can find in the cryptogram.
Does evidence have to be error-free?
Our job as an investigator will be easier if evidence does not contain any errors but this does not mean it will be impossible if there are; it depends on the severity of the errors and their proportion. A highly garbled message cannot be deciphered through cryptanalysis because there will not be enough useable clues: most of the clues will have been covered up or diluted; but if there is an error here and there success could still be possible. Indeed, after we have reconstructed the cipher it should be easy to point out where the errors are in the cryptogram and correct them if necessary.
When there are errors in the evidence some of the clues will have been covered up or diluted by these errors. As compensation we will need a larger body of evidence, especially at the beginning. This is one of the reasons why when collecting evidence at the beginning of an investigation we would like to collect as much as we can.
When evidence contains errors, will the errors not be transmitted to the conclusion (of the investigation)?
When we are reasoning, errors in the premises can be easily transmitted to the conclusion. But although we reason when carrying out investigations, reasoning is not the only thing we do when engaged in this kind of activity: the Art of Detection is not a branch of Logic; we therefore should not expect an investigation to have the same characteristics as reasoning.
In an investigation evidence does not function in the same way as premises. In an investigation if the proportion of errors is not high, the presence of these errors in the evidence may have no effect on the conclusion: a slightly garbled message can still be broken. If the proportion of errors in the evidence is high——as in a highly garbled message, these errors will not be transmitted to the conclusion. Instead, there will be no conclusion——because there will not be enough clues to allow the investigation to take any meaningful step forward. See also the answer to this question.
Do scientists and detectives believe the senses are infallible?
They are the last people to hold such a belief. From experience, sometimes painful, they know the senses often make mistakes. Indeed, as part of their training, scientists have to know the many ways in which our senses can err so that they can provide for it. As for detectives——have we not heard them ask questions like the following: ‘You thought you heard a gunshot. Are you sure it was not a car backfiring?’
Why do we look for empirical evidence?
Scientists perform experiments and make observations; these provide them with evidence. We call this kind of evidence empirical because they are collected using our senses.
Detectives collect physical evidence and they listen to eyewitnesses. Detectives, like scientists, are also interested in empirical evidence.
Why is there this need for empirical evidence? Why do we search for it when carrying out investigations? Why, when everybody knows——including scientists and detectives——that our senses often make mistakes?
We will do well to pay close attention to the answer to this question. A wrong answer could cause serious trouble in our attempt to understand the Art of Detection. The proper answer to this question, I think, is the following. Investigators look for empirical evidence precisely because they know our senses make mistakes; they know it so well that they can tell you the many, many ways in which this could happen. The best investigators are usually the best critics of the senses. They have to be in order that they can guard against this sort of error and in order that they can make the best use of empirical evidence. The important point to remember here is that in an investigation we do not need error-free evidence. So long as the proportion of errors in the evidence is not excessive success is still possible. If we know how the senses can make mistakes, we can lower the proportion. We may not be able to eliminate every single mistake our senses make but complete elimination is not our aim. Our aim is simply to reduce mistakes to a tolerable level——a level at which we can still detect clues and develop new ones.
Suppose I do not use empirical evidence; suppose I just take anything as ‘evidence’; what will happen?
This is an experiment anyone can try. If in an investigation we just take anything as ‘evidence’, no result can be obtained. This so-called evidence may appear to be relevant but in fact is likely to contain all kinds of mistakes because of the complete lack of discrimination. With such a body of evidence, even if you should find clues, you will not be able to develop new clues.
Suppose I use mostly empirical evidence but in addition, one or two suitably manufactured, artificial pieces; what will happen?
What will happen is that your deception will have a good chance of temporary success if deception is your intention. This in fact is the kind of deception professional perpetrators prefer. Take spies for example. Spies often have to pretend to be somebody they are not; they have to take on an assumed identity. What is the best thing to do when this need arises? Make up an identity from scratch or base it on a real person? Professionals will try the latter when possible. It is hard to make up a history for yourself, which can stand up to investigation. But if you assume the identity of a real person, your story will hold so long as the investigators do not pick up the one or two embarrassing details.
The most common kind of science fraud happens in the same way too. A paper on an experiment never carried out is hard to write and easy to detect; this kind of science fraud does not happen all that often. Experimental results slightly changed here and there are hard to detect but easy to produce; this kind is more frequent.
Yes, we can be misled by manufactured ‘evidence’ sometimes.
But not indefinitely! At least, not in every case. For spies are caught and science frauds are discovered. How is this done? What makes it possible? The answer is well known. In an investigation, whenever we are fed false information and this information matters, eventually the investigation will hit a brick wall. What do we do when we hit a brick wall? We backtrack and try to find out what mistakes we have made.
Manufactured evidence of the kind we are talking about here; manufactured evidence hidden among a large body of genuine, empirical evidence; can escape detection on the first pass, even the second, even the third, …. But for a detective sufficiently persistent, they can be discovered——so says the Art of Detection.
How do we look for additional evidence?
Very often in an investigation the evidence we have initially, although sufficient to set the investigation on its way, may not be sufficient to complete it. When this occurs we will have to look for additional evidence. Take our SBR cryptogram for example. There we have sufficient evidence if our intention is to find out what that particular message says. But if our intention is to find out how the whole cipher works——how all the twenty six letters of the alphabet are translated, we will need additional evidence.
How will we go about looking for this additional evidence? When we are actually engaged in a real investigation this question usually is not hard to answer. In our SBR example, to find out how the rest of the cipher works we need more cryptograms from the same cipher. Where can we find more cryptograms from the same cipher? Obviously, one of the first things we will try is to go back to the same source from which we obtained the original cryptogram. Suppose we obtained that cryptogram by tuning our radio to a certain frequency. To obtain more cryptograms we will again tune our radio to the same frequency.
Suppose this produces a new cryptogram. How do we know this new cryptogram is from the same cipher?
This again is easy to answer. We try to see if the new cryptogram can at least be partially deciphered by the part of the cipher we already know. If it can, then it is from the same cipher.
So in general how do we look for additional evidence? What do we do whenever we need additional evidence?
The right answer to this question, it seems to me, has to be the following. Whenever we need additional evidence, we will make use of whatever we know, or think we know, up to that point. We will make use of this ‘knowledge’ to guide us even though it may contain some highly fallible items. In our example, there is no saying that when we tune our radio to the same frequency we will hear anything. Also, as we have pointed out, even if that radio station is still transmitting they could have changed their cipher. Because the knowledge we rely on when looking for additional evidence is far from perfect, when we look for additional evidence there is always the possibility that we may not find it. But what more can we do?
Does the search for additional evidence depend on feedback?
From our answer to the last question we see that it does. Evidence already available enables us to find out part of the cipher. The partially reconstructed cipher helps us find more evidence by telling us whether the new cryptogram is from the same cipher.
Is this feedback positive or negative?
Clearly it is positive. Evidence leads to more evidence.
Why is it so important that we should notice that there are feedback loops in the clue-following process?
It is important that we should notice not just that there are feedback loops in the clue-following process, but positive feedback loops. The presence of these positive feedback loops explains why so often from a tiny beginning we can eventually find out so much by following clues.
Are positive feedback loops the same as circular reasoning?
No, they are not. Some radio circuits, to enable proper reception, rely on positive feedback to magnify incoming weak signals. Obviously, these radio circuits are not engaging in any kind of reasoning, circular or otherwise. Positive feedback is simply a way of doing things. By channelling part of the output back to the input it magnifies the effect of the original input. In following clues we often have the opportunity of doing exactly this. Evidence, by providing us with clues, can lead to knowledge we originally did not have. This increase in knowledge can in turn lead to the discovery of more evidence, producing even more knowledge.
That evidence can lead to the development of theory which in turn can lead to more evidence favourable to the developed theories and to the development of future theories have been noticed by many writers studying the history of science. But instead of recognising this for what it is——namely, positive feedback, they bring forward the charge of circular reasoning. They say scientists reason in a circle.[2] This charge can only be the result of misunderstanding. Positive feedback loops, although often represented graphically by circles, are not the same as circular reasoning, as we have explained above.
In progress …