False alarms and missed fires

 

When you have to make an either or choice, there are two directions you can be wrong in. Either you think something is the case when it isn’t, or you think something is not the case and it is. The first is a false alarm; the second is a missed fire.

Some specialists talk about signal detection. Signal detection theory is a component of engineering. I learned about it in psychology, however. Then, decades ago, I had no idea why it could be important. It was some theoretical thing that we had to know about. However, over the years I have come to see that this is really a very good way of looking at events.

In statistics and in experimental work they talk about false positives and false negatives. That’s not terribly informative in terms of language. They also talked about type I and type II errors. Again this is not that informative in terms of common language. However, the notions are sound.

Sometimes we miss what’s going on; we don’t see it; we don’t notice it; we mischaracterize it; we miss the boat. On the other hand sometimes we have false alarms and a lot of action is taken that is unnecessary and may be counterproductive.

In signal detection theory things are made very mathematical. People draw graphs; talk about the signal detection curve, the receiver operating characteristic; and undoubtedly, do other things which I’ve long ago forgotten. One thing I do remember is you can make the situation better by achieving by improving the accuracy of the mechanisms making the decisions. This means changing the apparatus that examines the issues, whether machine, detecting device or some other mechanism, which could be your own intuition or procedures. This is a matter of changing the system so it works better.

 

Signal Detection Outcomes

(Contingency Matrix)

We can show these situations for outcomes in the following matrix:

 

Signal Present

Signal Absent

Meets Criterion

true positives (TP)

(hits or detections)

false positives (FP)

(false alarms)

Type I error

Does Not

Meet Criterion

false negatives (FN)

(misses)

Type II error

true negatives (TN)

(correct rejection)

There’s another thing you can do. It just involves changing the bias in the system. You can change the threshold at which you say something has happened. You can make it higher so you’re less likely to say something has happened. You can make it lower so you’re more likely to say something has happened. However, in doing so, without improving the accuracy of the detection mechanism, you increase the number of errors of the other type. So if you improve the bias to reduce false alarms, you also miss more stuff. If you go the other way and improve the bias to detect more things, you will also increase the number of false alarms.

Without improving the capability of the system to determine truth from falsehood, you will change the operating characteristics of the system but you will not improve your overall accuracy. Changing the bias, changing the threshold, changing the triggering conditions does not make for more reliable choice-making or detection of signals.

Leave a Reply

Your email address will not be published. Required fields are marked *