We have a test for some condition, say it is the dreaded Lurgi, and are testing some sample of the population, say 10,000 folks.

We have reason to believe that the prevalence of Lurgi is 1% of our population. That means that 99% do not have the condition.

We have an estimate that this test correctly identifies 99% of the people with Lurgi where the condition exists, missing 1%. This former percentage is called the sensitivity of the test.

We also have an estimate that it correctly identifies 99% of the people without Lurgi where the condition does not exist, giving false alarms for 1%.  This former percentage is called the specificity or selectivity of the test.

Higher sensitivity or selectivity or both reduce the rate of errors. We can call this the power of the test.

If we test the 1% of the people with Lurgi, we can expect to identify 99% of that 1% or 0.99% of the sample correctly as having the condition, missing 1% of that 1% or 0.01% of the sample.

This also means that if we test the 99% of the people without Lurgi, we can expect to identify 99% of that 99% or 98.01% of the sample correctly as not having the condition, with false alarms of  1% of that 99% or 0.99% of the sample.

Since our sample is 10,000 people, we get the following results:

Prevalence	With Condition (1%)	Without Condition (99%)	Totals
Tests Positive	Hits = 99 (10,000 X .99 X .01)	False Alarms = 99 (10,000 X .01 X .99)	198
Tests Negative	Misses = 1 (10,000 X .01 X .01)	Correct Rejections = 9,801 (10,000 X .99 X .99)	9,802
Totals	100	9,900	10,000

You can see that for this population and test power, the number of false alarms is as great as the number of hits. That is, the number of hits is overstated by a factor of two, since we do not know from the test which ones are the false alarms, only their expected frequency.

The misses are not a problem. If you reverse the prevalence numbers, so that 99% have the condition, the false alarm problem goes away, and the miss problem becomes significant. For prevalence figures in the middle, the problems with errors become smaller.

With poor tests, less power, we get more errors. With even lower prevalence, we get more false alarms.

In medicine and statistics in general:

1. hits are called true positives
2. misses are called false negatives
3. correct rejections are called true negatives
4. false alarms are called false positives

You may encounter these terms. The language I have used is both more intuitively understood, and is also the language of the formal methods underlying this type of analysis, Signal Detection Theory. You can relate it to smoke detectors, fire alarms, or many common situations, once you understand it.

 

Overview

With respect to medical testing in general, the prevalence of a condition in the population has a huge effect on the number of false alarms and misses. Depending on the prevalence of the condition, and the performance of the test, the number of false alarms can exceed the number of hits. It is simple arithmetic, but a bit confusing.

If there is 0% infection in the population, any report of infection logically has to be a false alarm; there can be no hits. If there is 100% infection in the population, every report of infection logically has to be a hit; there can be no false alarms. The number of estimated false alarms drops as the percent infection rises.

If there is 0% infection in the population, logically there can be no misses. If there is 100% infection in the population, any report of no infection logically must be a miss. The number of estimated misses rises as the percent infection rises.

This is true even with a test of high sensitivity (detects infections if infection exists, with not too many misses) and high selectivity/specificity (detection of no infection if no infection exists, with not too many false alarms).

As sensitivity increases the number of hits increases and number of misses decreases. With a sensitivity of 100%, which does not happen, there would be no misses. See the graph below for a visual explanation of this.

As selectivity/specificity increases the number of correct rejections increases and the number of false alarms decreases. With a selectivity/specificity of 100%, which does not happen, there would be no false alarms. See the graph below for a visual explanation of this.

If selectivity/specificity is low, you get more false alarms. Combine this with a low prevalence, and the false alarms outshine the hits. This overstates the prevalence of the infection.

If the sensitivity is low, you get more misses. Combine this with a high prevalence, and the misses outshine the correct rejections. This understates the prevalence of the infection.

An Example

Use the calculations:

1. Hits = Sensitivity X Test Cases X Prevalence
2. False Alarms = (1-Selectivity) X Test Cases X  (1- Prevalence)
3. Misses = (1 – Sensitivity) X Test Cases X Prevalence
4. Correct Rejections = Selectivity X Test Cases X (1 – Prevalence)

Sensitivity	Selectivity/ specificity	Test Cases	Prevalence	Hits	False Alarms	Misses	Correct Rejections
95%	85%	10000	0.00%	0	1500	0	8500
95%	85%	10000	10.00%	950	1350	50	7650
95%	85%	10000	20.00%	1900	1200	100	6800
95%	85%	10000	30.00%	2850	1050	150	5950
95%	85%	10000	40.00%	3800	900	200	5100
95%	85%	10000	50.00%	4750	750	250	4250
95%	85%	10000	60.00%	5700	600	300	3400
95%	85%	10000	70.00%	6650	450	350	2550
95%	85%	10000	80.00%	7600	300	400	1700
95%	85%	10000	90.00%	8550	150	450	850
95%	85%	10000	100.00%	9500	0	500	0

Note the changes in 1) false alarms to hits, and 2) misses to correct rejections, as the prevalence increases.

Another Example Showing Ratios of Incorrect to Correct

For a selectivity/specificity of 99%, a sensitivity of 99%, and a prevalence of 1%, 50% of positives will be false alarms. It gets better as the prevalence increases (table below).

For a selectivity/specificity of 99%, a sensitivity of 99%, and a prevalence of 99%, 50% of negatives will be misses. It gets better as the prevalence decreases (table below).

 Prevalence

	0.00%	1.00%	2.00%	50.00%	98.00%	99.00%	100.00%
False Positive to True Positive (FP/TP)	Infinite	1.00	0.49	0.01	0.00	0.00	0.00
False Negative to True Negative Ratio (FN/TN)	0.00	0.00	0.00	0.01	0.49	1.00	Infinite
False Positive to All Positive (FP/(TP + FP))	100.00%	50.00%	33.11%	1.00%	0.02%	0.01%	0.00%
False Negative to All Negative (FN/(TN + FN))	0.00%	0.01%	0.02%	1.00%	33.11%	50.00%	100.00%

Measurement

In medical testing, we need to establish some test measure, some quantification of the condition of interest. We then need some method of assessment for determining this measurement, the values for that condition of interest. We need a consistent and reliable protocol for administering and scoring the test.

Discriminating Signal from Noise

We can look at the measure as the signal. We can look at spurious signals as noise, extraneous information which will make it hard to detect the signal.

We need some method to discriminate the signal from the noise. Different tests have different abilities to differentiate the signal from random noise. That is to say not all tests give the same level of performance.

The Right and the Wrong of It

In signal detection theory, there are two ways to be wrong: false alarms and misses, and there are two ways to be right: hits and correct rejections.

1. Hits are a measure of how many with the condition are correctly identified as having the condition. This is also called a true positive (TP).
2. False alarms are a measure of how many without the condition are incorrectly identified as having the condition. This is also called a false positive (FP).
3. Misses are a measure of how many with the condition are incorrectly identified as not having the condition. This is also called a false negative (FN).
4. Correct rejections are a measure of how many without the condition are correctly identified as not having the condition. This is also called a true negative (TN).

Measuring Performance

Sensitivity, given as a fraction or a percent, is the ability of a test to correctly identify those who have the condition. It can only be assessed against the percentage of those who have the condition, the prevalence. It gives a rate for hits, and when the one’s complement is taken, a rate for misses.

Selectivity/specificity, given as a fraction or a percent, is the ability of a test to correctly identify those who do not have the condition. It can only be assessed against the percentage of those who do not have the condition, the infrequency. It gives a rate of correct rejections, and when the one’s complement is taken, a rate for false alarms.

Setting a Threshold

For a given test method we establish a threshold, some cut-off value, for our measurement. We use this to determine if we are getting a signal, or just noise. Above the threshold a measure will be deemed to be a detected signal, below the threshold will be no detected signal. We can set the threshold to bias the detection one way or another. The resulting differences in type of error will be dependent upon the threshold setting.

The ratio of hits to misses depends on the threshold, as does the ratio of correct rejections to false alarms. A decreased threshold shifts the bias towards more hits and fewer misses. At the same time, it shifts the bias towards more false alarms and fewer correct rejections. So hits and false alarms rise and fall in the same direction according to the bias.

Improving the Detector

A better test gives better discrimination of correct versus incorrect results, that is, better accuracy. This can be accomplished by changing the test, or changing the test protocols.

Looking at Counts

We may have statistics on expected performance for our test, but we also want to calculate estimated statistics for some given number of tests. We will want to count the number of independent tests performed and use those numbers in our calculations.

Prevalence and Infrequency

Prevalence is the estimated measure of the percent of the total population who have the condition.

Infrequency is the complement of prevalence, and is the estimated measure of the percent of the total population who do not have the condition.

Note that I use the word infrequency, as an antonym to frequency, which itself is a synonym for prevalence. There may be another term in common use, but I did not discover such.

Accuracy, Discrimination and Errors

We can compute a simple measure of accuracy by taking the total errors and dividing by the total of correct plus erroneous observations. With a better test, more discriminatory power, more accuracy, the error rate decreases.

With decreasing prevalence, the number of false alarms increases, and the number of misses decreases.

With an increasing prevalence, the number of false alarms decreases, and the number of misses increases.

Ratio of Errors to Correct

The ratio of false alarms to hits is found by taking the infrequency rate multiplied by the expected false alarm percentage and then dividing this quantity by the prevalence multiplied by the expected hit percentage.

The ratio of misses to correct rejections is found by taking the prevalence rate multiplied by the expected miss rate and then dividing this quantity by the infrequency rate multiplied by the expected correct rejection percentage.

Working through an example

Let me work through an example:

1. We have an outbreak of Lurgi in Upper Middlemarsh. It is a terrible disease. ¹
2. We have reason to believe that 2% of the population is infected. This is the prevalence. So 98% will not be infected. This is the infrequency.
3. We have a test for Lurgi, validated so that it identifies 90% of those infected and misses 10%. This is the sensitivity of the test.
4. The test also identifies, gives a correct rejection, of 90% of those not infected, and gives a false alarm for 10% of those who are not infected. This is the selectivity/specificity of the test.
5. We test 1000 people once. This is the test count. Of those 1000 people, 2% or 20 will be infected. It follows that 98% or 980 will not be infected.
6. If we run our test on the 2% infected, it will identify correctly 90% or 18. These are the hits.
7. It follows that 10% or 2 of those infected will be missed. These are the misses.
8. If we run our test on that 98% not infected, it will identify correctly 90% or 882. These are the correct rejections.
9. It follows that 10% or 98 of those not infected will be identified as infected. These are the false alarms.
10. We have 18 hits versus 98 false alarms.
11. We have 2 misses verses 882 not infected, correct rejections.
12. It can be concluded that the false alarm rate to hit rate of 98/18 = 544% is very bad because of this mediocre level of test accuracy and the low disease prevalence.
13. It can be concluded that the misses to correct rejection rate of 2/882 = 23% is not as bad because of the high disease infrequency.
14. If we reverse this, changing the prevalence figure to 90% infected, the false alarms become small and the missed infections much larger.
15. If we have a better test, we can reduce the error rate.
16. We should try to get a better test.

I’m going to put these figures in the table below:

		Does The Condition Exist? Testing 1000 for Lurgi with Sensitivity of 90% and Selectivity/specificity of  90%
		Condition Exists 2% Estimated Prevalence = 20	Condition Is Absent 98% Estimated Infrequency = 980
Was the Effect Observed?	Effect Observed	90% x 20 = 18 Hits	10% x 980 = 98 False Alarms
	Effect Not Observed	10% x 20 = 2 Misses	90% x 980 = 882 Correct Rejections
Estimated Counts Based on Test Performance, Prevalence, and Number of Tests

1 https://gladbloke.wordpress.com/2009/06/04/the-dreaded-lurgi/

See also:

1. https://ephektikoi.ca/blog/2020/10/11/selectivity-sensitivity-and-prevalence/
2. https://ephektikoi.ca/blog/2020/10/13/bibliography-for-testing-and-uncertainty/
3. https://ephektikoi.ca/blog/2020/10/12/true-and-false-key-aspects-of-medical-tests/

 

“If 1 in 100 disease-free samples are wrongly coming up positive, the disease is not present, we call that a 1% false positive rate.

…

Because of the high false positive rate and the low prevalence, almost every positive test, a so-called case, identified by Pillar 2 since May of this year has been a FALSE POSITIVE. Not just a few percent. Not a quarter or even a half of the positives are FALSE, but around 90% of them. Put simply, the number of people Mr Hancock sombrely tells us about is an overestimate by a factor of about ten-fold. Earlier in the summer, it was an overestimate by about 20-fold.” — Dr Michael Yeadon, https://lockdownsceptics.org/lies-damned-lies-and-health-statistics-the-deadly-danger-of-false-positives/

Foreword

This is a write up on false positive, false negative, true positive, true negative thinking. It is nice to now have those words, “specificity” (true negative) and “sensitivity” (true positive) which I lacked before.

I realize this fits within the framework of signal detection theory. When I studied signal detection, I had no idea why it was considered important, particularly in Psychology, but over the years, I have often gone back to it, since it comes up routinely in a number of guises. It has to be statistical signal detection, which would be an obvious extension to binary, deterministic signal detection. This also fits within the framework of fuzzy logic, which I have looked at in years past.

Medical testing makes use of the ideas on signal detection and upon reflection, is no different than the consideration of any other sort of evidence – it has to be obtained, assessed for quality, and interpreted to understand the implications.

Anybody who tells you that such and such a test is any given percentage effective is misleading you, unintentionally or perhaps even deliberately.

There are four quadrants in the “gold standard” detection matrix: true positive, false positive, true negative, and false negative. There are at least two percentages to be considered, and they vary independently according to the bias in your test, the detection threshold decided upon. You can bias so that you find all the cases, with the consequence that you get a lot of false positives (false alarms). You can bias so that you get few false positives, and as a result, get a lot of false negatives (miss the fire). In addition, the baseline must be considered, with respect to a Bayesian statistical analysis. So, the prior information on general infection rates will make the percentages change. Low base rate of infection gives a high number of false positives. High base rate of infection gives a low number of false positives.

Bayesian reasoning: start with prior probabilities (assessed somehow) and see how probabilities change with new evidence.

If you do not have much noise masking the signal, results are easier to interpret. If you have large numbers and effects that are strong with respect to variability, the statistics should bear you out. Potentially confounding factors can be accounted for.

Introduction

It is routine for medical tests to be used to determine the health status of people. The simple view of a test is that it returns a true or false result, using some measure, some test instrument, and some testing protocol. Of course anyone reflecting on the issue even a little bit will realize that this is a much over-simplified view of things. For one thing, what threshold, what cut-off point, is being used to make the decision of true or false? What is the measure being used, and what is the measuring instrument? Most things we measure involve some sort of continuous scale. Are the measurements continuous or is it yes and no? What are the typical values that are used to make the judgement? What are the health implications? How are the numbers to be interpreted as a screening device or diagnostic tool? All of these considerations are important for understanding the test.

In this discussion I draw on ratios and proportions, odds, signal detection theory, statistics including Bayesian statistics, and simple arithmetic. I use these tools to examine the accuracy of testing.

Key Points to be Explained

Medical tests are not perfect; they give erroneous results along with correct results. We can estimate the accuracy of a test using scientific investigation. We can estimate how likely the test is to find that there is some condition (a hit). We can estimate how likely the test is to find there is not some condition (correct rejection). We can bias the test by changing the threshold, the cut-off value. We can increase hits and false alarms together, or reduce both together. In addition a low prevalence of the condition will give a lot of false alarms. A high prevalence of the condition will give a lot of misses. Also, a highly selective test will reduce the number of false alarms whereas a highly sensitive test will reduce the number of misses.

The Perfect Test

Here is a diagram which shows testing with no allowance for error.

		Does the Condition Exist?
Is the Effect Observed?	Effect Observed	Condition exists
	Effect Not Observed	Condition is absent
True or false – assuming no errors

The above chart shows:

1. Is the effect observed using the test?
2. Does the condition exist in the person tested?

As a result, we have two cases for the test result:

1. The condition is observed so is deemed to exist
2. The condition is not observed so is deemed to not exist

The Imperfect, Real-world Test

With a bit of thought, the question of errors for the test will come up. Is the test perfect? This would seem highly unlikely.

In testing, there are two ways for the results to be true, and two ways for the results to be false.

		Does The Condition Exist?
		Condition Exists	Condition Is Absent
Was the Effect Observed?	Effect Observed	Condition Is Correctly Considered To Exist     (HIT) (TRUE POSITIVE)	Condition Is Falsely Considered To Exist     (FALSE ALARM) (FALSE POSITIVE)
	Effect Not Observed	Condition Is Falsely Considered To Be Absent     (MISS) (FALSE NEGATIVE)	Condition Is Correctly Consider To Be Absent       (TRUE REJECTION) (TRUE NEGATIVE)
True or false – Assuming Errors

1. Observing an effect when the effect exists is a Hit
2. Not observing an effect when the effect exists is a Miss
3. Observing an effect when no effect exists is a False Alarm
4. Observing an effect when no effect exist is a Correct Rejection

Synonyms for these terms are:

1. Hit – True Positive (TP), Sensitivity
2. False Alarm – False Positive (FP), Type I Error
3. Miss – False Negative (FN), Type II Error
4. Correct Rejection – True Negative (TN), Specificity

I will use the abbreviations TP, FP, FN, TN in most of the discussion, although the meanings are probably not as easily grasped.

Proportions

The above matrix may be used to show more than one thing. It can be used to show proportions, expected percentages, the odds, for each cell of the matrix for some testing scenario. It may also be used to show the expected counts for each cell, given that we have an overall count for the number of tests.

In assigning proportions to these categories, these ratios can be expressed as fractions, decimal fractions or percentages. We have the following proportions of interest:

Overall Estimates of True Percentages:

1. percent of people who are actually infected
2. percent of people who are truly not infected

We will make this a binary split, not allowing for degrees of infection. That latter is important, but it is not important for this discussion.

Since you don’t know the percentage of infections, you must make an estimate. How this should be done is problematic in many cases. There may be little data, and the data may be suspect.

How we arrive at these estimated percentages is complex: scientific, statistical, and not without error. It should be done independently of the test being evaluated. We call these percentages prior odds, priors, or baseline values.

Test Performance

We also need to look at the performance of a given test for classifying the results.

1. Of those people who are actually infected, what percentages test as true positive (TP)?
2. We can get the false negative percentage (FN), the misses, by subtracting the true positive percentage from 100 percent. This is the arithmetic complement. Conversely, if you know the percentage of false negatives, you can take the complement of the number of cases to get the percentage of true positives.
3. Of those people who are truly not infected, what percentage test as true negative (TN)?
4. We can get the false positive percentage (FP), the false alarms, by subtracting the true negative percentage from 100 percent. This is the arithmetic complement. Conversely, if you know the percentage of false positives, you can take the complement of the number of cases to get the percentage of true negatives

At first glance, you might think that you can apply these percentages against the whole matrix, assuming that the matrix represents 100%, and each of the four cells has some fraction, all adding up to 100%. Things don’t work that way.

The test performance, the percentages, for separating false positives from true negatives only applies to those who are uninfected. Remember, this information on overall infection rates is obtained in some other manner, including other studies, some wild-assed guess, or a deity told you.

On the other side of the matrix, the test performance, the percentages, for separating false negatives from true positives only applies to those who are infected. Remember the sources of this information laid out above.

Here is a diagram adapted from a very good tutorial on this topic.  See “Confused by The Confusion Matrix: What’s the difference between Hit Rate, True Positive Rate, Sensitivity, Recall and Statistical Power?” by  The Curious Learner, https://learncuriously.wordpress.com/2018/10/21/confused-by-the-confusion-matrix/

Probabilities Based on Whether or Not the Effect Exists

		Does the Effect Exist?
		Effect Exists	Effect Doesn’t Exist
Was the Effect Observed?	Effect Observed	·         Hit Rate ·         True Positive Rate ·         Sensitivity ·         Statistical Power ·         (1 – Beta)	·         False Alarm Rate ·         False Positive Rate ·         Statistical Significance ·         Type I Error Rate (Alpha)
	Effect Not Observed	·         Miss Rate ·         False Negative Rate ·         Type II Error Rate (Beta)	·         Correct Rejection Rate ·         True Negative Rate

Testing as Evidence

Tests provide evidence. Evidence must be:

1. found or produced somehow.
2. assessed for reliability, quality, internal validity.
3. interpreted, examined for external validity, the implications made clear.

Tests can be given scores based on sensitivity (true positives) and selectivity (true negatives). As shown above, true positives and false negatives are complements of one and other, and also, true negatives and false positives are also complements of one and other.

Incorporating the Priors

Testing must take the priors into account when the calculations are done. It makes no sense to apply percentages for false positives and true negatives against the category of estimated infected. Likewise, it makes no sense to apply percentage for true positives and false negatives against the category of estimated uninfected. The false positives and true negative test percentages are based on the uninfected. The true positives and false negative test percentages are based on the infected.

Incorporating the Case Counts

We can work with percentages, but for analysis, we really want to see actual counts. We make use of the overall number of independent tests, the priors, and the test performance to make a two by two matrix of estimated test performance.

Threshold

You can set a threshold for a test score, setting the bias point. If you set the threshold, the sensitivity, to give more hits, you will get more false alarms and miss less often. If you set the threshold, the sensitivity, to give fewer hits, you will get fewer false alarms and miss more often.

False Versus True and Priors

With a low prior rate of infection, the number of false positives can be much greater than the number of true positives, even with an accurate test.

Test Performance and Receiver Operating Characteristic (ROC)

Estimated or actual values for a given test can be plotted, putting False Positives (X) against True Positives (Y) to give a curve. This plot is called the receiver operating characteristic (ROC) curve. Any point along the curve can be selected to give a cut-off point, a threshold. If this threshold is set to detect more cases, you also get more false positives. If this threshold is set to exclude more cases, you also get fewer false positives.

The area under the ROC curve also gives a measure of accuracy. The greater the area is, the more accurate the test. Since the axis both go from 0 to 1, the maximum area is 1 squared. A diagonal line for the ROC curve gives performance at chance levels.

Not all tests are equal. Some have much better accuracy overall. The more bowed the ROC curve is above the diagonal, the better the test.

Example Data for ROC Curve

See http://www.rad.jhmi.edu/jeng/javarad/roc/JROCFITi.html

False Positive Fraction (FPF)	True Positive Fraction (TPF)	Lower	Upper
0.0000	0.0000	0.0000	0.0000
0.0050	0.2301	0.0169	0.7407
0.0100	0.3135	0.0430	0.7718
0.0200	0.4168	0.0996	0.8061
0.0300	0.4860	0.1545	0.8282
0.0400	0.5384	0.2056	0.8449
0.0500	0.5807	0.2523	0.8587
0.0600	0.6159	0.2949	0.8705
0.0700	0.6461	0.3337	0.8808
0.0800	0.6723	0.3690	0.8901
0.0900	0.6955	0.4012	0.8985
0.1000	0.7161	0.4306	0.9062
0.1100	0.7347	0.4575	0.9132
0.1200	0.7515	0.4821	0.9198
0.1300	0.7668	0.5047	0.9258
0.1400	0.7809	0.5255	0.9314
0.1500	0.7938	0.5447	0.9366
0.2000	0.8454	0.6214	0.9577
0.2500	0.8822	0.6757	0.9723
0.3000	0.9096	0.7160	0.9824
0.4000	0.9466	0.7727	0.9934
0.5000	0.9691	0.8119	0.9978
0.6000	0.9832	0.8424	0.9994
0.7000	0.9918	0.8684	0.9999
0.8000	0.9967	0.8927	1.0000
0.9000	0.9992	0.9189	1.0000
0.9500	0.9998	0.9357	1.0000
1.0000	1.0000	1.0000	1.0000

Summary ROC Statistics

Number of Cases:   50

Number Correct:    42

Accuracy:          84%

Sensitivity:       88%

Specificity:       80%

Positive Cases Missed:  3

Negative Cases Missed:  5

(A rating of 3 or greater is considered positive.)

Fitted ROC Area:   0.905

Empiric ROC Area:  0.892

Plotting the ROC Curve

ROC Curve Type:   Fitted

Key for the ROC Plot

RED symbols and BLUE line:  Fitted ROC curve.
GRAY lines:  95% confidence interval of the fitted ROC curve.
BLACK symbols ± GREEN line:  Points making up the empirical ROC curve (does not apply to Format 5).

Testing Overall

The test can be viewed as the measure plus the measuring method. It can also include the procedures, the protocol for conducting the test. Differing protocols can change and confound the test results. Tests can be very accurate and still give a large number of false positives when the estimate of infection rates is low.

Testing Interpretation

The test results require interpretation by a skilled clinician. Sometimes, tests are used for screening, and sometimes for actual diagnosis. One test alone should not be relied upon. Tests should be repeated.

Testing Signal versus Noise

Test results for the same individual can vary because of “noise” masking the “signal.”  By noise we mean fluctuations in the measurement of interest that are based on other factors than the condition of interest, perhaps random factors.

Testing and Time Variance

Test results can vary for the same individual because the underlying conditions can change from one time to the next. Levels of any condition can fluctuate over time: hourly, daily, weekly, … . With heath conditions: you get infected, you get sick, you get better, you die, … .

The Calculations of the Estimates

False Positive Calculations

The simple calculations of false positive expected rates:

1. Prior baseline estimates for not infected
2. Test accuracy estimates for false positive
3. Number of tests

Multiply them together to get the expected count of false positives. False positives are only evaluated against the uninfected cases, not all test cases.

False Negative Calculations

Calculate the false negative expected rates

1. Prior baseline estimates for not infected
2. Test accuracy estimates for false negatives
3. Number of tests

Multiply them together to get the expected count of false negatives. False negatives are only evaluated against the infected cases, not all test cases.

True Positive Calculations

Calculate the true positive expected rates

1. Prior baseline estimates for infected
2. Test accuracy estimates for true positives
3. Number of tests

Multiply them together to get the expected count of true positives. True positives are only evaluated against the infected cases, not all test cases.

True Negative Calculations

Calculate the true negatives expected rates

1. Prior baseline estimates for not infected
2. Test accuracy estimates for true negatives
3. Number of tests

Multiply them together to get the expected count of true negatives. True negatives are only evaluated against the uninfected cases, not all test cases.

An Example

In the example below, I set the following parameters:

Number of Tests

1,000.00

 

Population Baseline Estimates
Prior Baseline Infection Rate Estimate	2 %
Baseline True Positives = Prior Baseline X Number of Tests	20
Baseline True Negatives = (1 – Prior Baseline) X Number of Tests	980

 

Testing Method Performance
Hit Rate (Sensitivity)	95%
Miss Rate = One’s Complement of Hit Rate	5%
False Alarm Rate	10%
Correct Rejection Rate (Specificity) = One’s Complement of False Alarm Rate	90%

 

Using these parameters, I calculate expected counts:

Expected Counts
True Positives (TP) = Baseline True Positives X Hit Rate	19
False Positives (FP) = Baseline True Negatives X False Alarm Rate	98
False Negatives (FN) = Baseline True Positives X Miss Rate	1
True Negatives (TN) = Baseline True Negatives X Correct Rejection Rate	882

Summaries

I summarize the calculated values in the matrix below. You can see that the number of false positives, under these assumptions, is 5 times the amount of true positives, i.e., very high. Also, the false negative rate is very low for this test and the prior infection rates. This is with a test selectivity of 90%, a test sensitivity of 95%, and an estimated infection rate of 2%.

		Does The Condition Exist?
		Condition Exists	Condition Is Absent
Was the Effect Observed?	Effect Observed	TP = 19	FP = 98
	Effect Not Observed	FN = 1	TN = 882
Estimated Counts Based on Test Performance, Priors, and Number of Tests

Measures of Test Performance

Below are various measures of test performance, test quality. They use the previous data from the previous example. The calculations presented here are simple. The interpretation takes more skill.

Core Set of Measures

Measures of Test Performance
Diagnostic Accuracy = (TP + TN) / TP + TN + FP + FN	0.90
Sensitivity = (TP) / (TP + FN)	0.95
Specificity = (TN) / (TN +FP)	0.90

Predictive Values

Positive Predictive Value (PPV) = (TP) / (TP + FP)	0.16
Negative Predictive Value (NPV) = (TN) / (TN + FN)	1.00

The Positive Predictive Value (PPV) and the Negative Predictive Value (NPV) give the probabilities based on whether of not the effect was observed. This contrasts with the sensitivity and selectivity that give probabilities based on the estimated existence of the effect.

Probabilities Based on Whether or Not the Effect was Observed

		Does the Effect Exist?
		Effect Exists	Effect Doesn’t Exist
Was the Effect Observed?	Effect Observed	True Discovery Rate Positive Predictive Value Precision	False Discovery Rate
	Effect Not Observed	False Omission Rate	True Omission Rate Negative Predictive Value

Adapted from “Confused by The Confusion Matrix: What’s the difference between Hit Rate, True Positive Rate, Sensitivity, Recall and Statistical Power?” by  The Curious Learner, https://learncuriously.wordpress.com/2018/10/21/confused-by-the-confusion-matrix/

Test Estimates with Differing Priors

Here are test estimates based upon five differing population baseline estimates, that is, differing estimates of priors. I vary the priors from 0.1 percent to 99.9 percent.

N.B. In order to avoid division by 0, I did not use 0.0 percent and 100 percent.

Number of Tests	1,000	1,000	1,000	1,000	1,000
Testing Method
Hit Rate (Sensitivity)	95%	95%	95%	95%	95%
Miss Rate = Ones Complement of Hit Rate	5%	5%	5%	5%	5%
False Alarm Rate	10%	10%	10%	10%	10%
Correct Rejection Rate (Specificity) = Ones Complement of False Alarm Rate	90%	90%	90%	90%	90%
Population Baseline Estimates
Prior Baseline	0.10%	2%	50%	98%	99.90%
Baseline True Positives = Prior Baseline X Number of Tests	1	20	500	980	999
Baseline True Negatives = (1 – Prior Baseline) X Number of Tests	999	980	500	20	1
Expected Counts
True Positives (TP) = Baseline True Positives X Hit Rate	0.95	19	475	931	949.05
False Positives (FP) = Baseline True Negatives X False Alarm Rate	99.9	98	50	2	0.1
False Negatives (FN) = Baseline True Positives X Miss Rate	0.05	1	25	49	49.95
True Negatives (TN) = Baseline True Negatives X Correct Rejection Rate	899.1	882	450	18	0.90
Quality Tests
Diagnostic Accuracy = (TP + TN) / TP + TN + FP + FN	0.90	0.90	0.93	0.95	0.95
Sensitivity = (TP) / (TP + FN)	0.95	0.95	0.95	0.95	0.95
Specificity = (TN) / (TN +FP)	0.90	0.90	0.90	0.90	0.90
Positive Predictive Value (PPV) = (TP) / (TP + FP)	0.01	0.16	0.90	1.00	1.00
Negative Predictive Value (NPV) = (TN) / (TN + FN)	1.00	1.00	0.95	0.27	0.02
Positive Predictive Likelihood Ratios = Sensitivity / (1 – Specificity)	9.50	9.50	9.50	9.50	9.50
Negative Predictive Likelihood Ratios = (1 – Sensitivity) / Specificity	0.06	0.06	0.06	0.06	0.06
Youden’s Index = (Sensitivity + Specificity) – 1	0.85	0.85	0.85	0.85	0.85
Diagnostic Odds Ratio (DOR) = (TP / FN) / (FP / TN)	171.00	171.00	171.00	171.00	171.00

 

 

 

Bibliography

1. Baratloo, Alireza, Mostafa Hosseini, Ahmed Negida, and Gehad El Ashal. “Part 1: Simple Definition and Calculation of Accuracy, Sensitivity and Specificity.” Emergency 3, no. 2 (2015): 48–49.
2. Harvey, Lew. “Detection Theory.” Psychology of Perception, 2014, 17.
3. learncuriously. “Confused by The Confusion Matrix Part 2: ‘Accuracy’ Is But One of Many Measures of Accuracy….” The Curious Learner (blog), October 27, 2018. https://learncuriously.wordpress.com/2018/10/28/confused-by-the-confusion-matrix-part-2/.
4. ———. “Confused by The Confusion Matrix: What’s the Difference between Hit Rate, True Positive Rate, Sensitivity, Recall and Statistical Power?” The Curious Learner (blog), October 20, 2018. https://learncuriously.wordpress.com/2018/10/21/confused-by-the-confusion-matrix/.
5. Lockdown Sceptics. “Lies, Damned Lies and Health Statistics – the Deadly Danger of False Positives.” Accessed October 6, 2020. https://lockdownsceptics.org/lies-damned-lies-and-health-statistics-the-deadly-danger-of-false-positives/.
6. Read “Intelligence Analysis: Behavioral and Social Scientific Foundations” at NAP.Edu. Accessed September 28, 2020. https://doi.org/10.17226/13062.
7.  “ROC Analysis: Web-Based Calculator for ROC Curves.” Accessed October 11, 2020. http://www.rad.jhmi.edu/jeng/javarad/roc/JROCFITi.html.
8. “ROC Curves – What Are They and How Are They Used?” Accessed October 4, 2020. https://acutecaretesting.org/en/articles/roc-curves-what-are-they-and-how-are-they-used.
9. Šimundić, Ana-Maria. “Measures of Diagnostic Accuracy: Basic Definitions.” EJIFCC 19, no. 4 (January 20, 2009): 203–11.

 

 

 

From a YouTube video, “Be Smart Wear a Mask”,  by Chris Martenson at https://www.youtube.com/watch?v=KANNNty9V3o

Preface

This is a write up on false positive, false negative, true positive, true negative thinking in testing for some condition, trying to detect if it is there or not. It is useful for discussing medical tests, but it applies to any sort of testing where decisions must be made.

I said I would write up testing, so I did. It always comes out seeming more complex than it seems at first glance, even though the calculations are simple arithmetic. I have another one in draft, hitting the material from a different angle, but unfortunately not making it simpler.

This fits within the framework of signal detection theory. When I studied signal detection, I had no idea why it was considered important, particularly in Psychology, but over the years, I have often gone back to it, since it comes up routinely in a number of guises. It has to be statistical signal detection, which would be an obvious extension to binary, deterministic signal detection. This also fits within the framework of fuzzy logic, which I have looked at in years past.

Medical testing makes use of the ideas on signal detection and upon reflection, is no different than the consideration of any other sort of evidence – it has to be obtained, assessed for quality, and interpreted to understand the implications.

The take away is that for low prevalence of a condition (e.g. infection), you can get a lot of false positives (false alarms), even with a fairly specific test.  For a high prevalence of a condition, you can get a lot of false negatives (misses). In addition, for a highly selective test, you get fewer false positives (false alarms), and for a highly sensitive test, you get fewer false negatives (misses).

In the example below, I show how false positives can be far greater than true positives, even for a fairly well performing test, when the prevalence is low:

Parameters	Complement of Parameters
Selectivity = .95	1 – selectivity = .05
Sensitivity = .99	1 – sensitivity = .01
Prevalence = .01	1 – prevalence = .99

We have this formula: Ratio of false positives (FP) to true positives (TP) = ( (1-selectivity) x (1-prevalence) ) / ((sensitivity) x (prevalence) ) .

False positives to true positives = ((.05) x (.99))/(.99) x (.01)) = 5

Using the formula, we get a 5 fold increase of false positives over true positives, for a test that has quite good performance. Below I will show how for any test with error rates, both low prevalence and high prevalence give more false results.

It is nice to now have those words, “specificity” (true negative) and “sensitivity” (true positive) for test performance which I lacked before. I give more explanation below.

Anybody who tells you that such and such a test is any given percentage effective is misleading you somewhat, unintentionally or perhaps even deliberately.

There are four quadrants in the “gold standard” detection matrix: true positive, false positive, true negative, and false negative. There are at least two percentages to be considered, and they vary independently according to the bias in your test, the detection threshold decided upon. You can bias so that you find all the cases, with the consequence that you get a lot of false positives (false alarms). You can bias so that you get few false positives, and as a result, get a lot of false negatives (miss the fire). In addition, the baseline must be considered, with respect to a Bayesian statistical analysis. So, the prior information on general infection rates will make the percentages change. Low base rate of infection gives a high number of false positives. High base rate of infection gives a low number of false positives.

Bayesian reasoning: start with prior probabilities (assessed somehow) and see how probabilities change with new evidence.

If you do not have much noise masking the signal, results are easier to interpret. If you have large numbers and effects that are strong with respect to variability, the statistics should bear you out. Potentially confounding factors can be accounted for.

See http://ephektikoi.ca/blog/2020/10/11/bibliography-on-testing-and-uncertainty/ for more information.

Introduction

It is routine for medical tests to be used to determine the health status of people. The simple view of a test is that it returns a true or false result, using some measure, some test instrument, and some testing protocol. Of course anyone reflecting on the issue even a little bit will realize that this is a much over-simplified view of things. For one thing, what threshold, what cut-off point, is being used to make the decision of true or false? What is the measure being used, and what is the measuring instrument? Most things we measure involve some sort of continuous scale. Are the measurements continuous or is it yes and no? What are the typical values that are used to make the judgement? What are the health implications? How are the numbers to be interpreted as a screening device or diagnostic tool? All of these considerations are important for understanding the test.

The Perfect Test

Here is a diagram of a test which does not consider that there might be some error.

	Does the Condition Exist?
Is the Effect Observed?	Observed	Condition exists
	Not observed	Condition is absent
True or false – assuming no errors

The Imperfect, Real-world Test

With a bit of thought, the question of errors for the test will come up. Is the test perfect? This would seem highly unlikely.

In testing, there are two ways for the results to be true, and two ways for the results to be false.

	Does The Condition Exist?
		Condition Exists	Condition Is Absent
Is The Effect Observed?	Observed	Condition Is Correctly Considered To Exist     (HIT) (TRUE POSITIVE)	Condition Is Falsely Considered To Exist     (FALSE ALARM) (FALSE POSITIVE)
	Not Observed	Condition Is Falsely Considered To Be Absent     (MISS) (FALSE NEGATIVE)	Condition Is Correctly Consider To Be Absent       (TRUE REJECTION) (TRUE NEGATIVE)
True Or False – Assuming Errors

 

1. Observing an effect when the effect exists is a Hit
2. Not observing an effect when the effect exists is a Miss
3. Observing an effect when no effect exists is a False Alarm
4. Observing an effect when no effect exist is a Correct Rejection

Synonyms for these terms are:

 

1. Hit – True Positive (TP), Sensitivity
2. False Alarm – False Positive (FP), Type I Error
3. Miss – False Negative (FN), Type II Error
4. Correct Rejection – True Negative (TN), Specificity

I will use the abbreviations TP, FP, FN, TN in most of the discussion, although the meanings are probably not as easily grasped.

Three factors for calculations:

1. Performance or discrimination ability of the test
2. Prevalence of the condition, present or absent to some percentage
3. Number of tests done independently

Performance of a test: selectivity and sensitivity

Sensitivity applies to a population that all truly have the condition. It is the ratio of hits to misses, usually as a percentage of hits to overall tests. Percentage is calculated (hits / (hits + misses) x 100). Sensitivity gives the percent hits. The one’s complement of sensitivity gives the percent misses. With poorer sensitivity tests, the number of misses gets larger.

Selectivity applies to a population were none have the condition. It is the ratio of correct rejections to false alarms, usually as a percentage of rejections to overall tests. Percentage is calculated as (rejections / (rejections + false alarms) x 100). Selectivity gives the percent correct rejections. The one’s complement of selectivity gives the false alarm percent. With poorer selectivity tests, the number of false alarms gets larger.

Prevalence gives the estimated number where the condition is present. The one’s complement of prevalence gives the estimated number where the condition is absent.

Lower prevalence gives more false positives or false alarms and also lower prevalence gives fewer false negatives or misses.

Higher prevalence gives more false negatives or misses and also higher prevalence gives fewer false positives or false alarms.

False negatives are a function of sensitivity and prevalence whereas false positives are a function of selectivity and prevalence.

The number of tests done independently gives the counts.

In calculating expected values for a condition and errors, use performance, prevalence and the number of tests

Hits	False alarms
Misses	Correct rejections

1. Hits = sensitivity x prevalence x number of tests
2. False alarms = (1- selectivity) x (1 – prevalence) x number of tests
3. Misses = (1 – sensitivity) x prevalence x number of tests
4. Correct rejections = selectivity x (1 – prevalence) x number of tests

Hits versus false alarms are a function of test performance and prevalence.

Considerations: It is a mistake to only use just performance and test count without taking prevalence into account – a big mistake

Given that you know the performance of a test, and the prevalence of the condition, you can calculate the ratios of:

1. False positives to true positives
2. False negatives to true negatives

Ratio of FP to TP ( (1-selectivity) x (1-prevalence) ) / ((sensitivity) x (prevalence) )

Ratio of FN to TN ( (1-sensitivity) x (prevalence) ) / ((selectivity) x (1 – prevalence) )

Example 1

Parameters	Complement
Selectivity = .95	1 – selectivity = .05
Sensitivity = .99	1 – sensitivity = .01
Prevalence = .01	1 – prevalence = .99

False positives to true positives = ((.05) x (.99))/(.99) x (.01)) = 5

Using the formula, we get a 5 fold increase of false positives over true positives

Example 2

Parameters	Complement
Selectivity = 1.00	1 – selectivity = 0.0
Sensitivity = 1.00	1 – sensitivity = 0.0
Prevalence = .50	1 – prevalence = .50

False positives to true positives = ((0) x (.5))/(1) x (.5)) = 0

Using the formula, we get a no increase in false positives over true positives; in fact, there are no false positives or false negatives.

Formulae Summarized

 Test Count = number of tests

 selectivity (fraction of correct rejections for those without condition)

complement of selectivity (fraction of false alarms for those without condition) = (1 – selectivity)

sensitivity (fraction of hits for those with condition)

complement of sensitivity (fraction of misses for those with condition) = (1 – sensitvity)

prevalence is actual incidence as percent of population with condition

complement of prevalance is percent of population without condition = (1 – prevalence)

Estimated True Positive (Hits, TP) = sensitivity x prevalence x case count

Estimated False Positive (False Alarms, FP) = (1 – selectivity) x (1 – prevalence) x (case count)

Estimated False Negatives (Misses, FN) = (1 – sensitivity) * (prevalence) x case count

Estimated True Negatives (Correct Rejections, TN) = (selectivity) x (1 – prevalence) x (case count)

Ratio of FP to TP ( (1-selectivity) x (1-prevalence) ) / ((sensitivity) x (prevalence) )

Ratio of FN to TN ( (1-sensitivity) x (prevalence) ) / ((selectivity) x (1 – prevalence) )

Worked Example For Different Test Parameters and Prevalence

The pattern for all is that the end regions of the curve get high inversely in proportion to the performance of the test. That is for low prevalence and high prevalence both, there are more errors. For low prevalence, the errors are false positives. For high prevalence, the errors are false negatives. The error rates for both types of errors are lower in the middle. High selectivity reduces false positive errors. High sensitivity reduces false negative errors. The ultimate high is the non-existent perfectly performancing test with selectivity of 100% and sensitivity of 100%.

Selectivity 100%, Sensitivity 100%
A Balanced Perfect Test						Counts
Count	Selectivity	Sensitivity	Prevalence	Ratio FP to TP	Ratio FN to TN	TP	FP	FN	TN
1000	1.00	1.00	0.00	0.00	0.00	0.00	0.00	0.00	1000.00
1000	1.00	1.00	0.02	0.00	0.00	20.00	0.00	0.00	980.00
1000	1.00	1.00	0.03	0.00	0.00	30.00	0.00	0.00	970.00
1000	1.00	1.00	0.04	0.00	0.00	40.00	0.00	0.00	960.00
1000	1.00	1.00	0.05	0.00	0.00	50.00	0.00	0.00	950.00
1000	1.00	1.00	0.10	0.00	0.00	100.00	0.00	0.00	900.00
1000	1.00	1.00	0.20	0.00	0.00	200.00	0.00	0.00	800.00
1000	1.00	1.00	0.30	0.00	0.00	300.00	0.00	0.00	700.00
1000	1.00	1.00	0.40	0.00	0.00	400.00	0.00	0.00	600.00
1000	1.00	1.00	0.50	0.00	0.00	500.00	0.00	0.00	500.00
1000	1.00	1.00	0.60	0.00	0.00	600.00	0.00	0.00	400.00
1000	1.00	1.00	0.70	0.00	0.00	700.00	0.00	0.00	300.00
1000	1.00	1.00	0.80	0.00	0.00	800.00	0.00	0.00	200.00
1000	1.00	1.00	0.90	0.00	0.00	900.00	0.00	0.00	100.00
1000	1.00	1.00	0.98	0.00	0.00	980.00	0.00	0.00	20.00
1000	1.00	1.00	1.00	0.00	0.00	1000.00	0.00	0.00	0.00

Selectivity 90%, Sensitivity 99%
An Unbalanced Test with Reasonable Performance						Counts
Count	Selectivity	Sensitivity	Prevalence	Ratio FP to TP	Ratio FN to TN	TP	FP	FN	TN
1000	0.90	0.99	0.01	10.00	0.00	9.90	99.00	0.10	891.00
1000	0.90	0.99	0.02	4.95	0.00	19.80	98.00	0.20	882.00
1000	0.90	0.99	0.03	3.27	0.00	29.70	97.00	0.30	873.00
1000	0.90	0.99	0.04	2.42	0.00	39.60	96.00	0.40	864.00
1000	0.90	0.99	0.05	1.92	0.00	49.50	95.00	0.50	855.00
1000	0.90	0.99	0.10	0.91	0.00	99.00	90.00	1.00	810.00
1000	0.90	0.99	0.20	0.40	0.00	198.00	80.00	2.00	720.00
1000	0.90	0.99	0.30	0.24	0.00	297.00	70.00	3.00	630.00
1000	0.90	0.99	0.40	0.15	0.01	396.00	60.00	4.00	540.00
1000	0.90	0.99	0.50	0.10	0.01	495.00	50.00	5.00	450.00
1000	0.90	0.99	0.60	0.07	0.02	594.00	40.00	6.00	360.00
1000	0.90	0.99	0.70	0.04	0.03	693.00	30.00	7.00	270.00
1000	0.90	0.99	0.80	0.03	0.04	792.00	20.00	8.00	180.00
1000	0.90	0.99	0.90	0.01	0.10	891.00	10.00	9.00	90.00
1000	0.90	0.99	0.98	0.00	0.54	970.20	2.00	9.80	18.00
1000	0.90	0.99	0.99	0.00	1.10	980.10	1.00	9.90	9.00

Selectivity 99%, Sensitivity 90%
An Unbalanced Test with Reasonable Performance						Counts
Count	Selectivity	Sensitivity	Prevalence	Ratio FP to TP	Ratio FN to TN	TP	FP	FN	TN
1000	0.99	0.90	0.01	1.10	0.00	9.00	9.90	1.00	980.10
1000	0.99	0.90	0.02	0.54	0.00	18.00	9.80	2.00	970.20
1000	0.99	0.90	0.03	0.36	0.00	27.00	9.70	3.00	960.30
1000	0.99	0.90	0.04	0.27	0.00	36.00	9.60	4.00	950.40
1000	0.99	0.90	0.05	0.21	0.01	45.00	9.50	5.00	940.50
1000	0.99	0.90	0.10	0.10	0.01	90.00	9.00	10.00	891.00
1000	0.99	0.90	0.20	0.04	0.03	180.00	8.00	20.00	792.00
1000	0.99	0.90	0.30	0.03	0.04	270.00	7.00	30.00	693.00
1000	0.99	0.90	0.40	0.02	0.07	360.00	6.00	40.00	594.00
1000	0.99	0.90	0.50	0.01	0.10	450.00	5.00	50.00	495.00
1000	0.99	0.90	0.60	0.01	0.15	540.00	4.00	60.00	396.00
1000	0.99	0.90	0.70	0.00	0.24	630.00	3.00	70.00	297.00
1000	0.99	0.90	0.80	0.00	0.40	720.00	2.00	80.00	198.00
1000	0.99	0.90	0.90	0.00	0.91	810.00	1.00	90.00	99.00
1000	0.99	0.90	0.98	0.00	4.95	882.00	0.20	98.00	19.80
1000	0.99	0.90	0.99	0.00	10.00	891.00	0.10	99.00	9.90

Selectivity 99%, Sensitivity 99%
A Balanced Test, with Excellent Performance						Counts
Count	Selectivity	Sensitivity	Prevalence	Ratio FP to TP	Ratio FN to TN	TP	FP	FN	TN
1000	0.99	0.99	0.01	1.00	0.00	9.90	9.90	0.10	980.10
1000	0.99	0.99	0.02	0.49	0.00	19.80	9.80	0.20	970.20
1000	0.99	0.99	0.03	0.33	0.00	29.70	9.70	0.30	960.30
1000	0.99	0.99	0.04	0.24	0.00	39.60	9.60	0.40	950.40
1000	0.99	0.99	0.05	0.19	0.00	49.50	9.50	0.50	940.50
1000	0.99	0.99	0.10	0.09	0.00	99.00	9.00	1.00	891.00
1000	0.99	0.99	0.20	0.04	0.00	198.00	8.00	2.00	792.00
1000	0.99	0.99	0.30	0.02	0.00	297.00	7.00	3.00	693.00
1000	0.99	0.99	0.40	0.02	0.01	396.00	6.00	4.00	594.00
1000	0.99	0.99	0.50	0.01	0.01	495.00	5.00	5.00	495.00
1000	0.99	0.99	0.60	0.01	0.02	594.00	4.00	6.00	396.00
1000	0.99	0.99	0.70	0.00	0.02	693.00	3.00	7.00	297.00
1000	0.99	0.99	0.80	0.00	0.04	792.00	2.00	8.00	198.00
1000	0.99	0.99	0.90	0.00	0.09	891.00	1.00	9.00	99.00
1000	0.99	0.99	0.98	0.00	0.49	970.20	0.20	9.80	19.80
1000	0.99	0.99	0.99	0.00	1.00	980.10	0.10	9.90	9.90

Selectivity 95%, Sensitivity 95%
A Balanced Test, with Decent Performance						Counts
Count	Selectivity	Sensitivity	Prevalence	Ratio FP to TP	Ratio FN to TN	TP	FP	FN	TN
1000	0.95	0.95	0.01	5.21	0.00	9.50	49.50	0.50	940.50
1000	0.95	0.95	0.02	2.58	0.00	19.00	49.00	1.00	931.00
1000	0.95	0.95	0.03	1.70	0.00	28.50	48.50	1.50	921.50
1000	0.95	0.95	0.04	1.26	0.00	38.00	48.00	2.00	912.00
1000	0.95	0.95	0.05	1.00	0.00	47.50	47.50	2.50	902.50
1000	0.95	0.95	0.10	0.47	0.01	95.00	45.00	5.00	855.00
1000	0.95	0.95	0.20	0.21	0.01	190.00	40.00	10.00	760.00
1000	0.95	0.95	0.30	0.12	0.02	285.00	35.00	15.00	665.00
1000	0.95	0.95	0.40	0.08	0.04	380.00	30.00	20.00	570.00
1000	0.95	0.95	0.50	0.05	0.05	475.00	25.00	25.00	475.00
1000	0.95	0.95	0.60	0.04	0.08	570.00	20.00	30.00	380.00
1000	0.95	0.95	0.70	0.02	0.12	665.00	15.00	35.00	285.00
1000	0.95	0.95	0.80	0.01	0.21	760.00	10.00	40.00	190.00
1000	0.95	0.95	0.90	0.01	0.47	855.00	5.00	45.00	95.00
1000	0.95	0.95	0.98	0.00	2.58	931.00	1.00	49.00	19.00
1000	0.95	0.95	0.99	0.00	5.21	940.50	0.50	49.50	9.50

Selectivity 90%, Sensitivity 90%
A Balanced Test, with OK Performance						Counts
Count	Selectivity	Sensitivity	Prevalence	Ratio FP to TP	Ratio FN to TN	TP	FP	FN	TN
1000	0.90	0.90	0.01	11.00	0.00	9.00	99.00	1.00	891.00
1000	0.90	0.90	0.02	5.44	0.00	18.00	98.00	2.00	882.00
1000	0.90	0.90	0.03	3.59	0.00	27.00	97.00	3.00	873.00
1000	0.90	0.90	0.04	2.67	0.00	36.00	96.00	4.00	864.00
1000	0.90	0.90	0.05	2.11	0.01	45.00	95.00	5.00	855.00
1000	0.90	0.90	0.10	1.00	0.01	90.00	90.00	10.00	810.00
1000	0.90	0.90	0.20	0.44	0.03	180.00	80.00	20.00	720.00
1000	0.90	0.90	0.30	0.26	0.05	270.00	70.00	30.00	630.00
1000	0.90	0.90	0.40	0.17	0.07	360.00	60.00	40.00	540.00
1000	0.90	0.90	0.50	0.11	0.11	450.00	50.00	50.00	450.00
1000	0.90	0.90	0.60	0.07	0.17	540.00	40.00	60.00	360.00
1000	0.90	0.90	0.70	0.05	0.26	630.00	30.00	70.00	270.00
1000	0.90	0.90	0.80	0.03	0.44	720.00	20.00	80.00	180.00
1000	0.90	0.90	0.90	0.01	1.00	810.00	10.00	90.00	90.00
1000	0.90	0.90	0.98	0.00	5.44	882.00	2.00	98.00	18.00
1000	0.90	0.90	0.99	0.00	11.00	891.00	1.00	99.00	9.00

Selectivity 50%, Sensitivity 50%
A Balanced Test, with Poor Performance						Counts
Count	Selectivity	Sensitivity	Prevalence	Ratio FP to TP	Ratio FN to TN	TP	FP	FN	TN
1000	0.50	0.50	0.01	99.00	0.01	5.00	495.00	5.00	495.00
1000	0.50	0.50	0.02	49.00	0.02	10.00	490.00	10.00	490.00
1000	0.50	0.50	0.03	32.33	0.03	15.00	485.00	15.00	485.00
1000	0.50	0.50	0.04	24.00	0.04	20.00	480.00	20.00	480.00
1000	0.50	0.50	0.05	19.00	0.05	25.00	475.00	25.00	475.00
1000	0.50	0.50	0.10	9.00	0.11	50.00	450.00	50.00	450.00
1000	0.50	0.50	0.20	4.00	0.25	100.00	400.00	100.00	400.00
1000	0.50	0.50	0.30	2.33	0.43	150.00	350.00	150.00	350.00
1000	0.50	0.50	0.40	1.50	0.67	200.00	300.00	200.00	300.00
1000	0.50	0.50	0.50	1.00	1.00	250.00	250.00	250.00	250.00
1000	0.50	0.50	0.60	0.67	1.50	300.00	200.00	300.00	200.00
1000	0.50	0.50	0.70	0.43	2.33	350.00	150.00	350.00	150.00
1000	0.50	0.50	0.80	0.25	4.00	400.00	100.00	400.00	100.00
1000	0.50	0.50	0.90	0.11	9.00	450.00	50.00	450.00	50.00
1000	0.50	0.50	0.98	0.02	49.00	490.00	10.00	490.00	10.00
1000	0.50	0.50	0.99	0.01	99.00	495.00	5.00	495.00	5.00

 

 

Originally published on August 5, 2020 at https://www.opednews.com/articles/Masks-and-Motivated-Reason-by-Mike-Zimmer-Belief_Bias_Covid-19_Empathy-200805-253.html

 

“How to Disgree

Paul Graham

March 2008

The web is turning writing into a conversation. Twenty years ago, writers wrote and readers read. The web lets readers respond, and increasingly they do—in comment threads, on forums, and in their own blog posts.

Many who respond to something disagree with it. That’s to be expected. Agreeing tends to motivate people less than disagreeing. And when you agree there’s less to say. You could expand on something the author said, but he has probably already explored the most interesting implications. When you disagree you’re entering territory he may not have explored.

The result is there’s a lot more disagreeing going on, especially measured by the word. That doesn’t mean people are getting angrier. The structural change in the way we communicate is enough to account for it. But though it’s not anger that’s driving the increase in disagreement, there’s a danger that the increase in disagreement will make people angrier. Particularly online, where it’s easy to say things you’d never say face to face.

If we’re all going to be disagreeing more, we should be careful to do it well. What does it mean to disagree well? Most readers can tell the difference between mere name-calling and a carefully reasoned refutation, but I think it would help to put names on the intermediate stages. So here’s an attempt at a disagreement hierarchy: … ” more How to Disgree

A bibliography of articles – studies and opinion, on the wearing of masks for reducing infection spread and severity.

I consider it prudent to wear a mask when in close contact with people not in your personal “bubble.” I do not feel it makes sense to wear one when outside in the fresh air while maintaining a couple of meters of distance from those who are not in my “bubble.” This is risk management 101. I also think that claims of harm, physical and psychological from the wearing of masks are overblown to the point of absurdity. The other issue, that of social control, seems possibly overblown as well, although I am aghast at some of the draconian steps taken by authorities is some places. There are also a number of ill-informed interpretations of the research literature.

I am by most indicators some sort of socialist libertarian, so yes, a libertarian. I still am not sold on the idea of masks being some part of a plot to control us. I believe that a lot of conspiracist views are well founded, since there are clearly many deep state conspiracies. I am not so sure about this one.

See for instance: https://www.opednews.com/articles/Motivated-Reasoning-by-Mike-Zimmer-Belief_Bias_Ideas_Motivated-Reasoning-200804-599.html and https://www.opednews.com/articles/Masks-and-Motivated-Reason-by-Mike-Zimmer-Belief_Bias_Covid-19_Empathy-200805-253.html