December 30, 2009

A professor of medicine wrote:

      "Well-known, well taught, still neglected.  In many applications the true positive rate is only 1 %, meaning that usually false positives will far outnumber true positives. (......) The info content of criteria is best when the pre-criteria positivity estimate is close to

50 %.  lf we survey a population, which only has 1 % RA in it, like the full population, then we shall have a lot of false positives and only a few true positives.  Google 'false positives' and look for Wikipedia refs!".


Comment by B. Soric:

      "False positive paradox" (from Wikipedia)


      "The false positive paradox is a situation where the incidence of a condition is lower than the false positive rate of a test, and therefore, when the test indicates that a condition exists, it is probable that the result is a false positive". 

      For example, if a subject has the disease, a medical test may be, say, 99% likely to correctly indicate that she does, and if a subject does not have the disease, it may be 99% likely to correctly indicate that she doesn't. Suppose that a disease occurs in 1 out of 10,000 people.  The numbers from the Wikipedia example are here shown in a contingency table:

      p=0.99 ;    p'=0.99 ;    1-p'=0.01

                       True positive    True negative     Sum _                 

Test positive:      a= 99            b=     9999          10,098 = S

Test negative:     c=   1            d=989,901        989,902

               Sum:   R=100          N= 999,900     1,000,000 = M


      1.  If we suppose that all the values are known, we calculate:

Positive predictive value = a/(a+b) = 1-P =  99/10,098 = 0.0098 

Proportion of false positives =  b(a+b)  = P = 0.9902

      2.  But even if we know only  p,  p',  M,  and  S,  we can still calculate:

R = [S-(1-p')M] /(p+p'-1) = 100 ;    N = M-R = 999,900

P = (1-p')N/S = 0.9902

      So, the "false positive paradox" may be well-known, but so far I have not found anywhere that the latter possibility of calculation (under 2.) has been published by anybody. It is very simple, but it seems to be very important for researchers!


December 31, 2009

Another professor wrote this:

      The proportion (P) of false positives cannot be exactly calculated, even for very large samples, because sensitivity and specificity are known only approximately.  


Comment by B. Soric:

      That may be true, but unless we consider S/M or calculate  P,  we don't know at all whether  P  is too large or not.

      Suppose, for example, that we apply ACR criteria to 800 patients, whose diagnoses  may be even totally unknown to us, and 417 of them are classified as RA.  So, we know:  M= 800 ;   S= 417 ;   S/M= 0.521 ;   p= 0.935 ;   p'= 0.893   while  R,  N  and  P  are still unknown.  Without calculating  P  or taking into consideration  S/M  we have no idea of the percentage of wrong RA diagnoses (false positives);  but we can calculate as follows: 

      R= [S-(1-p')M] /(p+p'-1) = 400 ;   N= M-R = 400 ;   and:

      P= (1-p')N/S = 0.103   -  though this value of  P  is just an approximate or expected value.  (The same value of  P  we find in Table 2. for S/M = 0.521 ;  see on page "Home" of this web-site!).  So, we may expect or hope that a large part of those 417 patients indeed have rheumatoid arthritis.

      On the contrary, if, say, only 120 of the 800 patients are classified as RA, we find R=42 ;  N=758 ;  P=0.676   In the latter case we certainly cannot assume that most of those 120 patients indeed have RA.