Another very simple way of looking at Bayes theorem

In an earlier post (back in 2011) I mentioned that it would be interesting to visit Eliezer Yudkowsky's site and in particular his explanation of Bayes' theorem. It still is. In the post I posed the question that Eliezer uses on his site and gave the answer but I suggested you go and visit his site to find out how to calculate it. I didn't realise how common that question was back then. I was thinking about it recently and thought it might be nice to show another, more graphical way of getting to grips with the calculation.

Here is the question:

1% of women at age forty who participate in routine screening have breast cancer. 80% of women with breast cancer will get positive mammographies. 9.6% of women without breast cancer will also get positive mammographies. A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer?

Remember that in tests, only 15% of doctors got the correct answer. The same is true in courts where probability of guilt is calculated. There are lots of examples and it's pretty serious. Bayes' Theorem really needs to be better understood.

Step 1

Right, we can draw the following tree.

1% of women (0.01) have cancer. 

Therefore, 99% do not (0.99). 

We mark the values on the branches as shown
            
                
        0.01    
      --------- C [cancer]
     |          
     |          
     |          
     |           
     |
     |
start
     |
     | 
     |           
     |          
     |          
     |  0.99    
      --------- NC [no cancer]
                 

Step 2

What else do we know?

80% (0.8) of women with breast cancer will get positive mammographies. 9.6% (0.096) of women without breast cancer will also get positive mammographies.


                     0.8
                 ---------- P [positive]
                | 
                |
        0.01    |
      --------- C [cancer]
     |          |
     |          |
     |          |
     |           ---------- N [negative]
     |
     |
start
     |
     |              0.096
     |           ---------- P [positive]
     |          |
     |          |
     |  0.99    |
      --------- NC [no cancer]
                |
                |
                |
                 ---------- N  [negative]  


So the question is, given the above tree, a woman gets a positive mammography in a routine screening. What is the probability that she actually has breast cancer?

Step 3

I want to fill in the missing branches just to make it complete. So before we get into calculating our answer we'll tidy up.

                    0.8
                 ---------- P [positive]
                | 
                |
        0.01    |
      --------- C [cancer]
     |          |
     |          |
     |          |   0.2 
     |           ---------- N [negative]
     |
     |
start
     |
     |              0.096
     |           ---------- P [positive]
     |          |
     |          |
     |  0.99    |
      --------- NC [no cancer]
                |
                |
                |   0.904
                 ---------- N  [negative]  


The thing to do here, is to make the related branches add up to 1.0. 
So 0.8 + 0.2 = 1.0
And from the other node, 0.096 + 0.904 = 1.0
That is the probability of going down one of those two routes at each node is 100% (1.0)

Step 4

We calculate the final nodes, (they are intersections actually but that terminology might be confusing in this diagram. The intersections between C and P, C and N etc)
But 

                    0.8
                 ---------- P  C ∩ P = 0.01x0.8 = 0.008 
                | 
                |
        0.01    |
      --------- C [cancer]
     |          |
     |          |
     |          |   0.2 
     |           ---------- N  C ∩ N = 0.01x0.2 = 0.002
     |
     |
start
     |
     |              0.096
     |           ---------- P  NC ∩ P = 0.99x0.096 = 0.095 
     |          |
     |          |
     |  0.99    |
      --------- NC [no cancer]
                |
                |
                |   0.904
                 ---------- N  NC ∩ N = 0.99x0.904 = 0.895  

Step 5

Now answer the question.
A woman gets a positive mammography in a routine screening. What is the probability that she actually has breast cancer?

So we write this as,

P( P | C)   
      
which means Probability( Positive | Cancer), 
or, the probability of getting a Positive, given that the woman has Cancer.

This is conditional probability.
 P( P | C ) can be converted to:

 C ∩ P          Note that  C ∩ P = P ∩ C  
   P

So we plug in the numbers:



 C ∩ P     =      0.008             
   P          0.008 + 0.095   <--(all of the posibilities of being positive)

           =  0.008 
              0.103

           = 0.77 or 7.7%  

most of the doctors when tested, estimated the figure to be around 70%!

The equation

It doesn't look too much like Bayes' theorem though does it. It might be good to work toward the equation now though we don't need to know it to do conditional probablity.

P(A|B) 

is read as the probability of A given B. So we know B, or B has happened, so what is the probability of A.


    P(A|B) = P(A ∩ B)

               P(B)

we can rearranged this as

    P(A ∩ B) = P(A|B).P(B)

It can be noted that 

    P(A ∩ B) = P(B ∩ A)

   

Also,

    P(A ∩ B) = P(A|B).P(B)

    P(A ∩ B) = P(B|A).P(A)

So,

          P(A|B).P(B) = P(B|A).P(A)

which can be arranged to give us the usual equation:

    P(A|B) = P(B|A).P(A)

                P(B)

The very best Bayes Explanation?

Is by the very interesting Eliezer S. Yudkowsky. Spend some time on his site. It's good to think.

The Intuitive Explanation of Bayes' Theorem

Why is Bayes important?

Here's a story problem about a situation that doctors often encounter:

1% of women at age forty who participate in routine screening have breast cancer.  80% of women with breast cancer will get positive mammographies.  9.6% of women without breast cancer will also get positive mammographies.  A woman in this age group had a positive mammography in a routine screening.  What is the probability that she actually has breast cancer?

Next, suppose I told you that most doctors get the same wrong answer on this problem - usually, only around 15% of doctors get it right. Most doctors estimate the probability to be between 70% and 80%, which is wildly incorrect.

What do you think the answer is?
(see below for the answer. See Yudkowsky for why.)

R - Cheat sheet - Vectors

Return to the R Cheat sheet main index page

Create a vector. (Note the c)
> x<-c(1,2,5,9,15)
> x
[1]  1  2  5  9 15


Create a vector with elements of consecutive integer values.
> x<-1:7
> x
[1] 1 2 3 4 5 6 7

> x<-1:10
> x<-x*2
> x
 [1]  2  4  6  8 10 12 14 16 18 20


Some commands

min(x) - minimum of the elements in x.

max(x) - maximum of the elements in x.

sort(x)

sort(x, decreasing=T)

> y<-c(5,8,2,0,2)

> sort(y, decreasing=T)

[1] 8 5 2 2 0


length(x) - length of vector
x[n] - get nth element from vector. first element is position 1, not position 0.
x[3:7] - get range of elements

> x<-1:10
> x
 [1]  1  2  3  4  5  6  7  8  9 10
> x[3:7]
[1] 3 4 5 6 7

x[-5] Get all elements from the vector except the 5th element.

x[x>5]  Get the elements whose values are greater than 5.

x > 5 Show which elements in the vector have values greater than b5.

> x
 [1]  1  2  3  4  5  6  7  8  9 10
> x> 5
 [1] FALSE FALSE FALSE FALSE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE


Creating sequences
seq(n)


> seq(10)
 [1]  1  2  3  4  5  6  7  8  9 10
> seq(0,1,length=10)
 [1] 0.0000000 0.1111111 0.2222222 0.3333333 0.4444444 0.5555556 0.6666667
 [8] 0.7777778 0.8888889 1.0000000
> seq(0,1,by=0.2)
[1] 0.0 0.2 0.4 0.6 0.8 1.0

Modulus of a Complex number

For, z = a + bi , the modulus is
|z| = sqrt(a^2 + b^2)

e.g, 
z = (1 + 2i)^2
z = (1 + 2i) . (1 + 2i)
  = 1 + 2i + 2i + 4i^2    (remember that, i^2 = -1 )
  = 1 + 4i + 4(-1)
  = -3 + 4i


|z| = sqrt( -3^2 + 4^2)
    = sqrt(9 + 16)
    = sqrt(25)
|z| = 5

A bit more on Complex numbers

We know that, 
i = sqrt (-1)

and therfore, 

i^2 = -1

Also, sqrt (9) = 3

 sqrt (-9) = sqrt(-1).sqrt(9) = i.sqrt(9) = 3i
We can check this. 
(3i)^2 = 3^2.i^2 = 9.-1 = -9

So, 

(3i)^2 = -9

3i = sqrt(-9)

3i is an imaginary number.

Complex number are imaginary and real numbers together.

e.g.

6 + 3i is a complex number.

Addition of complex numbers 

Here we have two complex numbers zi and z2

zi = a + bi
z2 = c + di
we add the real parts then add the imaginary parts.

zi + z2 = (a + c) + (bi + di)

        = (a + c) + (b + d)i

Subtraction of complex numbers 

Here we have two complex numbers zi and z2

zi = a + bi

z2 = c + di

we subtract the real parts then subtract the imaginary parts.

zi - z2 = (a - c) + (bi - di)

        = (a - c) + (b - d)i

Multiplication of complex numbers 

Again we have two complex numbers zi and z2

zi = a + bi

z2 = c + di

zi . z2 = (a + bi) . (c + di)

        = a(c+di) + bi(c+di)

        = ac+adi + cbi + (bi.di)          ....... (eqn 1)

let's sort out the (bi.di)

(bi.di) = bd.i^2

we know that i^2 = -1

so, db.i^2 = -bd

Returning to where we were in eqn (1),

zi . z2 = ac + adi + cbi - (bi.di)

        = ac + adi + cbi - bd

        = (ac - bd) + (adi + cbi)

        = (ac - bd) + (ad + cb).i

Division of complex numbers 

Again we have two complex numbers zi and z2

zi = a + bi

z2 = c + di

zi / z2 = (a+bi)/(c+di)

We can use the rule:

(a+b).(a-b) = a^2-b^2

The Conjugate of a complex number is a reverse of the direction of the imaginary number.

The Conjugate of (a + bi)is (a - bi)

The Conjugate is written with a bar over the top, so the conjugate of z1 is written z1 bar. 

How do I type that here? I don't know!

When we multiply an imaginary number by its conjugate we get a Real number. Here's the trick:

zi     (a+bi)   c-di   ac-adi + bci-bdi^2

--- =  ------ . ---- = -------------------

z2     (c+di)   c-di   c^2 + d^2

      ac-adi + bci-bdi^2         [remember that , i^2 = -1]

    = ------------------- 

         c^2 + d^2

      ac-adi + bci+bd

    = ----------------- 

         c^2 + d^2

       (ac+bd) + (bc-ad)i

    = --------------------- 

         c^2 + d^2

       ac+bd         bc-ad

    =  --------  +  ------- . i

       c^2+d^2      c^2+d^2

example.

1+2i    (1+2i)   2-3i   (1.2) + (1.(-3i)) + (2.2i) + (2i.(-3i))

---- =  ------ . ---- = ---------------------------------------

2+3i    (2+3i)   2-3i   (2.2) + (2.(-3i) + (3i.(2) + (3i.(-3i))

       2 - 3i + 4i - 6i^2

     = ------------------

       4 - 6i + 6i + 9i^2

        2 - 3i + 4i + 6

     =  ------------------

             4 + 9

     

        8 + i

     =  -----

         13  

     

     =  8      1

       ---  + --- i     

       13     13

Basic imaginary numbers from definition

i^0  = 1
i^1 = i
i^2 = -1
i^3 = -i
i^4 = 1

i^5  = i

i^6 = -1

i^7 = -i

i^8 = 1

i^9 = i

e.g.

So, i^ 325 = ?

325/4 = 8 remainder 1

( note 1 is the modulus)

same as i^1 = i

Remember, i = sqrt( -1 )

OK, a little bit More
I should really quickly note the definition:

We know that, 
i = sqrt -1
and therefore
i^2 = -1


therefore,
i^1 = i    [ anything to the power of 1 is itself ]
i^2 = -1   [ given above ]
i^3 = i^2.i^1 = -1.i = -i   

i^4 = i^3.i^2.i^1 = -i.-1.i = 1




A bit more on complex numbers