advertisement

Probability Randomness • Long-Run (limiting) behavior of a chance (nondeterministic) process • Relative Frequency: Fraction of time a particular outcome occurs • Some cases the structure is known (e.g. tossing a coin, rolling a dice) • Often structure is unknown and process must be simulated • Probability of an event E (where n is number of trials) : Number of times E occurs P ( E ) lim n n Set Notation • • • • • S a set of interest Subset: A S A is contained in S Union: A B Set of all elements in A or B or both Intersection: AB Set of all elements in both A and B Complement: Ā Set of all elements not in A A A S A( B C ) AB AC A ( BC ) ( A B)( A C ) DeMorgan' s Laws : A B A B n n Ai Ai i 1 i 1 AB A B n n Ai Ai i 1 i 1 Probability • • • Sample Space (S)- Set of all possible outcomes of a random experiment. Mutually exclusive and exhaustive form. Event- Any subset of a sample space Probability of an event A (P(A)): 1. P(A) ≥ 0 2. P(S) = 1 3. If A1,A2,... is a sequence of mutually exclusive events (AiAj = ): P Ai P Ai i 1 i 1 Counting Rules for Probability (I) • Multiplicative Rule- Brute-force method of defining number of elements in S – – – – Experiment consists of k stages The jth stage has nj possible outcomes Total number of outcomes = n1...nk Tabular structure for k = 2 Stage 1\ Stage2 1 2 … n2 1 1 2 … n2 2 n2+1 n2+2 … 2n2 … … … … … n1 (n1-1)n2+1 (n1-1)n2+2 … n1 n2 Counting Rules for Probability (II) • Permutations: Ordered arrangements of r objects selected from n distinct objects without replacement (r ≤ n) – Stage 1: n possible objects – Stage 2: n-1 remaining possibilities – Stage r : n-r+1 remaining possibilities n! P n(n 1)...( n r 1) (n r )! n r Counting Rules for Probability (III) • Combinations: Unordered arrangements of r objects selected from n distinct objects without replacement (r ≤ n) – The number of distinct orderings among a set of r objects is r ! – # of combinations where order does not matter = number of permutations divided by r ! n P n! r r ! r ! ( n r )! n r Counting Rules for Probability (IV) • Partitions: Unordered arrangements of n objects partitioned into k groups of size n1,...nk (where n1 +...+ nk = n) – Stage 1: # of Combinations of n1 elements from n objects – Stage 2: # of Combinations of n2 elements from n-n1 objects – Stage k: # of Combinations of nk elements from nk objects n n n1 n n1 ... nk 1 ... nk n1 n2 (n n1 ... nk 1 )! n! (n n1 )! ... n1!(n n1 )! n2 !(n n1 n2 )! nk !(n n1 ... nk 1 nk )! n! n1!...nk ! Counting Rules for Probability (V) • Runs of binary outcomes (String of m+n trials) – Observe n Successes (S) – Observe m Failures (F) • k = minimum # of “runs” of S or F in the ordered outcomes of the m+n trials Case 1 : Equal runs of S and F (Series begins with one, ends with other) : m 1 n 1 2 k 1 k 1 P (2k runs ) m n m Case 2 : One more run of S or F (Series begins and ends with same one) : m 1 n 1 m 1 n 1 k k 1 k 1 k P (2k 1 runs ) m n m Runs Examples – 2006/7 UF NCAA Champs Football: n 14 W, m 1 L: Wins : 2 runs Losses: 1 Run 3 2k 1 k 1 Note: 15 ways to have 1 Loss (Game 1, Game 2,..., Game 15) 2 Ways to have 2 Runs (Game 1 Loss or Game 15 Loss) 2 13 P (3 runs) 1 P(2 runs) 1- 15 15 Basketball: n 35 W, m 5 L: Wins : 5 runs Losses: 4 Runs 9 2k 1 k 4 5-1 35 1 5-1 35 1 4 4 1 4-1 4 5984 185504 P(9 runs) .2910 658008 40 5 Conditional Probability and Independence • In many situations one event in a multi-stage experiment occurs temporally before a second event • Suppose event A can occur prior to event B • We can consider the probability that B occurs given A has occurred (or vice versa) • For B to occur given A has occurred: – At first stage A has to have occurred – At second stage, B has to occur (along with A) • Assuming P(A), P(B) > 0, we can obtain “Probability of B Given A” and “Probability of A Given B” as follow: P( AB) P( B | A) P( A) P( AB) P( A | B) P( B) If P(B|A) = P(B) and P(A|B) = P(A), A and B are said to be INDEPENDENT Rules of Probability Complement ary Events : P A 1 P ( A) Additive Rule : P A B P( A) P( B) P( AB) If A and B are mutually exclusive : P A B P( A) P( B ) Multiplica tive Rule : P ( AB) P ( A) P ( B | A) P ( B) P ( A | B ) If A and B are independen t : P AB P( A) P( B) Bayes’ Rule - Updating Probabilities • Let A1,…,Ak be a set of events that partition a sample space such that (mutually exclusive and exhaustive): – each set has known P(Ai) > 0 (each event can occur) – for any 2 sets Ai and Aj, P(Ai and Aj) = 0 (events are disjoint) – P(A1) + … + P(Ak) = 1 (each outcome belongs to one of events) • If C is an event such that – 0 < P(C) < 1 (C can occur, but will not necessarily occur) – We know the probability C will occur given each event Ai: P(C|Ai) • Then we can compute probability of Ai given C occurred: P C P(C | A1 ) P( A1 ) P(C | Ak ) P( Ak ) P(C | Ai ) P( Ai ) P( Ai and C ) P( Ai | C ) P(C | A1 ) P( A1 ) P(C | Ak ) P( Ak ) P(C ) Example - OJ Simpson Trial • Given Information on Blood Test (T+/T-) – Sensitivity: P(T+|Guilty)=1 – Specificity: P(T-|Innocent)=.9957 P(T+|I)=.0043 • Suppose you have a prior belief of guilt: P(G)=p* • What is “posterior” probability of guilt after seeing evidence that blood matches: P(G|T+)? P (T ) P (T G ) P (T I ) P (G ) P (T | G ) P ( I ) P (T | I ) p * (1) (1 p*)(.0043) P (T G ) P (G ) P (T | G ) p * (1) p* P (G | T ) P (T ) P (T ) p * (1) (1 p*)(.0043) .9957 p * .0043 Source: B.Forst (1996). “Evidence, Probabilities and Legal Standards for Determination of Guilt: Beyond the OJ Trial”, in Representing OJ: Murder, Criminal Justice, and the Mass Culture, ed. G. Barak pp. 22-28. Harrow and Heston, Guilderland, NY OJ Simpson Posterior (to Positive Test) Probabilities P(G ) .10 Prior Probabilit y of Guilt : .10(1) .10 P(G | T ) .9627 .10(1) .90(.0043) .10387 P(G|T+) as function of P(G) 1 0.8 P(G|T+) 0.6 0.4 0.2 0 0 0.2 0.4 0.6 P(G) 0.8 1 1.2 Northern Army at Battle of Gettysburg Regiment I Corps II Corps III Corps V Corps VI Corps XI Corps XII Corps Cav Corps Arty Reserve Sum Label A1 A2 A3 A4 A5 A6 A7 A8 A9 Initial # 10022 12884 11924 12509 15555 9839 8589 11501 2546 95369 Casualties 6059 4369 4211 2187 242 3801 1082 852 242 23045 P(Ai) 0.1051 0.1351 0.1250 0.1312 0.1631 0.1032 0.0901 0.1206 0.0267 1 P(C|Ai) 0.6046 0.3391 0.3532 0.1748 0.0156 0.3863 0.1260 0.0741 0.0951 P(C|Ai)*P(Ai) 0.0635 0.0458 0.0442 0.0229 0.0025 0.0399 0.0113 0.0089 0.0025 0.2416 P(C) P(Ai|C) 0.2630 0.1896 0.1828 0.0949 0.0105 0.1650 0.0470 0.0370 0.0105 1.0002 • Regiments: partition of soldiers (A1,…,A9). Casualty: event C • P(Ai) = (size of regiment) / (total soldiers) = (Column 3)/95369 • P(C|Ai) = (# casualties) / (regiment size) = (Col 4)/(Col 3) • P(C|Ai) P(Ai) = P(Ai and C) = (Col 5)*(Col 6) •P(C)=sum(Col 7) • P(Ai|C) = P(Ai and C) / P(C) = (Col 7)/.2416 CRAPS • Player rolls 2 Dice (“Come out roll”): – – – – 2,3,12 - Lose (Miss Out) 7,11 - Win (Pass) 4,5,6.8,9,10 - Makes point. Roll until point (Win) or 7 (Lose) Probability Distribution for first (any) roll: Roll Probability Outcome 2 1/36 Lose 3 2/36 Lose 4 3/36 Point 5 4/36 Point 6 5/36 Point 7 6/36 Win 8 5/36 Point 9 4/36 Point 10 3/36 Point 11 2/36 Win After first roll: •P(Win|2) = P(Win|3) = P(Win|12) = 0 •P(Win|7) = P(Win|11) = 1 •What about other conditional probabilities if make point? 12 1/36 Lose CRAPS • Suppose you make a point: (4,5,6,8,9,10) – – – – You win if your point occurs before 7, lose otherwise and stop Let P mean you make point on a roll Let C mean you continue rolling (neither point nor 7) You win for any of the mutually exclusive events: • P, CP, CCP, …, CC…CP,… • If your point is 4 or 10, P(P)=3/36, P(C)=27/36 • By independence, and multiplicative, and additive rules: k 27 3 27 3 P (CP) C P ) P (CC 36 36 36 36 k Win P CP CCC CP 3 P( P) 36 k 3 27 3 27 3 3 27 P ( Win) 36 36 36 36 36 36 36 i 0 1 1 3 1 ( 4 ) 36 1 27 / 36 3 12 i CRAPS • Similar Patterns arise for points 5,6,8, and 9: – For 5 and 9: P(P) = 4/36 P(C) = 26/36 – For 6 and 8: P(P) = 5/36 P(C) = 25/36 i 1 4 26 4 4 36 2 Points 5 and 9 : P( Win ) 36 i 0 36 36 1 26 / 36 36 10 5 i 1 5 25 5 5 36 5 Points 6 and 8 : P( Win ) 36 i 0 36 36 1 25 / 36 36 11 11 Finally, we can obtain player’s probability of winning: CRAPS - P(Winning) Come Out Roll 2 3 4 5 6 7 8 9 10 11 12 Sum P(Roll) 1/36 2/36 3/36 4/36 5/36 6/36 5/36 4/36 3/36 2/36 1/36 1 P(Win|Roll) 0 0 1/3 2/5 5/11 1 5/11 2/5 1/3 1 0 P(Roll&Win) 0 0 1/36 2/45 25/396 1/6 25/396 2/45 1/36 1/18 0 P(Roll&Win) 0 0 0.02777778 0.04444444 0.06313131 0.16666667 0.06313131 0.04444444 0.02777778 0.05555556 0 0.49292929 P(Win) P(Roll|Win) 0 0 0.05635246 0.09016393 0.12807377 0.33811475 0.12807377 0.09016393 0.05635246 0.11270492 0 1 Note in the previous slides we derived P(Win|Roll), we multiply those by P(Win) to obtain P(Roll&Win) and sum those for P(Win). The last column gives the probability of each come out roll given we won. Odds, Odds Ratios, Relative Risk • Odds: Probability an event occurs divided by probability it does not occur odds(A) = P(A)/P(Ā) – Many gambling establishments and lotteries post odds against an event occurring • Odds Ratio (OR): Odds of A occurring for one group, divided by odds of A for second group • Relative Risk (RR): Probability of A occurring for one group, divided by probability of A for second group P( A) odds ( A) P( A) odds ( A | Group 1) OR (Group 1/Group 2) odds ( A | Group 2) P( A | Group 1) RR (Group 1/Group 2) P( A | Group 2) Example – John Snow Cholera Data • 2 Water Providers: Southwark & Vauxhall (S&V) and Lambeth (L) Provider\Status Cholera Death No Cholera Death Total S&V 3706 263919 267625 Lambeth 411 171117 171528 Total 4117 435036 439153 3706 .013848 .013848 odds( D | S & V ) .014042 267625 1 .013848 411 .002396 P ( D | L) .002396 odds ( D | L) .002402 171528 1 .002396 .013848 .014042 RR 5.78 OR 5.85 .002396 .002402 P( D | S & V )