*************************************************************; * Examples of stratified 2x2 tables: * (i) Simpson's Paradox * (ii) The Mantel-Haenszel (strata) test cures Simpson's Paradox * (iii) The MH strata test with three strata and a loss of power * in the combined table * * (i) Suppose that we have a set of 2x2 contingency tables for the * same row and column variables for different instances of a * ``strata'' variable, which might (for example) be sex or time. * Here `strata' (singular: `stratum') is a Latin word meaning * ``layers''. * * A naive way to test for a lack of independence between the row and * column variables for such a set of tables is to combine counts * across strata and test the combined 2x2 table. However, the * combined 2x2 table can have qualitatively different properties * than the within-strata tables. * * As an example, consider the following two hypothetical tables for * a comparison between a New and Standard drug for two sexes: * * Females Males * Improved NotImpr %Impr Improved NotImpr %Impr * New 80 20 80% 120 180 40% * Standard 210 90 70% 30 70 30% * * Thus 80% (80/100) of females improved on the new drug in comparison * with 70% (210/300) for the standard drug. For males, 40% (120/300) * improved on the new drug and 30% (30/100) on the standard drug. * While the treated condition appeared to be more resistant in * males, both sexes did better on the new drug. * * In contrast, for the COMBINED table for both sexes: * * Both sexes together * Improved NotImpr Total %Impr * New 200 200 400 50% * Standard 240 160 400 60% * * Now 50% (200/400) improved on the new drug in comparison with * 60% (240/400) on the standard drug. In contrast to the sexes * individually, males and females combined did WORSE on the new * drug. This might be summarized for a lay person by saying that * females did better on the new drug, that males also did better * on the new drug, but that humans as a whole did worse. * * This is an example of what is called SIMPSON'S PARADOX. * * Note that the phi coefficients in the output below are positive * for the two individual tables, but negative for the combined * table. This is another indication of the ``difference in * direction'' between the individual and the combined tables. * * The basic reason for Simpson's Paradox can be illustrated using * the language of random variables. Consider a probability space * built from the 800 individuals in the study, each equally * weighted. For these 800 individuals, define three random * variables * * X=1,2 for New drug, Standard drug (respectively) * Y=1,2 for Improved, Not improved * S=1,2 for Female, Male * * An identity that is easy to derive using probability theory is * * Cov(X,Y) = E[ Cov(X,Y|S) ] + Cov[ E(X|S), E(Y|S) ] (*) * * Here Cov(X,Y) = E[(X-E(X))(Y-E(Y)] and ``|S'' means * ``conditional on that value of S''. This can be stated as the * jingle, * ``The (combined) covariance is the mean of the conditional * covariances plus the covariance of the conditional means.'' * * The %Improvements in the example above are more comparable to * correlation coefficients than to covariances, but covariances * will illustrate the basic principles. * * Note that Cov(X,Y|S=1)>0 and Cov(X,Y|S=2)>0 for both strata in * this case, since ``New drug'' and ``Improved'' are positively * correlated within each strata. * * Similarly, Cov(X,Y)<0, since ``New drug'' and ``Improved'' are * negatively correlated in the combined 2x2 table. Thus the third * term in (*) must be sufficiently negative so that Cov(X,Y)<0. * * Here E(X|S=1)-1 is the proportion of the female sample that was * given the new drug and E(X|S=2)-1 is same proportion for the * male sample. If this proportion were the same for both sexes, * then g(s)=E(X|S=s) would satisfy g(1)=g(2) and X would be a * random variable that is constant on the probability space. * In general * * Cov(X,Y) = E[(X-E(X))(Y-E(Y)] = 0 * * if either X or Y is a constant random variable, since in that * case X-E(X)=0 or Y-E(Y)=0 identically. This implies that * * Cov(X,Y) = E[ Cov(X,Y|S) ] (**) * * whenever E(X|S) or E(Y|S) is the same across strata. Eqn (**) * says that the combined covariance is then a weighted average of * within-strata covariances. This means that Simpson's Paradox * cannot occur. * * In the example above E(X|S) could easily have been controlled by * the experimenters, since it is essentially just the within-strata * sample sizes. In contrast, E(Y|S) could not. Thus Simpson's * Paradox in this case is a result of a poorly designed experiment * with variable E(X|S). The FDA (Federal Drug Administration) and * their statistical consultants are very wary about this kind of * problem with statistical procedures, since the signs of the * covariances could just as easily have been in the opposite * directions and be used as evidence that the New drug is worth * marketing. * * It follows from (*) that Simpson's Paradox could not occur here * unless Cov[E(X|S), E(Y|S)] < 0. The data above imply * * E(X|S) E(Y|S) * S=1 (Female) 1.75 1.275 * S=2 (Male) 1.25 1.625 * * with * E(X) E(Y) * 1.50 1.50 * * In particular * * (E(X|S)-E(X))(E(Y|S)-E(Y)) < 0 for S=1,2 * * which implies Cov[E(X|S),E(Y|S)] < 0. To see if this covariance * is sufficiently negative to overwhelm E[Cov(X,Y|S)] > 0, we would * have to compute numerically the expressions in the ``jingle'', * but this at least shows how Simpson's Paradox is possible. * * * (ii) THE MANTEL-HAENSZEL STRATA TEST: * HOW TO ANALYZE 2x2 TABLES IN A WAY TO AVOID SIMPSON'S PARADOX * * A better way to analyze stratified 2x2 tables than combining the * tables across strata and hoping that Simpson's Paradox does not * occur is to use the Mantel-Haenszel (strata) test. * * The Mantel-Haenszel test for 2x2 tables is based on summing across * strata one of the entries in the within-strata tables (for * example, the upper-left entry) with a correction for the expected * values under the assumption of no associations within strata. * Specifically, the numerator of the MH strata test is * * Q = Sum(s=1,S) (a_s - r1_s*c1_s/n_s) * * where s=1,...,S are the strata, a_s is the UL entry of the 2x2 * table in strata #s, and r1_s,c1_s,n_s are the row sum for the * first row, the column sum for the first column, and the total * number of counts. The MH test statistic is Z=Q/sqrt{Var(Q)} for * * Var(Q) = Sum(s=1,S) Var(a_s) * * = Sum(s=1,S) (r1_s)(r2_s)(c1_s)(c2_s)/(n_s^2*(n_s-1)) * * where Var(a_s) is the variance of a_s for the same probability * model in the s-th stratum as is used for the Fisher exact text. * Under the assumption of no associations within-strata, Z has an * asymptotically N(0,1) distribution, and X=Z^2 is chi-square with * one degree of freedom. * * This test has the advantages: * (a) it avoids Simpson's Paradox, since a tendency for * (for example) the upper-left entry to be larger than expected * is summed across strata, and * (b) it tests directly for a common trend in deviations away * from no association. This is often what one is interested in for * stratified 2x2 tables. That is, it tests for a trend rather than * being an omnibus test for deviation from independence, and so * should have greater power to detect a trend. * * SAS uses Cochran-Mantel-Haenszel (CMH) for the Mantel-Hanszel * strata test. This has the advantage that it avoids confusion * with the Mantel-Haenszel trend test for a single rxc table. * * * A second example below gives a situation in which the combined * table for three strata is not significant but the Mantel-Haenszel * strata test is highly significant. In this case, combining data * across strata into a combined table has caused information about * a significant trend in the data to be lost. This example also * gives another example of the use of SAS arrays in a data step. * * The example above was adapted from DeGroot's book, ``Probability * and Statistics'', Section 9.5. *************************************************************; title 'Example of Mantel-Haenzel strata test - YOURNAME'; options ls=75 ps=60 nocenter pageno=1; *************************************************************; * Create a SAS dataset from the two stratified 2x2 tables above. Here * * row = New drug/Standard drug * col = Improved/Not improved * sex = stratum * num = cell count *************************************************************; data tab1; input row$ f1-f2 m1-m2; * Create four records for this row; sex='Female'; col='Improved '; num=f1; output; sex='Female'; col='NotImproved'; num=f2; output; sex='Male '; col='Improved '; num=m1; output; sex='Male '; col='NotImproved'; num=m2; output; datalines; New 80 20 120 180 Standard 210 90 30 70 ; proc print; title2 'The data as SAS sees it:'; run; *************************************************************; * The (Cochran-)Mantel-Haenszel (strata) test: * * If you say tables A*C*D or tables A*B*C*D in `proc freq', * then you get output for a series of C*D 2x2 tables for all * possible values of A or A and B. * * Thus the default `proc freq' output for tables sex *row*col / chisq * is a list of output for within-strata 2x2 contingency table * tests without any overall tests. *************************************************************; proc freq order=data; title2 'Chi-square tests for two 2x2 tables:'; title3 'Note the signs of the two phi values.'; tables sex * row*col / chisq; weight num; run; *************************************************************; * Proc freq normally combines records into cells. Thus if `sex' is * omitted above, `proc freq' automatically combines the counts into * a single 2x2 combined table. *************************************************************; proc freq; title2 'THE COMBINED TABLE - Chi-square test'; title3 'Note the difference in sign in the phi value.'; title4 "THIS SHOWS SIMPSON'S PARADOX"; tables row*col / chisq; weight num; run; *************************************************************; * The option `cmh' tells SAS to carry out the stratified * (Cochran-)Mantel-Haenzel (strata) test. The option `noprint' * below suppresses the output for the individual within-strata 2x2 * tables, which we have seen already, and may not want to look at * at all if there are a large number of strata. * * The syntax * * tables sex * row*col / cmh chisq; * * tells SAS to analyze both the within-strata tables and to carry out * the CMH test across strata in one procedure.* *************************************************************; proc freq order=data; title2 '(Cochran-)Mantel-Haenszel test for 2 tables'; title3 'Note that the BRESLOW-DAY test is NOT SIGNIFICANT'; tables sex * row*col / cmh noprint; weight num; run; *************************************************************; * A second CMH example: Here the combined table is not significant, * most likely because combining the values across 2x2 tables * has hidden evidence for a significant trend across strata. * * EXERCISE: What would happen if you left off the semicolon * at the end of the comment "Age is `strata'" ? * What would the 2x2 tables look like? * *************************************************************; data tab2; input row$ t1$ c1-c2 t2$ c3-c4 t3$ c5-c6; * Read in 10 values; * Here age (=t1,t2,t3) is the stratum variable; * Output 6 records from this row:; age=t1; col='Sick'; num=c1; output; col='Well'; num=c2; output; age=t2; col='Sick'; num=c3; output; col='Well'; num=c4; output; age=t3; col='Sick'; num=c5; output; col='Well'; num=c6; output; * The following deletes these variables from the dataset; drop t1-t3 c1-c6; datalines; Treated Young 99 190 Middle 41 159 Old 40 42 Control Young 306 381 Middle 193 618 Old 108 107 ; proc print; title2 'A SECOND EXAMPLE WITH 3 2x2 TABLES:'; title3 'The data as SAS sees it (9 variables were dropped):'; run; *************************************************************; * If we did NOT include the option `order=data' in `proc freq' * below, then individual 2x2 tables would be analyzed in the * order `Middle-Old-Young' instead of `Young-Middle-Old', since * the first ordering is the alphabetical ordering. *************************************************************; proc freq order=data; title3 'Within-strata tests for 3 tables'; title4 ' AND the (Cochran-)Mantel Haenszel test'; tables age * row*col / cmh chisq; weight num; run; proc freq order=data; title3 'The INCORRECT combined table.'; table row*col / chisq; weight num; run; *************************************************************; * The same example with arrays in the datastep: * The syntax `k+1' is equivalent to k=k+1: That is, increase k by one. *************************************************************; title3 'USING SAS ARRAYS in the data step:'; data tab3; input row$ t1$ c1-c2 t2$ c3-c4 t3$ c5-c6; * Read in 10 values; array nn(*) c1-c6; * An array for the six counts; array colc(*) $ cc1-cc2 ('Sick' 'Well'); *An array for columns; array ageptr(*) $ t1-t3; * An array for `Age'; * Output 6 records from each row:; k=0; * k increases over 6 values, so k=1,2,...,6; do i=1 to 3; do j=1 to 2; age=ageptr(i); col=colc(j); k+1; num=nn(k); output; end; end; * Two `ends' to close two do-loops; drop i j k cc1-cc2 t1-t3 c1-c6; datalines; Treated Young 99 190 Middle 41 159 Old 40 42 Control Young 306 381 Middle 193 618 Old 108 107 ; proc print; title4 'The data as SAS sees it'; title5 'This should be exactly the same as before.'; run;