Program to illustrate finding exact P-values for permutation tests Wilcoxon signed rank and rank sum tests (This is RanksSims.txt on the Ma408 Web site.) Written by XXXX WILCOXON SIGNED-RANK TEST: Signed rank (After-Before) data: Data (n=9): -0.952 0.147 -1.022 -0.43 -0.62 -0.59 -0.49 0.08 -0.01 Signed ranks with signs (n=9): -8 +3 -9 -4 -7 -6 -5 +2 -1 Observed signed-rank statistic: 5 Expected value: 22.5 Centered at mean for a 2-sided P-value: Observed (centered) Wilcoxon signed-rank statistic: 17.5 Exact permutation-test P-value: P = 0.0390625 (2-sided) Estimating the true P-value by doing 10000 random permutations: Starting random-number seed: 123456 Exact P-value: 0.0390625 403 `success(es)' in 10000 trials (That is, randomized centered scores >= 17.5) Simulated P-value: 403/10000 = 0.0403 95% CONFIDENCE INTERVAL FOR TRUE P-VALUE: (The middle value is the estimated P-value.) (0.0364, 0.0403, 0.0442) EXACT: 0.0391 WILCOXON RANK-SUM TEST: Sample I (m=9): 23 21 22 14 23 19 15 15 16 Sample I midranks (m=9): 1 2.5 2.5 4 5.5 9.5 12.5 15.5 15.5 Sample II (n=14): 27 27 22 21 27 28 19 20 31 24 21 23 23 21 Sample II midranks (n=14): 5.5 7 9.5 9.5 9.5 12.5 15.5 15.5 18 20 20 20 22 23 Joint midranks in one array (n=23): 1 2.5 2.5 4 5.5 9.5 12.5 15.5 15.5 5.5 7 9.5 9.5 9.5 12.5 15.5 15.5 18 20 20 20 22 23 Observed W_X score: 68.5 Expected value: 108 Centered at expected value for 2-sided test: 39.5 Normal-approx permutation-test P-value: P = 0.0118 (2-sided) Estimating the true P-value by doing 10000 random permutations (specifically, by doing 10000 random shuffles): Starting random-number seed: 123456 Exact P-value: 0.0118 116 `success(es)' in 10000 trials (That is, randomized centered scores >= 39.5) Simulated P-value: 116/10000 = 0.0116 95% CONFIDENCE INTERVAL FOR TRUE P-VALUE: (The middle value is the estimated P-value.) (0.0095, 0.0116, 0.0137) NORMAL APPROXIMATION: 0.0118 Random numbers used: 220,000