RIDGE REGRESSION for apple taste - YOURNAME 1 USING Inv(Z`Z+k*I) INSTEAD OF Inv(Z`*Z) IN PARAMETER ESTIMATES THE DATA AS SAS SEES IT 02:37 Tuesday, November 1, 2005 Obs con yy nat kk pp shade water 1 1 2876 20.0 38 2488 2.42 216 2 1 2078 11.1 13 2998 1.62 321 3 1 3052 19.8 31 3835 2.79 376 4 1 2265 13.9 19 2360 1.65 265 5 1 940 17.0 24 233 0.86 18 6 1 2815 16.9 26 3922 2.70 369 7 1 2661 11.6 16 4343 2.40 453 8 1 2181 14.3 22 3110 2.05 267 9 1 2052 10.5 13 2869 1.63 286 10 1 2064 18.2 31 2335 2.17 252 11 1 1551 8.3 8 1784 0.84 185 12 1 2338 20.4 36 2601 2.47 275 13 1 1753 8.7 18 2124 1.27 201 14 1 2110 7.5 4 4408 1.85 411 RIDGE REGRESSION for apple taste - YOURNAME 2 USING Inv(Z`Z+k*I) INSTEAD OF Inv(Z`*Z) IN PARAMETER ESTIMATES RIDGE REGRESSION USING PROC REG 02:37 Tuesday, November 1, 2005 THE FIRST 25 RECORDS IN `ridgest' Obs _MODEL_ _TYPE_ _DEPVAR_ _RIDGE_ _PCOMIT_ _RMSE_ Intercept 1 MODEL1 PARMS yy . . 247.820 299.645 2 MODEL1 RIDGEVIF yy 0.00 . . . 3 MODEL1 RIDGE yy 0.00 . 247.820 299.645 4 MODEL1 RIDGEVIF yy 0.01 . . . 5 MODEL1 RIDGE yy 0.01 . 249.214 502.000 6 MODEL1 RIDGEVIF yy 0.02 . . . 7 MODEL1 RIDGE yy 0.02 . 250.196 504.316 8 MODEL1 RIDGEVIF yy 0.03 . . . 9 MODEL1 RIDGE yy 0.03 . 251.141 499.631 10 MODEL1 RIDGEVIF yy 0.04 . . . 11 MODEL1 RIDGE yy 0.04 . 252.033 495.388 12 MODEL1 RIDGEVIF yy 0.05 . . . 13 MODEL1 RIDGE yy 0.05 . 252.865 492.510 14 MODEL1 RIDGEVIF yy 0.06 . . . 15 MODEL1 RIDGE yy 0.06 . 253.639 490.957 16 MODEL1 RIDGEVIF yy 0.07 . . . 17 MODEL1 RIDGE yy 0.07 . 254.362 490.526 18 MODEL1 RIDGEVIF yy 0.08 . . . 19 MODEL1 RIDGE yy 0.08 . 255.043 491.014 20 MODEL1 RIDGEVIF yy 0.09 . . . 21 MODEL1 RIDGE yy 0.09 . 255.688 492.249 22 MODEL1 RIDGEVIF yy 0.10 . . . 23 MODEL1 RIDGE yy 0.10 . 256.307 494.092 24 MODEL1 RIDGEVIF yy 0.11 . . . 25 MODEL1 RIDGE yy 0.11 . 256.903 496.433 Obs nat kk pp shade water yy 1 -10.7723 43.6183 0.345 -179.100 1.7524 -1 2 26.2153 80.3780 144.711 274.066 31.4078 -1 3 -10.7723 43.6183 0.345 -179.100 1.7524 -1 4 8.5230 8.9268 11.175 9.192 12.8519 -1 5 -16.7991 26.1634 0.175 263.665 1.3506 -1 6 6.0578 5.4613 5.481 3.247 7.5178 -1 7 -12.4188 22.5149 0.161 298.292 1.3032 -1 8 4.6093 3.9307 3.424 1.781 4.9768 -1 9 -8.7112 20.3050 0.156 307.876 1.2890 -1 10 3.6473 3.0235 2.388 1.177 3.5620 -1 11 -5.7433 18.7258 0.153 310.895 1.2847 -1 12 2.9699 2.4219 1.782 0.863 2.6918 -1 13 -3.3491 17.5158 0.151 311.467 1.2840 -1 14 2.4733 1.9970 1.395 0.676 2.1176 -1 15 -1.3887 16.5498 0.149 310.967 1.2845 -1 16 2.0978 1.6837 1.131 0.554 1.7184 -1 17 0.2404 15.7563 0.148 309.956 1.2852 -1 18 1.8068 1.4451 0.943 0.470 1.4294 -1 19 1.6120 15.0905 0.146 308.692 1.2859 -1 20 1.5765 1.2588 0.804 0.409 1.2133 -1 21 2.7801 14.5223 0.145 307.305 1.2862 -1 22 1.3911 1.1103 0.698 0.363 1.0472 -1 23 3.7849 14.0304 0.144 305.863 1.2862 -1 24 1.2394 0.9898 0.616 0.327 0.9169 -1 25 4.6567 13.5997 0.143 304.404 1.2859 -1 RIDGE REGRESSION for apple taste - YOURNAME 3 USING Inv(Z`Z+k*I) INSTEAD OF Inv(Z`*Z) IN PARAMETER ESTIMATES RIDGE REGRESSION USING PROC REG 02:37 Tuesday, November 1, 2005 PARAMETER ESTIMATES AS A FUNCTION OF K _RIDGE_ _TYPE_ _RMSE_ Intercept nat kk pp shade water . PARMS 247.820 299.645 -10.7723 43.6183 0.34494 -179.100 1.75238 0.00 RIDGE 247.820 299.645 -10.7723 43.6183 0.34494 -179.100 1.75238 0.01 RIDGE 249.214 502.000 -16.7991 26.1634 0.17531 263.665 1.35059 0.02 RIDGE 250.196 504.316 -12.4188 22.5149 0.16135 298.292 1.30316 0.03 RIDGE 251.141 499.631 -8.7112 20.3050 0.15604 307.876 1.28897 0.04 RIDGE 252.033 495.388 -5.7433 18.7258 0.15298 310.895 1.28471 0.05 RIDGE 252.865 492.510 -3.3491 17.5158 0.15082 311.467 1.28398 0.06 RIDGE 253.639 490.957 -1.3887 16.5498 0.14913 310.967 1.28447 0.07 RIDGE 254.362 490.526 0.2404 15.7563 0.14772 309.956 1.28523 0.08 RIDGE 255.043 491.014 1.6120 15.0905 0.14650 308.692 1.28587 0.09 RIDGE 255.688 492.249 2.7801 14.5223 0.14540 307.305 1.28622 0.10 RIDGE 256.307 494.092 3.7849 14.0304 0.14440 305.863 1.28622 0.11 RIDGE 256.903 496.433 4.6567 13.5997 0.14348 304.404 1.28588 0.12 RIDGE 257.482 499.183 5.4188 13.2188 0.14261 302.948 1.28520 0.13 RIDGE 258.048 502.272 6.0896 12.8789 0.14180 301.506 1.28421 0.14 RIDGE 258.604 505.642 6.6833 12.5732 0.14102 300.086 1.28295 0.15 RIDGE 259.154 509.248 7.2116 12.2966 0.14028 298.689 1.28143 0.16 RIDGE 259.700 513.052 7.6839 12.0446 0.13957 297.317 1.27970 0.17 RIDGE 260.243 517.022 8.1078 11.8138 0.13888 295.971 1.27776 0.18 RIDGE 260.785 521.133 8.4897 11.6014 0.13821 294.650 1.27566 0.19 RIDGE 261.329 525.361 8.8349 11.4051 0.13757 293.353 1.27339 0.20 RIDGE 261.873 529.690 9.1478 11.2228 0.13694 292.080 1.27099 0.21 RIDGE 262.421 534.104 9.4322 11.0530 0.13633 290.829 1.26847 0.22 RIDGE 262.971 538.588 9.6913 10.8942 0.13573 289.599 1.26584 0.23 RIDGE 263.526 543.133 9.9279 10.7453 0.13514 288.390 1.26312 0.24 RIDGE 264.085 547.727 10.1443 10.6052 0.13457 287.199 1.26031 0.25 RIDGE 264.649 552.363 10.3426 10.4731 0.13401 286.028 1.25743 0.26 RIDGE 265.219 557.033 10.5245 10.3481 0.13346 284.873 1.25448 0.27 RIDGE 265.793 561.730 10.6917 10.2297 0.13291 283.735 1.25148 0.28 RIDGE 266.374 566.450 10.8455 10.1171 0.13238 282.614 1.24842 0.29 RIDGE 266.960 571.187 10.9871 10.0100 0.13185 281.507 1.24532 0.30 RIDGE 267.552 575.937 11.1176 9.9078 0.13134 280.415 1.24218 RIDGE REGRESSION for apple taste - YOURNAME 4 USING Inv(Z`Z+k*I) INSTEAD OF Inv(Z`*Z) IN PARAMETER ESTIMATES RIDGE REGRESSION USING PROC REG 02:37 Tuesday, November 1, 2005 VARIANCES OF PARAMETER ESIMATES/MSE AS A FUNCTION OF K _RIDGE_ _TYPE_ nat kk pp shade water 0.00 RIDGEVIF 26.2153 80.3780 144.711 274.066 31.4078 0.01 RIDGEVIF 8.5230 8.9268 11.175 9.192 12.8519 0.02 RIDGEVIF 6.0578 5.4613 5.481 3.247 7.5178 0.03 RIDGEVIF 4.6093 3.9307 3.424 1.781 4.9768 0.04 RIDGEVIF 3.6473 3.0235 2.388 1.177 3.5620 0.05 RIDGEVIF 2.9699 2.4219 1.782 0.863 2.6918 0.06 RIDGEVIF 2.4733 1.9970 1.395 0.676 2.1176 0.07 RIDGEVIF 2.0978 1.6837 1.131 0.554 1.7184 0.08 RIDGEVIF 1.8068 1.4451 0.943 0.470 1.4294 0.09 RIDGEVIF 1.5765 1.2588 0.804 0.409 1.2133 0.10 RIDGEVIF 1.3911 1.1103 0.698 0.363 1.0472 0.11 RIDGEVIF 1.2394 0.9898 0.616 0.327 0.9169 0.12 RIDGEVIF 1.1138 0.8907 0.550 0.299 0.8126 0.13 RIDGEVIF 1.0085 0.8081 0.497 0.276 0.7277 0.14 RIDGEVIF 0.9194 0.7384 0.453 0.257 0.6578 0.15 RIDGEVIF 0.8432 0.6790 0.417 0.242 0.5994 0.16 RIDGEVIF 0.7776 0.6281 0.386 0.229 0.5500 0.17 RIDGEVIF 0.7206 0.5839 0.360 0.217 0.5080 0.18 RIDGEVIF 0.6708 0.5454 0.338 0.208 0.4718 0.19 RIDGEVIF 0.6270 0.5115 0.318 0.199 0.4405 0.20 RIDGEVIF 0.5882 0.4816 0.301 0.192 0.4131 0.21 RIDGEVIF 0.5538 0.4550 0.286 0.185 0.3890 0.22 RIDGEVIF 0.5230 0.4313 0.273 0.179 0.3677 0.23 RIDGEVIF 0.4953 0.4100 0.261 0.174 0.3488 0.24 RIDGEVIF 0.4704 0.3908 0.251 0.169 0.3319 0.25 RIDGEVIF 0.4478 0.3734 0.241 0.165 0.3167 0.26 RIDGEVIF 0.4273 0.3576 0.232 0.161 0.3030 0.27 RIDGEVIF 0.4086 0.3432 0.225 0.157 0.2905 0.28 RIDGEVIF 0.3915 0.3300 0.218 0.154 0.2792 0.29 RIDGEVIF 0.3758 0.3179 0.211 0.151 0.2689 0.30 RIDGEVIF 0.3614 0.3068 0.205 0.148 0.2594 RIDGE REGRESSION for apple taste - YOURNAME 5 USING Inv(Z`Z+k*I) INSTEAD OF Inv(Z`*Z) IN PARAMETER ESTIMATES RIDGE REGRESSION USING PROC IML: 02:37 Tuesday, November 1, 2005 XX is the n by r+1 design matrix for r covariates XXC is the n by r design matrix for covariates centered at column means with the intercept column dropped ZZ is the n by r normalized centered design matrix, so that ZPZ=ZZ'*ZZ is the correlation matrix of the r covariates. N R P 14 5 6 Beta coefficients and RMSE=Root(MSE) using both XX and XXC NAMES BETAXX BETAXC RMSEX RMSEC Sodium -10.7723 -10.7723 247.820 247.820 Potassium 43.6183 43.6183 Phosphorus 0.3449 0.3449 Shade -179.0998 -179.0998 Water 1.7524 1.7524 ZPZ for centered and normalized design matrix (ZPZ is the correlation matrix of the covariates.) ZPZ 1.00 0.95 -0.14 0.60 -0.15 0.95 1.00 -0.17 0.58 -0.19 -0.14 -0.17 1.00 0.70 0.98 0.60 0.58 0.70 1.00 0.67 -0.15 -0.19 0.98 0.67 1.00 ZPZINV 26.22 21.11 44.67 -63.90 7.21 21.11 80.38 88.35 -137.34 24.57 44.67 88.35 144.71 -181.57 4.17 -63.90 -137.34 -181.57 274.07 -42.62 7.21 24.57 4.17 -42.62 31.41 Beta coefficients using XX and ZZ and Variances/MSE NAMES BETAXX BETAZZ RMSEX RMSEZ ZVAR Sodium -10.7723 -10.7723 247.820 247.820 26.215 Potassium 43.6183 43.6183 80.378 Phosphorus 0.3449 0.3449 144.711 Shade -179.0998 -179.0998 274.066 Water 1.7524 1.7524 31.408 Eigenvalues and Eigenvectors of ZPZ: EIGENZ NAMES EIGENVECTORS 2.64996 Sodium 0.300 0.568 0.738 -0.120 0.171 2.27738 Potassium 0.281 0.582 -0.606 0.272 0.377 0.04931 Phosphorus 0.485 -0.402 -0.100 -0.574 0.514 0.02130 Shade 0.607 0.094 -0.177 -0.189 -0.745 0.00205 Water 0.477 -0.410 0.219 0.739 0.097 RIDGE REGRESSION for apple taste - YOURNAME 6 USING Inv(Z`Z+k*I) INSTEAD OF Inv(Z`*Z) IN PARAMETER ESTIMATES RIDGE REGRESSION USING PROC IML: 02:37 Tuesday, November 1, 2005 RIDGE TRACE (RIDGE REGRESSION OUTPUT): (ZVAR = variance of parameter estimate/MSE) (SSEFAC = Error SS/Full model Error SS.) STOP as soon as the estimates seem to stabilize and the ZVARs do not become too small Eigenvalues, beta coefficients, and Zvars using ZZ`ZZ+kkr*I EIGENV NAMES KKR BETAK ZVAR RMSE SSEFAC 2.650 Sodium 0.000 -10.77 26.22 247.82 1.000 2.277 Potassium 43.62 80.38 0.049 Phosphorus 0.34 144.71 0.021 Shade -179.10 274.07 0.002 Water 1.75 31.41 Eigenvalues, beta coefficients, and Zvars using ZZ`ZZ+kkr*I EIGENV NAMES KKR BETAK ZVAR RMSE SSEFAC 2.660 Sodium 0.010 -16.80 8.52 249.21 1.011 2.287 Potassium 26.16 8.93 0.059 Phosphorus 0.18 11.17 0.031 Shade 263.66 9.19 0.012 Water 1.35 12.85 Eigenvalues, beta coefficients, and Zvars using ZZ`ZZ+kkr*I EIGENV NAMES KKR BETAK ZVAR RMSE SSEFAC 2.670 Sodium 0.020 -12.42 6.06 250.20 1.019 2.297 Potassium 22.51 5.46 0.069 Phosphorus 0.16 5.48 0.041 Shade 298.29 3.25 0.022 Water 1.30 7.52 Eigenvalues, beta coefficients, and Zvars using ZZ`ZZ+kkr*I EIGENV NAMES KKR BETAK ZVAR RMSE SSEFAC 2.680 Sodium 0.030 -8.71 4.61 251.14 1.027 2.307 Potassium 20.31 3.93 0.079 Phosphorus 0.16 3.42 0.051 Shade 307.88 1.78 0.032 Water 1.29 4.98 Eigenvalues, beta coefficients, and Zvars using ZZ`ZZ+kkr*I EIGENV NAMES KKR BETAK ZVAR RMSE SSEFAC 2.690 Sodium 0.040 -5.74 3.65 252.03 1.034 2.317 Potassium 18.73 3.02 0.089 Phosphorus 0.15 2.39 0.061 Shade 310.89 1.18 0.042 Water 1.28 3.56 RIDGE REGRESSION for apple taste - YOURNAME 7 USING Inv(Z`Z+k*I) INSTEAD OF Inv(Z`*Z) IN PARAMETER ESTIMATES RIDGE REGRESSION USING PROC IML: 02:37 Tuesday, November 1, 2005 Eigenvalues, beta coefficients, and Zvars using ZZ`ZZ+kkr*I EIGENV NAMES KKR BETAK ZVAR RMSE SSEFAC 2.700 Sodium 0.050 -3.35 2.97 252.86 1.041 2.327 Potassium 17.52 2.42 0.099 Phosphorus 0.15 1.78 0.071 Shade 311.47 0.86 0.052 Water 1.28 2.69 Eigenvalues, beta coefficients, and Zvars using ZZ`ZZ+kkr*I EIGENV NAMES KKR BETAK ZVAR RMSE SSEFAC 2.710 Sodium 0.060 -1.39 2.47 253.64 1.048 2.337 Potassium 16.55 2.00 0.109 Phosphorus 0.15 1.39 0.081 Shade 310.97 0.68 0.062 Water 1.28 2.12 Eigenvalues, beta coefficients, and Zvars using ZZ`ZZ+kkr*I EIGENV NAMES KKR BETAK ZVAR RMSE SSEFAC 2.720 Sodium 0.070 0.24 2.10 254.36 1.053 2.347 Potassium 15.76 1.68 0.119 Phosphorus 0.15 1.13 0.091 Shade 309.96 0.55 0.072 Water 1.29 1.72 Eigenvalues, beta coefficients, and Zvars using ZZ`ZZ+kkr*I EIGENV NAMES KKR BETAK ZVAR RMSE SSEFAC 2.730 Sodium 0.080 1.61 1.81 255.04 1.059 2.357 Potassium 15.09 1.45 0.129 Phosphorus 0.15 0.94 0.101 Shade 308.69 0.47 0.082 Water 1.29 1.43 Eigenvalues, beta coefficients, and Zvars using ZZ`ZZ+kkr*I EIGENV NAMES KKR BETAK ZVAR RMSE SSEFAC 2.740 Sodium 0.090 2.78 1.58 255.69 1.065 2.367 Potassium 14.52 1.26 0.139 Phosphorus 0.15 0.80 0.111 Shade 307.30 0.41 0.092 Water 1.29 1.21 Eigenvalues, beta coefficients, and Zvars using ZZ`ZZ+kkr*I EIGENV NAMES KKR BETAK ZVAR RMSE SSEFAC 2.750 Sodium 0.100 3.78 1.39 256.31 1.070 2.377 Potassium 14.03 1.11 0.149 Phosphorus 0.14 0.70 RIDGE REGRESSION for apple taste - YOURNAME 8 USING Inv(Z`Z+k*I) INSTEAD OF Inv(Z`*Z) IN PARAMETER ESTIMATES RIDGE REGRESSION USING PROC IML: 02:37 Tuesday, November 1, 2005 EIGENV NAMES KKR BETAK ZVAR RMSE SSEFAC 0.121 Shade 305.86 0.36 0.102 Water 1.29 1.05 Eigenvalues, beta coefficients, and Zvars using ZZ`ZZ+kkr*I EIGENV NAMES KKR BETAK ZVAR RMSE SSEFAC 2.760 Sodium 0.110 4.66 1.24 256.90 1.075 2.387 Potassium 13.60 0.99 0.159 Phosphorus 0.14 0.62 0.131 Shade 304.40 0.33 0.112 Water 1.29 0.92 Eigenvalues, beta coefficients, and Zvars using ZZ`ZZ+kkr*I EIGENV NAMES KKR BETAK ZVAR RMSE SSEFAC 2.770 Sodium 0.120 5.42 1.11 257.48 1.079 2.397 Potassium 13.22 0.89 0.169 Phosphorus 0.14 0.55 0.141 Shade 302.95 0.30 0.122 Water 1.29 0.81 Eigenvalues, beta coefficients, and Zvars using ZZ`ZZ+kkr*I EIGENV NAMES KKR BETAK ZVAR RMSE SSEFAC 2.850 Sodium 0.200 9.15 0.59 261.87 1.117 2.477 Potassium 11.22 0.48 0.249 Phosphorus 0.14 0.30 0.221 Shade 292.08 0.19 0.202 Water 1.27 0.41 Eigenvalues, beta coefficients, and Zvars using ZZ`ZZ+kkr*I EIGENV NAMES KKR BETAK ZVAR RMSE SSEFAC 2.950 Sodium 0.300 11.12 0.36 267.55 1.166 2.577 Potassium 9.91 0.31 0.349 Phosphorus 0.13 0.20 0.321 Shade 280.41 0.15 0.302 Water 1.24 0.26