Homework 1 - Solutions. Problem 2

Size: px

Start display at page:

Download "Homework 1 - Solutions. Problem 2"

Warren Mason
5 years ago
Views:

1 Homework 1 - Solutions Problem 2 a) soma<-read.table("/users/basecamp/teaching/2101/bgsall.txt",he=t) attach(soma) soma.girls<-soma[sex==1,] names(soma.girls)=c("sex.g","wt2.g","ht2.g","wt9.g","ht9.g","lg9.g","st9.g", "WT18.g","HT18.g","LG18.g","ST18.g","Soma.g") attach(soma.girls) > library(stats) > round(cor(soma.girls[2:11]),3) WT2.g HT2.g WT9.g HT9.g LG9.g ST9.g WT18.g HT18.g LG18.g ST18.g WT2.g HT2.g WT9.g HT9.g LG9.g ST9.g WT18.g HT18.g LG18.g ST18.g > round(colmeans(soma.girls[2:11]),3) WT2.g HT2.g WT9.g HT9.g LG9.g ST9.g WT18.g HT18.g LG18.g ST18.g > t(lapply(soma.girls[,2:11],sd)) WT2.g HT2.g WT9.g HT9.g LG9.g ST9.g WT18.g HT18.g LG18.g ST18.g \begin{verbatim} soma.boys<-soma[sex==0,] names(soma.boys)=c("sex.b","wt2.b","ht2.b","wt9.b","ht9.b","lg9.b","st9.b", 1

2 "WT18.b","HT18.b","LG18.b","ST18.b","Soma.b") attach(soma.boys) fit1.g<-lm(soma.g~ht2.g+wt2.g+ht9.g+wt9.g+st9.g) summary(fit1.g) lm(formula = Soma.g ~ HT2.g + WT2.g + HT9.g + WT9.g + ST9.g) (Intercept) *** HT2.g * WT2.g HT9.g WT9.g e-08 *** ST9.g Signif. codes: 0 *** ** 0.01 * Residual standard error: on 6 degrees of freedom Multiple R-Squared: 0.211, F-statistic: on and 6 DF, p-value: 3.309e-09 postscript("soma-fig1.ps") plot(fit1.g) ˆσ = and R 2 = Confidence for any β i is constructed using the quantiles of the Student distribution t 6,0.02 = and t 6,0.97 = Therefore, CI j = [β j sd j, β j sd j ] for any 0 j. The diagnostic plots are shown in Figure 1. 2

3 Residuals vs Fitted Normal Q Q Residuals Theoretical Quantiles Scale Location Residuals vs Leverage Cook s distance Leverage Figure 1: Plots for fit1.g c) fit2.g<-lm(soma.g~ht9.g+wt9.g+st9.g) summary(fit2.g) lm(formula = Soma.g ~ HT9.g + WT9.g + ST9.g) (Intercept) ** HT9.g WT9.g e-09 *** ST9.g Signif. codes: 0 *** ** 0.01 * Residual standard error: 0.60 on 66 degrees of freedom 3

4 Residuals vs Fitted Normal Q Q Residuals Theoretical Quantiles Scale Location 32 Residuals vs Leverage Cook s distance Leverage Figure 2: Plots for fit2.g Multiple R-Squared: 0.601,Adjusted R-squared: 0.36 F-statistic: 18.7 on 3 and 66 DF, p-value: 6.61e-09 postscript("soma-fig2.ps") plot(fit2.g) postscript("soma-fig3.ps") plot(fit2.g$fitted,fit2.g$resid,pch=as.character(soma.g)) One can see that confidence intervals are slightly narrower for the second model (larger number of degrees of freedom). t 66,0.02 = and t 6,0.97 = Akaike Information Criterion favors the first model. However, notice that both model should not have an intercept (Why?). If we eliminate the intercept from both models we can immediately notice that the second model is more appropriate. fit.g<-lm(soma.g~ht2.g+wt2.g+ht9.g+wt9.g+st9.g-1)

5 fit2.g$resid fit2.g$fitted Figure 3: Residual Plot for fit2.g with residuals identified using response values. summary(fit.g) lm(formula = Soma.g ~ HT2.g + WT2.g + HT9.g + WT9.g + ST9.g - 1) HT2.g WT2.g HT9.g * WT9.g e-06 *** ST9.g * Signif. codes: 0 *** ** 0.01 * Residual standard error: on 6 degrees of freedom Multiple R-Squared: 0.981,Adjusted R-squared: F-statistic: 80.8 on and 6 DF, p-value: < 2.2e-16

6 fit3.g<-lm(soma.g~ht9.g+wt9.g+st9.g-1) summary(fit3.g) lm(formula = Soma.g ~ HT9.g + WT9.g + ST9.g - 1) HT9.g e-06 *** WT9.g e-07 *** ST9.g * Residual standard error: on 67 degrees of freedom Multiple R-Squared: ,Adjusted R-squared: F-statistic: 1318 on 3 and 67 DF, p-value: < 2.2e-16 postscript("soma-fig.ps") plot(fit3.g$fitted,fit3.g$resid,pch=as.character(soma.g)) d) fit1.b<-lm(soma.b~ht2.b+wt2.b+ht9.b+wt9.b+st9.b) summary(fit1.b) lm(formula = Soma.b ~ HT2.b + WT2.b + HT9.b + WT9.b + ST9.b)

7 fit3.g$resid fit3.g$fitted Figure : Residual Plot for fit3.g with residuals identified using response values. (Intercept) ** HT2.b WT2.b ** HT9.b WT9.b e-07 *** ST9.b ** Signif. codes: 0 *** ** 0.01 * Residual standard error: on 60 degrees of freedom Multiple R-Squared: ,Adjusted R-squared: 0.33 F-statistic: 7.6 on and 60 DF, p-value: 1.9e-0 postscript("soma-fig.ps") plot(fit1.b) fit2.b<-lm(log(soma.b)~ht2.b+wt2.b+ht9.b+wt9.b+st9.b) summary(fit2.b) 7

8 lm(formula = log(soma.b) ~ HT2.b + WT2.b + HT9.b + WT9.b + ST9.b) (Intercept) ** HT2.b WT2.b ** HT9.b WT9.b e-06 *** ST9.b ** Signif. codes: 0 *** ** 0.01 * Residual standard error: on 60 degrees of freedom Multiple R-Squared: ,Adjusted R-squared: F-statistic: on and 60 DF, p-value:.08e-0 postscript("soma-fig6.ps") plot(fit2.b) fit3.b<-lm(soma.b~ht9.b+wt9.b+st9.b) summary(fit3.b) lm(formula = Soma.b ~ HT9.b + WT9.b + ST9.b)

9 (Intercept) ** HT9.b * WT9.b e-0 *** ST9.b ** Signif. codes: 0 *** ** 0.01 * Residual standard error: 1.26 on 62 degrees of freedom Multiple R-Squared: 0.272,Adjusted R-squared: F-statistic: on 3 and 62 DF, p-value: postscript("soma-fig7.ps") plot(fit3.b) fit.b<-lm(soma.b~ht9.b+wt9.b+st9.b-1) summary(fit.b) lm(formula = Soma.b ~ HT9.b + WT9.b + ST9.b - 1) HT9.b * WT9.b ** ST9.b ** Signif. codes: 0 *** ** 0.01 * Residual standard error: on 63 degrees of freedom Multiple R-Squared: 0.8,Adjusted R-squared: 0.87 F-statistic: on 3 and 63 DF, p-value: < 2.2e-16 9

10 postscript("soma-fig8.ps") plot(fit.b) fit.b<-lm(log(soma.b)~ht9.b+wt9.b+st9.b-1) summary(fit.b) lm(formula = log(soma.b) ~ HT9.b + WT9.b + ST9.b - 1) HT9.b ** WT9.b ** ST9.b ** Signif. codes: 0 *** ** 0.01 * Residual standard error: 0.2 on 63 degrees of freedom Multiple R-Squared: 0.861,Adjusted R-squared: F-statistic: 11. on 3 and 63 DF, p-value: < 2.2e-16 postscript("soma-fig9.ps") plot(fit.b) e) fit.a<-lm(ht18~wt2) fit.b<-lm(wt9~wt2) postscript("soma-fig10.ps") 10

11 fit.a$resid fit.b$resid Figure : The added variable plot plot(fit.b$resid,fit.a$resid) The added variable plot shows no linear pattern. The lack of dependence is confirmed by the lack of statistical significance for the coefficient of WT9 in the model below. add.fit<-lm(fit.a$resid~fit.b$resid) summary(add.fit) lm(formula = fit.a$resid ~ fit.b$resid) (Intercept) 1.009e e e fit.b$resid 3.116e e

12 Residual standard error: 7.76 on 13 degrees of freedom Multiple R-Squared: ,Adjusted R-squared: F-statistic: on 1 and 13 DF, p-value: fit.c<-lm(ht18~wt2+wt9) summary(fit.c) lm(formula = HT18 ~ WT2 + WT9) (Intercept) < 2e-16 *** WT e-06 *** WT Signif. codes: 0 *** ** 0.01 * Residual standard error: 7.78 on 133 degrees of freedom Multiple R-Squared: ,Adjusted R-squared: F-statistic: on 2 and 133 DF, p-value: 1.83e-08 Problem 3 brake<-read.table("/users/basecamp/teaching/2101/stopping.txt",he=t) names(brake)=c("speed","distance") attach(brake) fit<-lm(distance~speed) summary(fit) lm(formula = Distance ~ Speed) 12

13 Residuals vs Fitted Normal Q Q Residuals Theoretical Quantiles Scale Location Residuals vs Leverage Cook s distance Leverage Figure 6: Plots for the model Distance = β 0 + β 1 Speed (Intercept) e-08 *** Speed < 2e-16 *** Signif. codes: 0 *** ** 0.01 * Residual standard error: on 60 degrees of freedom Multiple R-Squared: ,Adjusted R-squared: F-statistic: 30.6 on 1 and 60 DF, p-value: < 2.2e-16 postscript("speed-fig1.ps") plot(fit) The plots indicate that the assumptions about the noise structure are violated. b) We start looking to transform X using the method presented in class. 13

14 Speed2<-Speed*log(Speed) fit1<-lm(distance~speed2) summary(fit1) lm(formula = Distance ~ Speed2) (Intercept) ** Speed < 2e-16 *** Signif. codes: 0 *** ** 0.01 * Residual standard error: 10.8 on 60 degrees of freedom Multiple R-Squared: 0.897,Adjusted R-squared: F-statistic: 22.6 on 1 and 60 DF, p-value: < 2.2e-16 The suggested transformation is either X 2 or X 3/2. Since X 2 is used in d) we look at the second now. Speed3<-Speed^(3/2) fit2<-lm(distance~speed3-1) summary(fit2) lm(formula = Distance ~ Speed3-1) Speed <2e-16 *** Signif. codes: 0 *** ** 0.01 *

15 Residuals vs Fitted Normal Q Q Residuals Theoretical Quantiles Scale Location Residuals vs Leverage Cook s distance Leverage Figure 7: Plots for the model Distance = β 0 + β 1 Speed 3/2. Residual standard error: 10.3 on 61 degrees of freedom Multiple R-Squared: ,Adjusted R-squared: F-statistic: 172 on 1 and 61 DF, p-value: < 2.2e-16 postscript("speed-fig2.ps") plot(fit2) c) lambda<-seq(-2,2,0.11) n<-length(speed) a<-lambda for(i in 1:length(lambda)) { y.new<-((distance^lambda[i]-1)/lambda[i]) rss<-sum((lm(y.new~speed)$resid)^2) a[i]<-(-n/2)*log(rss) + (lambda[i]-1)*sum(log(distance)) } postscript("speed-fig3.ps") plot(lambda,a) 1

16 a lambda Figure 8: Plot for the Box-Cox Transformation. It suggests Distance ## Box-Cox suggest Distance.new<-sqrt(Distance) Distance.new<-sqrt(Distance) fit3<-lm(distance.new~speed-1) summary(fit3) lm(formula = Distance.new ~ Speed - 1) Speed <2e-16 *** Signif. codes: 0 *** ** 0.01 * Residual standard error: on 61 degrees of freedom Multiple R-Squared: 0.982,Adjusted R-squared:

17 Residuals vs Fitted Normal Q Q Residuals Theoretical Quantiles Scale Location Residuals vs Leverage Cook s distance Leverage Figure 9: Plots for the model Distance = β 1 Speed. F-statistic: 319 on 1 and 61 DF, p-value: < 2.2e-16 postscript("speed-fig.ps") plot(fit3) d) sq.speed<-speed^2 inv.sq.speed<-1/sq.speed fit<-lm(distance~speed+sq.speed-1,weights=inv.sq.speed) summary(fit) lm(formula = Distance ~ Speed + sq.speed - 1, weights = inv.sq.speed)

18 Residuals vs Fitted Normal Q Q Residuals Theoretical Quantiles Scale Location Residuals vs Leverage Cook s distance Leverage Figure 10: Plots for the model Distance = β 1 Speed + β 2 Speed 2. Speed e-06 *** sq.speed e-1 *** Signif. codes: 0 *** ** 0.01 * Residual standard error: 0.97 on 60 degrees of freedom Multiple R-Squared: 0.968,Adjusted R-squared: 0.91 F-statistic: 3.3 on 2 and 60 DF, p-value: < 2.2e-16 postscript("speed-fig.ps") plot(fit) Problem landrent<-read.table("/users/basecamp/teaching/2101/landrent.txt",he=t) attach(landrent) fit.full<-lm(y~x1+x2+x3+x) summary(fit.full) 18

19 lm(formula = Y ~ X1 + X2 + X3 + X) (Intercept) X < 2e-16 *** X *** X X Signif. codes: 0 *** ** 0.01 * Residual standard error: on 62 degrees of freedom Multiple R-Squared: 0.80,Adjusted R-squared: F-statistic: 81.6 on and 62 DF, p-value: < 2.2e-16 postscript("landrent-fig1.ps") plot(fit.full) #uneven variances ee<-fit.full$resid A<-model.matrix(fit.full) A[,1]<-ee pairs(a) n<-length(y) a<-lambda for(i in 1:length(lambda)) { 19

20 y.new<-((y^lambda[i]-1)/lambda[i]) rss<-sum((lm(y.new~x1+x2+x3+x)$resid)^2) a[i]<-(-n/2)*log(rss) + (lambda[i]-1)*sum(log(y)) } postscript("landrent-fig2.ps") plot(lambda,a) Y.new<-sqrt(Y) fit2<-lm(y.new~x1+x2+x3+x) summary(fit2) lm(formula = Y.new ~ X1 + X2 + X3 + X) (Intercept) e-09 *** X < 2e-16 *** X e-06 *** X X Signif. codes: 0 *** ** 0.01 * Residual standard error: on 62 degrees of freedom Multiple R-Squared: ,Adjusted R-squared:

21 F-statistic: 10. on and 62 DF, p-value: < 2.2e-16 postscript("landrent-fig3.ps") plot(fit2) ee<-fit2$resid A<-model.matrix(fit2) A[,1]<-ee pairs(a) fit3<-lm(y.new~x1+x2) summary(fit3) lm(formula = Y.new ~ X1 + X2) (Intercept) < 2e-16 *** X < 2e-16 *** X e-08 *** Signif. codes: 0 *** ** 0.01 * Residual standard error: 0.6 on 6 degrees of freedom Multiple R-Squared: 0.868,Adjusted R-squared: 0.86 F-statistic: 211. on 2 and 6 DF, p-value: < 2.2e-16 postscript("landrent-fig.ps") plot(fit3) Problem Define X, Y the original data and X, Ỹ the collapsed data (instead of replicates we consider their 21

22 average, instead of having each row repeated times in X we have only one in X. We define the matrix A such that AX = X and AY = Ỹ. library(mvtnorm) n<- 2 k<- no.replic<-100 e.k<-rep(1/k,k) A<-kronecker(diag(1,n),t(e.k)) sam.red<-matrix(0,nrow=no.replic,ncol=3) sam.unred<-matrix(0,nrow=no.replic,ncol=3) sigma.red<-c() sigma.unred<-c() beta.red<-matrix(0,nrow=no.replic,ncol=3) beta.unred<-matrix(0,nrow=no.replic,ncol=3) for(r in 1:no.replic) { #a<-matrix(rnorm(n,mean=0,sd=2),ncol=2) ; a<-cbind(rep(1,n*k),a) X1<-rep(1,n*k) X2<-rep(rnorm(n,mean=1),rep(k,n)) X3<-rep(rnorm(n, mean=0),rep(k,n)) #X<-rep(rnorm(n,mean=2),rep(k,n)) X<-cbind(X1,X2,X3) Y<-rmvnorm(1,mean=X%*%c(1,2,3),sigma=diag(1,n*k)) sam.red[r,]<-diag(solve(t(x)%*%t(a)%*%a%*%x))/k sam.unred[r,]<-diag(solve(t(x)%*%x)) sol.unred<-solve(t(x)%*%x)%*%t(x)%*%y[1,] beta.unred[r,]<-t(sol.unred[,1]) sigma.unred[r]<-sum((t(y)-x%*%t(t(beta.unred[r,])))^2)/(n*k-3) 22

23 sol.red<-solve(t(x)%*%t(a)%*%a%*%x)%*%t(x)%*%t(a)%*%a%*%y[1,] beta.red[r,]<-t( sol.red[,1]) sigma.red[r]<-sum((t(y)-x%*%t(t(beta.red[r,])))^2)/(n-3) } sd.red<-sqrt(sigma.red) sd.unred<-sqrt(sigma.unred) sqrt(var(sd.red)) sqrt(var(sd.unred))

wine 1 wine 2 wine 3 person person person person person

wine 1 wine 2 wine 3 person person person person person 1. A trendy wine bar set up an experiment to evaluate the quality of 3 different wines. Five fine connoisseurs of wine were asked to taste each of the wine and give it a rating between 0 and 10. The order