To: Professor Roger Bohn & Hyeonsu Kang Subject: Big Data, Assignment April 13th. From: xxxx (anonymized) Date: 4/11/2016

Transcription

1 To: Professor Roger Bohn & Hyeonsu Kang Subject: Big Data, Assignment April 13th. From: xxxx (anonymized) Date: 4/11/2016 Data Preparation: 1. Separate trany variable into Manual which takes value of 1 and 0 (1 means manual, 0 means automatic), and Speed that takes category of 3-speed, 4 speed and 5 speed. 2. Change variable tcharger and scharger to dummy variables that take value of 1 and 0 (1 means tcharger, 0 means not tcharger; same with scharger ) 3. Convert displ into gallon instead of liter. 4. Convert year variable in to dummy variable after 2014 which indicates whether it is before or after Generate an interaction term using after 2014 * Manual Part A: Summary: Our target is MPG. While including many factors that we think are important to MPG, we also tried different model variations such as adding interaction terms manual ## after 2014, subtracting variables one by one. Our conclusion is that variables that included in Model 1 explains MPG the best, also the interpretation of the result makes sense in the real world. Model 1: Based on my understanding of the topic, I believe there are several important factors that affects the MPG of a car: # of cylinders, engine displacement, drive type, fuel type, manual or automatic, year made, turbocharged or supercharged. Unit: all relevant variable in the model is in the unit of gallon. Target: combo08 Input: cylinders, displ_gal, drive, fueltype1, manual, speed, year, tcharger, scharger Other parameters: seed: , partition: 70/30/0

2 MPG = * # of cylinders 7.25 * Engine displacement {0.58 if drive4-wheel Drive, 1.58 if drive4-wheel or All-Wheel Drive, if driveall-wheel Drive, if drivefront- Wheel Drive, 1.77 if drive Part-time 4-Wheel Drive, 0.17 if driverear-wheel Drive} {5.19 if fuletype1 Midgrade, 7.85 if fueltype1nuaturalgas, 6.99 if fueltype1premium, 7.1 if fueltype1regular} + {0.69 if manual, 0 if automatic} * # of year {1.19 if turbocharged, 0 if not} {0.84 if supercharged, 0 if not} This training output shows that all variables included in the model are significant except the drive type driverear-wheel Drive and driveall-wheel Drive. In addition, # of cylinders, Engine displacement, drive types, fueltype, turbocharged, and supercharged are all negatively associated with MPG; while # of year and transmission type is positively associated with MPG. Evaluation: The result actually makes sense, as technology getting more and more advanced, the cars that were made more recently will have higher MPG. Also, manual cars on average have higher MPG than automatic cars. The Pseudo R-square is 0.80, which means 80% of the variation in y can explained by all the predictors. Figure 1 Model 2: Considering the fact that far more cars in 2014 are automatic than manual, but that may not be true at the start of the data set, we generated a after 2014 variable which denotes the cars

3 are produced after 2014 if it is 1, and before 2014 if it is 0. We created an interaction term of after 2014 * manual, to see is there an additional effect of being manual car that were produced after Unit: all relevant variable in the model is in the unit of gallon. Target: combo08 Input: cylinders, displ_gal, drive, fueltype1, manual, year after 2014, tcharger, scharger, year after 2014##manual Other parameters: seed: , partition: 70/30/0 MPG = * # of cylinders 6.24 * Engine displacement + {1.35 if drive4-wheel Drive, if drive4-wheel or All-Wheel Drive, 1.66 if driveall-wheel Drive, 3.43 if drivefront-wheel Drive, if drive Part-time 4-Wheel Drive, 0.17 if driverear-wheel Drive} + {0.55 if manual, 0 if automatic} + {3.28 if year after 2014, 0 if before 2014} {0.46 if turbocharged, 0 if not} {0.68 if supercharged, 0 if not} + {-0.19 if manual after 2014, 0 if not manual after 2014} Evaluation: As Figure 2 shows, the Pseudo R-square is 0.66, which is lower than Model 1. In addition, reading the actual regression equation for Model 2 above, the interpretation doesn t make as much sense as Model 1. Especially for the coefficient on the interaction term manual after 2014, it is hard to comprehend why there is a negative effect. Figure 2:

4 Part B: Summary: The target is Manual. We comparing different models trying to find out the one that has the highest accuracy prediction percentage based on confusion matrix. We conclude that Model 2 including Speed and vclass is the best model based on analysis, which can correctly predict 92% of validation dataset. Model 1: Unit: all relevant variable in the model is in the unit of gallon. Target: Manual Input: city08, Co2TailpipeGpm, cylinders, displ_gal, drive, fueltype1, year, highway08, tcharger, scharger Other parameters: seed: , partition: 70/30/0 Manual (0,1) = β1 * mpg in city + β2 * Co2TailpipeGpm + β3 * # OF CYLINDERS + β4 * displacement in gallon + β5* drive type + β6 * fuel type + β7 * year + β8 * mgp in highway + β9 * turbo charged + β10 * super changed Coefficients Table:

5 Table 1 and 2 show that the model can correctly predict 7003 automatic cases, which is 63% of all the validation data; and it can also correctly predict 947 manual cases, which is 9% of all the validation data. For the rest of 28%, the model failed to predict. Table 1 Confusion Matrix of Model 1 Prediction on Validation dataset (count) Predicted Automatic Manual Actual Automatic Manual Table 2 Confusion Matrix of Model 1 Prediction on Validation dataset (percentage) Predicted Error Rate Automatic Manual Actual Automatic Manual Model 2: We include speed and vclass variable, while dropping drive and fueltype1. Unit: all relevant variable in the model is in the unit of gallon. Target: Manual Input: city08, Co2TailpipeGpm, cylinders, displ_gal, drive, fueltype1, year after 2014, highway08, tcharger, scharger Other parameters: seed: , partition: 70/30/0 Manual (0,1) = β1 * mpg in city + β2 * Co2TailpipeGpm + β3 * # OF CYLINDERS + β4 * displacement in gallon + β5i* speed type + β6i * EPA vehicle size class + β7 * year + β8 * mgp in highway + β9 * turbo charged + β10 * super changed

6

7 *As we can see from the coefficient table, the Speed variable is insignificant in the model. Thus, we tried to exclude Speed; however, in the confusion matrix showed a dramatic accuracy percentage drop. Thus, we decide to keep it in our model. Table 3 and 4 show that the model can correctly predict 6698 automatic cases, which is 62% of all the validation data; and it can also correctly predict 3227 manual cases, which is 30% of all the validation data. For the rest of 8%, the model failed to predict. The accuracy rate is higher than the previous model. Table 3 Confusion Matrix of Model 2 Prediction on Validation dataset (count) Predicted Automatic Manual Actual Automatic Manual Table 4 Confusion Matrix of Model 2 Prediction on Validation dataset (percentage) Predicted Error Rate Automatic Manual Actual Automatic Manual

8 Part C: Summary: After running a linear regression and classifying cars according to the transmission type, we want to create a model that determines how the variables interact with other variables to play an effect on determining the outcome of the combined MPG for fuel type 1. The inclusion of these interaction terms will provide a more accurate model that assesses the relationship between the inputs and target variable. Specifically, the age of the car should be interacted with the input variables because cars are engineered to become more efficient over time. For our working model, we categorized the years into decades, where the variable names are: eighties, nineties, twothousand, and twoten. Respectively, these variables translate into these values: values from the 1980s, values from the 1990s, values from the 2000s, and values from the 2010s. Model without interaction terms Unit: all relevant variable in the model is in the unit of gallon. Target: combo08 Input: cylinders, displ_gal, drive, fueltype1, manual, year dummies, tcharger, scharger Other parameters: seed: 12345, partition: 70/30/0 Before we create models with interacted terms, we ran a model with the created year dummy variables. We will use the results from this model as our baseline case to compare our interacted models. The Rattle output is shown in Appendix A while the graphical results are shown below: A) Graphical depiction of the data distribution

9 B) Predicted vs. Observed Model

10 Model with year##manual Next, we included the interaction term between years and whether the car runs on manual transmission or not. This is an important interaction to observe because we can hypothesize that cars that manufactured in more recent years are less likely to be run on manual transmission. Thus, this pattern could be correlated with patterns observed in MPG. The model is as follows: comb08=b 0 +city08+b 1 co2tailpipegpm+ B 2 cylinders + B 3 displ_ga+ B 4 drive +.B K eighties*manual +B K+1 nineties*manual + B k+2 twothousand*manual+ B k+3 twoten*manual When the model was placed into Rattle, it produced the following results: 1 A) Predicted vs. Observed Model 1 The summary of the linear regression model is shown in Appendix B.

11 The model shows the Pseudo R-squared is.9952, which shows that the observed points fit the predicted model very well. Also, the regression analysis shows that two of the interactions showed statistical significance, implying that they have some effect on determining the target variable. Model with year##pv4 In addition, we created an interaction term between the 4-door passenger volume and year. This is an important interaction to observe because we hypothesize that cars with greater volume would have lower fuel efficiency as the car would need to move more weight. comb08=b 0 +city08+b 1 co2tailpipegpm+ B 2 cylinders + B 3 displ_ga+ B 4 drive +.B K eighties*pv4 +B K+1 nineties* pv4 + B k+2 twothousand* pv4+ B k+3 twoten* pv4 Rattle produced the following results: 2 A) Predicted vs. Observed Model 2 The summary of the linear regression model is shown in Appendix C.

12 While the pseudo r-squared stayed the same, including this particular interaction term changed the B coefficients for the model. However, none of the coefficients of the interaction terms showed statistical significance, implying that the interaction between the terms did not have an effect on the target variable. Model with year##co2tailpipgpm For our third interaction term, we interacted tailpipe CO2 in grams/mile and year. This is an important interaction to observe because newer cars that face higher emission standards tend to have lower emission of tailpipe CO2. We want to observe if this decrease in emission of CO2 also reflects a relationship with better MPG. comb08=b 0 +city08+b 1 co2tailpipegpm+ B 2 cylinders + B 3 displ_ga+ B 4 drive +.B K eighties* co2tailpipgpm +B K+1 nineties* co2tailpipgpm + B k+2 twothousand* co2tailpipgpm + B k+3 twoten* co2tailpipgpm Rattle produced the following results: 3 A) Predicted vs. Observed Model 3 The summary of the linear regression model is shown in Appendix D.

13 Once again, the r-squared value stayed the same, reflecting that the observed data points fit the predicted model well. In this model, all interaction terms showed high statistical significance, implying that all interaction terms had an effect on the target variable. Other Issues: In addition to the inputs provided in the vehicle data from the U.S. Department of Energy, it would be useful to have information on whether the car has air conditioning and the other technological systems (sound system) given that they also expend energy and thus would also play a role in influencing the MPG efficiency. Furthermore, it would be useful to include the total weight of the car because we would hypothesize that heavier cars consume more energy to move the car.

14 Appendix A Call: lm(formula = comb08 ~., data = crs$dataset[crs$train, c(crs$input, crs$target)]) Residuals: Min 1Q Median 3Q Max Coefficients: (26 not defined because of singularities) Estimate Std. Error t value Pr(> t ) (Intercept) city < 2e-16 co2tailpipegpm e-12 cylinders e-11 displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga

15 displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ_ga displ NA NA NA NA drive4-wheel Drive drive4-wheel or All-Wheel Drive driveall-wheel Drive drivefront-wheel Drive drivepart-time 4-Wheel Drive driverear-wheel Drive engid e-05 CA.model fuelcost e-05 fueltype1midgrade Gasoline e-15 fueltype1natural Gas e-10 fueltype1premium Gasoline fueltype1regular Gasoline < 2e-16 highway < 2e-16 pv tranyauto (AV-S8) tranyauto (AV) tranyautomatic (A1) tranyautomatic (A6) tranyautomatic (AM5) tranyautomatic (AV-S6) tranyautomatic (AV) tranyautomatic (S4) tranyautomatic (S5) tranyautomatic (S6) tranyautomatic (S7) tranyautomatic (S8) tranyautomatic (S9) tranyautomatic (variable gear ratios) tranyautomatic 3-spd

16 tranyautomatic 4-spd tranyautomatic 5-spd tranyautomatic 6-spd tranyautomatic 6spd tranyautomatic 7-spd tranyautomatic 8-spd tranyautomatic 9-spd tranymanual 3-spd tranymanual 4-spd tranymanual 5-spd tranymanual 6-spd tranymanual 7-spd Manual NA NA NA NA TransmissionAutomatic NA NA NA NA TransmissionManual NA NA NA NA Speed(A6) NA NA NA NA Speed(AM5) NA NA NA NA Speed(AV-S6) NA NA NA NA Speed(AV-S8) NA NA NA NA Speed(AV) NA NA NA NA Speed(S4) NA NA NA NA Speed(S5) NA NA NA NA Speed(S6) NA NA NA NA Speed(S7) NA NA NA NA Speed(S8) NA NA NA NA Speed(S9) NA NA NA NA Speed(variable NA NA NA NA Speed3-spd NA NA NA NA Speed4-spd NA NA NA NA Speed5-spd NA NA NA NA Speed6-spd NA NA NA NA Speed6spd NA NA NA NA Speed7-spd NA NA NA NA Speed8-spd NA NA NA NA Speed9-spd NA NA NA NA VClassLarge Cars VClassMidsize Cars VClassMidsize Station Wagons VClassMidsize-Large Station Wagons VClassMinicompact Cars VClassMinivan - 2WD VClassMinivan - 4WD VClassSmall Pickup Trucks VClassSmall Pickup Trucks 2WD VClassSmall Pickup Trucks 4WD VClassSmall Sport Utility Vehicle 2WD VClassSmall Sport Utility Vehicle 4WD VClassSmall Station Wagons VClassSpecial Purpose Vehicle VClassSpecial Purpose Vehicle 2WD VClassSpecial Purpose Vehicle 4WD VClassSpecial Purpose Vehicles VClassSpecial Purpose Vehicles/2wd VClassSpecial Purpose Vehicles/4wd VClassSport Utility Vehicle - 2WD VClassSport Utility Vehicle - 4WD

17 VClassStandard Pickup Trucks VClassStandard Pickup Trucks 2WD VClassStandard Pickup Trucks 4WD VClassStandard Pickup Trucks/2wd VClassStandard Sport Utility Vehicle 2WD VClassStandard Sport Utility Vehicle 4WD VClassSubcompact Cars VClassTwo Seaters VClassVans VClassVans Passenger VClassVans, Cargo Type VClassVans, Passenger Type after e-05 year yousavespend tcharger scharger decade Eighties Nineties Twothousand NA NA NA NA Twoten NA NA NA NA --- Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual standard error: on degrees of freedom (1463 observations deleted due to missingness) Multiple R-squared: , Adjusted R-squared: F-statistic: 3.394e+04 on 150 and DF, p-value: < 2.2e-16 ==== ANOVA ==== Analysis of Variance Table Response: comb08 Df Sum Sq Mean Sq F value Pr(>F) city < 2.2e-16 *** co2tailpipegpm < 2.2e-16 *** cylinders < 2.2e-16 *** displ_ga < 2.2e-16 *** drive < 2.2e-16 *** engid < 2.2e-16 *** CA.model e-14 *** fuelcost < 2.2e-16 *** fueltype < 2.2e-16 *** highway < 2.2e-16 *** pv * trany e-08 *** VClass e-10 *** after e-07 *** year * yousavespend ** tcharger ** scharger decade

18 Eighties e-06 *** Nineties Residuals Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 [1] "\n" Time taken: 1.81 secs Rattle timestamp: :30:01 seungkookang Appendix B Call: lm(formula = comb08 ~., data = crs$dataset[crs$train, c(crs$input, crs$target)]) Residuals: Min 1Q Median 3Q Max Coefficients: (2 not defined because of singularities) Estimate Std. Error t value Pr(> t ) (Intercept) e-13 *** city < 2e-16 *** co2tailpipegpm < 2e-16 *** cylinders * displ engid * camodel fuelcost e-06 *** highway < 2e-16 *** pv manual e-06 *** after ** yousavespend e-05 *** tcharger *** scharger eighties e-07 *** nineties ** twothousand *** twoten NA NA NA NA drive * vclass e-05 *** manual_eighties ** manual_nineties ** manual_twothousand manual_twoten NA NA NA NA --- Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual standard error: on degrees of freedom (913 observations deleted due to missingness) Multiple R-squared: 0.995, Adjusted R-squared: F-statistic: 2.238e+05 on 22 and DF, p-value: < 2.2e-16

19 ==== ANOVA ==== Analysis of Variance Table Response: comb08 Df Sum Sq Mean Sq F value Pr(>F) city < 2.2e-16 *** co2tailpipegpm < 2.2e-16 *** cylinders < 2.2e-16 *** displ *** engid < 2.2e-16 *** camodel < 2.2e-16 *** fuelcost < 2.2e-16 *** highway < 2.2e-16 *** pv ** manual *** after yousavespend *** tcharger ** scharger eighties * nineties twothousand *** drive vclass *** manual_eighties * manual_nineties ** manual_twothousand Residuals Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 [1] "\n" Time taken: 0.16 secs Rattle timestamp: :02:48 seungkookang Appendix C Call: lm(formula = comb08 ~., data = crs$dataset[crs$train, c(crs$input, crs$target)]) Residuals: Min 1Q Median 3Q Max Coefficients: (2 not defined because of singularities) Estimate Std. Error t value Pr(> t ) (Intercept) e-13 *** city < 2e-16 *** co2tailpipegpm < 2e-16 *** cylinders * displ

20 engid camodel fuelcost e-06 *** highway < 2e-16 *** pv manual e-07 *** after * yousavespend e-05 *** tcharger *** scharger eighties e-05 *** nineties * twothousand ** twoten NA NA NA NA drive * vclass e-05 *** pv4_eighties pv4_nineties pv4_twothousand pv4_twoten NA NA NA NA --- Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual standard error: on degrees of freedom (913 observations deleted due to missingness) Multiple R-squared: , Adjusted R-squared: F-statistic: 2.237e+05 on 22 and DF, p-value: < 2.2e-16 ==== ANOVA ==== Analysis of Variance Table Response: comb08 Df Sum Sq Mean Sq F value Pr(>F) city < 2.2e-16 *** co2tailpipegpm < 2.2e-16 *** cylinders < 2.2e-16 *** displ *** engid < 2.2e-16 *** camodel < 2.2e-16 *** fuelcost < 2.2e-16 *** highway < 2.2e-16 *** pv ** manual *** after yousavespend *** tcharger ** scharger eighties * nineties twothousand *** drive vclass *** pv4_eighties pv4_nineties pv4_twothousand

21 Residuals Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 [1] "\n" Time taken: 0.15 secs Rattle timestamp: :05:26 seungkookang ====================================================================== Appendix D Call: lm(formula = comb08 ~., data = crs$dataset[crs$train, c(crs$input, crs$target)]) Residuals: Min 1Q Median 3Q Max Coefficients: (2 not defined because of singularities) Estimate Std. Error t value Pr(> t ) (Intercept) < 2e-16 *** city < 2e-16 *** co2tailpipegpm < 2e-16 *** cylinders e-05 *** displ engid e-06 *** camodel fuelcost e-10 *** highway < 2e-16 *** pv manual e-08 *** after yousavespend e-10 *** tcharger ** scharger eighties < 2e-16 *** nineties < 2e-16 *** twothousand *** twoten NA NA NA NA drive ** co2_eighties < 2e-16 *** co2_nineties < 2e-16 *** co2_twothousand e-07 *** co2_twoten NA NA NA NA vclass e-08 *** --- Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual standard error: on degrees of freedom (913 observations deleted due to missingness) Multiple R-squared: 0.995, Adjusted R-squared: F-statistic: 2.253e+05 on 22 and DF, p-value: < 2.2e-16 ==== ANOVA ====

22 Analysis of Variance Table Response: comb08 Df Sum Sq Mean Sq F value Pr(>F) city < 2.2e-16 *** co2tailpipegpm < 2.2e-16 *** cylinders < 2.2e-16 *** displ e-08 *** engid < 2.2e-16 *** camodel < 2.2e-16 *** fuelcost < 2.2e-16 *** highway < 2.2e-16 *** pv ** manual e-09 *** after yousavespend e-05 *** tcharger ** scharger eighties * nineties twothousand *** drive co2_eighties < 2.2e-16 *** co2_nineties e-16 *** co2_twothousand e-06 *** vclass e-08 *** Residuals Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 [1] "\n" Time taken: 0.14 secs Rattle timestamp: :07:00 seungkookang ======================================================================