[This article was first published on NYC Data Science Academy » R, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
Check out my:
Source: http://archive.ics.uci.edu/ml/datasets/Student+Performance
This project is based upon two datasets of the academic performance of Portuguese students in two different classes: Math and Portuguese. Initially, I show the simplicity of predicting student performance using linear regression. Later, I show that it is still possible, yet more difficult, to predict the final grade without Period 1 and Period 2 grades but we we learn from those predictions provides much deeper insight. I ask deeper questions about the mathematical structure of student performance and potential indicators that can be used for early support and intervention.
Preparation¶
Load R and packages.
In [1]:
<span class="o">%</span><span class="k">load_ext</span> <span class="n">rpy2</span><span class="o">.</span><span class="n">ipython</span>
In [2]:
<span class="o">%%</span>R suppressPackageStartupMessages<span class="p">(</span>library<span class="p">(</span>ggplot2<span class="p">))</span> suppressPackageStartupMessages<span class="p">(</span>library<span class="p">(</span>dplyr<span class="p">))</span> suppressPackageStartupMessages<span class="p">(</span>library<span class="p">(</span>caret<span class="p">))</span> suppressPackageStartupMessages<span class="p">(</span>library<span class="p">(</span>gridExtra<span class="p">))</span> suppressPackageStartupMessages<span class="p">(</span>library<span class="p">(</span>MASS<span class="p">))</span> suppressPackageStartupMessages<span class="p">(</span>library<span class="p">(</span>leaps<span class="p">))</span> suppressPackageStartupMessages<span class="p">(</span>library<span class="p">(</span>relaimpo<span class="p">))</span> suppressPackageStartupMessages<span class="p">(</span>library<span class="p">(</span>mgcv<span class="p">))</span>
Read in data.
In [3]:
<span class="o">%%</span>R student.mat <span class="o"><-</span> read.csv<span class="p">(</span><span class="s">"student-mat.csv"</span><span class="p">,</span>sep<span class="o">=</span><span class="s">";"</span><span class="p">)</span> student.por <span class="o"><-</span> read.csv<span class="p">(</span><span class="s">"student-por.csv"</span><span class="p">,</span>sep<span class="o">=</span><span class="s">";"</span><span class="p">)</span> head<span class="p">(</span>student.mat<span class="p">)</span>
school sex age address famsize Pstatus Medu Fedu Mjob Fjob reason 1 GP F 18 U GT3 A 4 4 at_home teacher course 2 GP F 17 U GT3 T 1 1 at_home other course 3 GP F 15 U LE3 T 1 1 at_home other other 4 GP F 15 U GT3 T 4 2 health services home 5 GP F 16 U GT3 T 3 3 other other home 6 GP M 16 U LE3 T 4 3 services other reputation guardian traveltime studytime failures schoolsup famsup paid activities 1 mother 2 2 0 yes no no no 2 father 1 2 0 no yes no no 3 mother 1 2 3 yes no yes no 4 mother 1 3 0 no yes yes yes 5 father 1 2 0 no yes yes no 6 mother 1 2 0 no yes yes yes nursery higher internet romantic famrel freetime goout Dalc Walc health 1 yes yes no no 4 3 4 1 1 3 2 no yes yes no 5 3 3 1 1 3 3 yes yes yes no 4 3 2 2 3 3 4 yes yes yes yes 3 2 2 1 1 5 5 yes yes no no 4 3 2 1 2 5 6 yes yes yes no 5 4 2 1 2 5 absences G1 G2 G3 1 6 5 6 6 2 4 5 5 6 3 10 7 8 10 4 2 15 14 15 5 4 6 10 10 6 10 15 15 15
Linear Model¶
For determining the best linear model, we will use student.mat as a training set and student.por as a test set.
In [4]:
<span class="o">%%</span>R train <span class="o"><-</span> student.mat test <span class="o"><-</span> student.por
Saturated Model¶
Let’s fit a linear model to all of the variables. The saturated model will overfit the data, but it will provide a control that can be used to test against.
In [5]:
<span class="o">%%</span>R fit <span class="o"><-</span> lm<span class="p">(</span>G3 <span class="o">~</span> .<span class="p">,</span> train<span class="p">)</span>
Compare Adjusted R2, BIC, and Mallow’s CP With Best Subsets¶
5 variables give the lowest BIC and Mallow’s CP while providing an optimal Adjusted R2.
In [6]:
<span class="o">%%</span>R
subs <span class="o"><-</span> regsubsets<span class="p">(</span>G3 <span class="o">~</span> .<span class="p">,</span> data <span class="o">=</span> train<span class="p">)</span>
df <span class="o"><-</span> data.frame<span class="p">(</span>est <span class="o">=</span> c<span class="p">(</span>summary<span class="p">(</span>subs<span class="p">)</span><span class="o">$</span>adjr2<span class="p">,</span>
summary<span class="p">(</span>subs<span class="p">)</span><span class="o">$</span>cp<span class="p">,</span>
summary<span class="p">(</span>subs<span class="p">)</span><span class="o">$</span>bic<span class="p">),</span>
x <span class="o">=</span> rep<span class="p">(</span><span class="m">1</span><span class="o">:</span><span class="m">8</span><span class="p">,</span> <span class="m">33</span><span class="p">),</span>
type <span class="o">=</span> rep<span class="p">(</span>c<span class="p">(</span><span class="s">"adjr2"</span><span class="p">,</span> <span class="s">"cp"</span><span class="p">,</span> <span class="s">"bic"</span><span class="p">),</span>
each <span class="o">=</span> <span class="m">8</span><span class="p">))</span>
qplot<span class="p">(</span>x<span class="p">,</span> est<span class="p">,</span> data <span class="o">=</span> df<span class="p">,</span> geom <span class="o">=</span> <span class="s">"line"</span><span class="p">)</span> <span class="o">+</span>
theme_bw<span class="p">()</span> <span class="o">+</span> facet_grid<span class="p">(</span>type <span class="o">~</span> .<span class="p">,</span> scales <span class="o">=</span> <span class="s">"free_y"</span><span class="p">)</span>
From the summary, we need to pick the top 5 variables. G1, G2, absences, and famrel will be the first four and the fifth will either be age or activities.
In [7]:
<span class="o">%%</span>R fit <span class="o"><-</span> lm<span class="p">(</span>formula <span class="o">=</span> G3 <span class="o">~</span> .<span class="p">,</span> data <span class="o">=</span> train<span class="p">)</span> summary<span class="p">(</span>fit<span class="p">)</span>
Call:
lm(formula = G3 ~ ., data = train)
Residuals:
Min 1Q Median 3Q Max
-7.9339 -0.5532 0.2680 0.9689 4.6461
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -1.115488 2.116958 -0.527 0.598573
schoolMS 0.480742 0.366512 1.312 0.190485
sexM 0.174396 0.233588 0.747 0.455805
age -0.173302 0.100780 -1.720 0.086380 .
addressU 0.104455 0.270791 0.386 0.699922
famsizeLE3 0.036512 0.226680 0.161 0.872128
PstatusT -0.127673 0.335626 -0.380 0.703875
Medu 0.129685 0.149999 0.865 0.387859
Fedu -0.133940 0.128768 -1.040 0.298974
Mjobhealth -0.146426 0.518491 -0.282 0.777796
Mjobother 0.074088 0.332044 0.223 0.823565
Mjobservices 0.046956 0.369587 0.127 0.898973
Mjobteacher -0.026276 0.481632 -0.055 0.956522
Fjobhealth 0.330948 0.666601 0.496 0.619871
Fjobother -0.083582 0.476796 -0.175 0.860945
Fjobservices -0.322142 0.493265 -0.653 0.514130
Fjobteacher -0.112364 0.601448 -0.187 0.851907
reasonhome -0.209183 0.256392 -0.816 0.415123
reasonother 0.307554 0.380214 0.809 0.419120
reasonreputation 0.129106 0.267254 0.483 0.629335
guardianmother 0.195741 0.252672 0.775 0.439046
guardianother 0.006565 0.463650 0.014 0.988710
traveltime 0.096994 0.157800 0.615 0.539170
studytime -0.104754 0.134814 -0.777 0.437667
failures -0.160539 0.161006 -0.997 0.319399
schoolsupyes 0.456448 0.319538 1.428 0.154043
famsupyes 0.176870 0.224204 0.789 0.430710
paidyes 0.075764 0.222100 0.341 0.733211
activitiesyes -0.346047 0.205938 -1.680 0.093774 .
nurseryyes -0.222716 0.254184 -0.876 0.381518
higheryes 0.225921 0.500398 0.451 0.651919
internetyes -0.144462 0.287528 -0.502 0.615679
romanticyes -0.272008 0.219732 -1.238 0.216572
famrel 0.356876 0.114124 3.127 0.001912 **
freetime 0.047002 0.110209 0.426 0.670021
goout 0.012007 0.105230 0.114 0.909224
Dalc -0.185019 0.153124 -1.208 0.227741
Walc 0.176772 0.114943 1.538 0.124966
health 0.062995 0.074800 0.842 0.400259
absences 0.045879 0.013412 3.421 0.000698 ***
G1 0.188847 0.062373 3.028 0.002645 **
G2 0.957330 0.053460 17.907 < 2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 1.901 on 353 degrees of freedom
Multiple R-squared: 0.8458, Adjusted R-squared: 0.8279
F-statistic: 47.21 on 41 and 353 DF, p-value: < 2.2e-16
ANOVA¶
The ANOVA test tells us that the best model is the one with age.
In [8]:
<span class="o">%%</span>R model1 <span class="o"><-</span> lm<span class="p">(</span>G3<span class="o">~</span> G1 <span class="o">+</span> G2 <span class="o">+</span> absences <span class="o">+</span> famrel <span class="o">+</span> age<span class="p">,</span> data <span class="o">=</span> train<span class="p">)</span> model2 <span class="o"><-</span> lm<span class="p">(</span>G3<span class="o">~</span> G1 <span class="o">+</span> G2 <span class="o">+</span> absences <span class="o">+</span> famrel <span class="o">+</span> activities<span class="p">,</span> data <span class="o">=</span> train<span class="p">)</span> anova<span class="p">(</span>fit<span class="p">,</span> model1<span class="p">,</span> model2<span class="p">)</span>
Analysis of Variance Table
Model 1: G3 ~ school + sex + age + address + famsize + Pstatus + Medu +
Fedu + Mjob + Fjob + reason + guardian + traveltime + studytime +
failures + schoolsup + famsup + paid + activities + nursery +
higher + internet + romantic + famrel + freetime + goout +
Dalc + Walc + health + absences + G1 + G2
Model 2: G3 ~ G1 + G2 + absences + famrel + age
Model 3: G3 ~ G1 + G2 + absences + famrel + activities
Res.Df RSS Df Sum of Sq F Pr(>F)
1 353 1275.5
2 389 1376.1 -36 -100.589 0.7733 0.8248
3 389 1391.4 0 -15.309
Test Set¶
Very quickly, we have an accurate model that did a great job predicting our test set. Notice the darker alpha areas snugly against the line.
We can visually compare the success of the final model versus the saturated model by graphing the predicted values versus the actual values. The line represents a perfect model.
Please note the outliers around the actual values of 0. I will go into more detail about this group later in this project.
In [9]:
<span class="o">%%</span>R
<span class="c1">#Saturated Model</span>
control.model <span class="o"><-</span> lm<span class="p">(</span>G3 <span class="o">~</span> .<span class="p">,</span> data <span class="o">=</span> test<span class="p">)</span>
control.graph <span class="o"><-</span> qplot<span class="p">(</span>G3<span class="p">,</span> predict<span class="p">(</span>control.model<span class="p">),</span> data <span class="o">=</span> test<span class="p">,</span> geom <span class="o">=</span> <span class="s">"point"</span><span class="p">,</span>
position <span class="o">=</span> <span class="s">"jitter"</span><span class="p">,</span> alpha<span class="o">=</span><span class="m">.5</span><span class="p">,</span> main<span class="o">=</span><span class="s">"Saturated Model"</span><span class="p">)</span> <span class="o">+</span>
geom_abline<span class="p">(</span>intercept<span class="o">=</span><span class="m">0</span><span class="p">,</span> slope<span class="o">=</span><span class="m">1</span><span class="p">)</span> <span class="o">+</span>
theme<span class="p">(</span>legend.position<span class="o">=</span><span class="s">"none"</span><span class="p">)</span>
<span class="c1">#Final Model</span>
final.model <span class="o"><-</span> lm<span class="p">(</span>G3<span class="o">~</span> G1 <span class="o">+</span> G2 <span class="o">+</span> absences <span class="o">+</span> famrel <span class="o">+</span> age<span class="p">,</span> data <span class="o">=</span> test<span class="p">)</span>
final.graph <span class="o"><-</span> qplot<span class="p">(</span>G3<span class="p">,</span> predict<span class="p">(</span>final.model<span class="p">),</span> data <span class="o">=</span> test<span class="p">,</span> geom <span class="o">=</span> <span class="s">"point"</span><span class="p">,</span>
position <span class="o">=</span> <span class="s">"jitter"</span><span class="p">,</span> alpha<span class="o">=</span><span class="m">.5</span><span class="p">,</span> main<span class="o">=</span><span class="s">"Final Model"</span><span class="p">,</span> guide<span class="o">=</span><span class="kc">FALSE</span><span class="p">)</span> <span class="o">+</span>
geom_abline<span class="p">(</span>intercept<span class="o">=</span><span class="m">0</span><span class="p">,</span> slope<span class="o">=</span><span class="m">1</span><span class="p">)</span> <span class="o">+</span>
theme<span class="p">(</span>legend.position<span class="o">=</span><span class="s">"none"</span><span class="p">)</span>
grid.arrange<span class="p">(</span>control.graph<span class="p">,</span>final.graph<span class="p">,</span>nrow<span class="o">=</span><span class="m">2</span><span class="p">)</span>
Diagnostics¶
Overall, our model looks pretty good. The main issue with our model is the cluster when G3 is 0.
It affects the residuals at the lower end of our distribution.
In [10]:
<span class="o">%%</span>R plot<span class="p">(</span>final.model<span class="p">)</span>
The 0-Cluster¶
Upon further inspection of the data, it becomes obvious that this cluster most likely belongs to students who dropped the course.
- They have G1 and/or G2 grades but final grades of 0.
- There are no G1s of 0 but there are G2s with 0 value.
- The exploratory model predicts these students as scoring between 0 and 10 which would constitute failing grades.
As a result, we should drop these data points before continuing our analysis since they will not be useful for the question we are researching.
In [11]:
<span class="o">%%</span>R score0 <span class="o"><-</span> subset<span class="p">(</span>student.por<span class="p">,</span> G3<span class="o">==</span><span class="m">0</span><span class="p">)</span> score0
school sex age address famsize Pstatus Medu Fedu Mjob Fjob
164 GP M 18 U LE3 T 1 1 other other
441 MS M 16 U GT3 T 1 1 at_home services
520 MS M 16 R GT3 T 2 1 other services
564 MS M 17 U GT3 T 2 2 other other
568 MS M 18 R GT3 T 3 2 services other
584 MS F 18 R GT3 T 2 2 other other
587 MS F 17 U GT3 T 4 2 teacher services
598 MS F 18 R GT3 T 2 2 at_home other
604 MS F 18 R LE3 A 4 2 teacher other
606 MS F 19 U GT3 T 1 1 at_home services
611 MS F 19 R GT3 A 1 1 at_home at_home
627 MS F 18 R GT3 T 4 4 other teacher
638 MS M 18 R GT3 T 2 1 other other
640 MS M 19 R GT3 T 1 1 other services
641 MS M 18 R GT3 T 4 2 other other
reason guardian traveltime studytime failures schoolsup famsup fatherd
164 course mother 1 1 2 no no no
441 home mother 2 2 0 no yes no
520 reputation mother 2 2 0 no no no
564 course mother 1 1 1 no no no
568 course mother 1 1 1 no no no
584 other mother 2 1 1 no no no
587 home mother 1 2 0 yes yes no
598 course mother 3 2 1 no no no
604 reputation mother 1 2 0 no no no
606 other father 2 1 1 no no no
611 course other 2 2 3 no yes no
627 other father 3 2 0 no yes no
638 other mother 2 1 0 no no no
640 other mother 2 1 1 no no no
641 home father 2 1 1 no no yes
activities nursery higher internet romantic famrel freetime goout Dalc Walc
164 no yes no yes yes 2 3 5 2 5
441 yes yes yes no yes 5 4 5 4 5
520 yes yes yes yes no 5 2 1 1 1
564 yes yes yes no yes 1 2 1 2 3
568 no yes no yes no 2 3 1 2 2
584 no yes no yes yes 5 5 5 1 1
587 yes yes yes yes no 5 5 5 1 3
598 yes yes yes no yes 4 3 3 1 1
604 yes yes yes yes yes 5 3 1 1 1
606 no yes no no no 5 5 5 2 3
611 yes yes no no yes 3 5 4 1 4
627 no no yes yes yes 3 2 2 4 2
638 yes no yes yes yes 4 4 3 1 3
640 no yes yes no no 4 3 2 1 3
641 no yes yes no no 5 4 3 4 3
health absences G1 G2 G3
164 4 0 11 9 0
441 3 0 7 0 0
520 2 0 8 7 0
564 5 0 7 0 0
568 5 0 4 0 0
584 3 0 8 6 0
587 5 0 8 8 0
598 4 0 9 0 0
604 5 0 5 0 0
606 2 0 5 0 0
611 1 0 8 0 0
627 5 0 7 5 0
638 5 0 7 7 0
640 5 0 5 8 0
641 3 0 7 7 0
Final Model¶
Here is the final model for students who finish the course.
In [12]:
<span class="o">%%</span>R
<span class="c1">#Final Model</span>
test <span class="o"><-</span> subset<span class="p">(</span>train<span class="p">,</span> G3<span class="o">!=</span><span class="m">0</span><span class="p">)</span>
final.model.no0 <span class="o"><-</span> lm<span class="p">(</span>G3<span class="o">~</span> G1 <span class="o">+</span> G2 <span class="o">+</span> absences <span class="o">+</span> famrel <span class="o">+</span> age<span class="p">,</span> data <span class="o">=</span> test<span class="p">)</span>
qplot<span class="p">(</span>G3<span class="p">,</span> predict<span class="p">(</span>final.model.no0<span class="p">),</span> data <span class="o">=</span> test<span class="p">,</span> geom <span class="o">=</span> <span class="s">"point"</span><span class="p">,</span>
position <span class="o">=</span> <span class="s">"jitter"</span><span class="p">,</span> alpha<span class="o">=</span><span class="m">.5</span><span class="p">,</span> main<span class="o">=</span><span class="s">"Final Model"</span><span class="p">,</span> guide<span class="o">=</span><span class="kc">FALSE</span><span class="p">)</span> <span class="o">+</span>
geom_abline<span class="p">(</span>intercept<span class="o">=</span><span class="m">0</span><span class="p">,</span> slope<span class="o">=</span><span class="m">1</span><span class="p">)</span> <span class="o">+</span>
theme<span class="p">(</span>legend.position<span class="o">=</span><span class="s">"none"</span><span class="p">)</span>
Deeper Questions and Analysis¶
Our model does a great job at predicting student success; however, there are deeper questions that this model doesn’t address. In particular, it doesn’t demonstrate how we can pick which students are most likely to fail classes at an early age when they lack the best predictors in this model.
As we’ve seen, the best predictors of success are current grades within the course (G1 and G2), age, quality of family relationships, and absences.
Current grades are already present once a problem exists.
Let’s try to see if we can determine what factors can be more useful at preventing student failure and promoting academic success.
Let’s start by looking at all the variables within a linear model, but remove our strongest indicators, G1 and G2, which overshadow other potential factors.
In [13]:
<span class="o">%%</span>R fit <span class="o"><-</span> lm<span class="p">(</span>G3 <span class="o">~</span> . <span class="o">-</span>G1 <span class="o">-</span>G2<span class="p">,</span> student.mat<span class="p">)</span>
Our predictions stop at 15 but actual scores rise until 20. Without G1 and G2, our model is unable to make predictions that are any higher.
A score of 15 shows a clear dividing line where the "potential" futures merge into current academic success. This line is important in that it can help us determine what deeper differences successful students have from their peers and also allows to create a definition of a "successful" student that we can use.
For this section, it becomes clear that two models will need to be analyzed: one for grades below 15 and another for grades above 15.
In [14]:
<span class="o">%%</span>R
qplot<span class="p">(</span>G3<span class="p">,</span> predict<span class="p">(</span>fit<span class="p">),</span> data <span class="o">=</span> student.mat<span class="p">,</span> geom <span class="o">=</span> <span class="s">"point"</span><span class="p">,</span> position <span class="o">=</span> <span class="s">"jitter"</span><span class="p">,</span> alpha<span class="o">=</span><span class="m">.8</span><span class="p">)</span> <span class="o">+</span>
geom_abline<span class="p">(</span>intercept<span class="o">=</span><span class="m">0</span><span class="p">,</span> slope<span class="o">=</span><span class="m">1</span><span class="p">)</span> <span class="o">+</span>
theme<span class="p">(</span>legend.position<span class="o">=</span><span class="s">"none"</span><span class="p">)</span>
Breaking Up the Analysis¶
So far, the data has shown that it should be broken into three parts in order to analyze deeper predictors of future success.
Students who drop 1. The first isolates students who drop a course. Their final outcome is 0 even though they should have a higher predicted outcome. These students have predicted scores below 10.
Students who finish 2. Between 0 and 15, one set of predictors (one model) will be used to predict student outcomes. 3. Between 15 and 20, a different set of predictors (a different model) will be used.
In [15]:
<span class="o">%%</span>R <span class="c1">#Prep Data</span> score0 <span class="o"><-</span> subset<span class="p">(</span>student.mat<span class="p">,</span> G3<span class="o">==</span><span class="m">0</span><span class="p">)</span> score.no0 <span class="o"><-</span> subset<span class="p">(</span>student.mat<span class="p">,</span> G3<span class="o">!=</span><span class="m">0</span><span class="p">)</span> score14 <span class="o"><-</span> subset<span class="p">(</span>score.no0<span class="p">,</span> G3<span class="o"><</span><span class="m">15</span><span class="p">)</span> score15 <span class="o"><-</span> subset<span class="p">(</span>score.no0<span class="p">,</span> G3<span class="o">></span><span class="m">14</span><span class="p">)</span>
Students Above 15¶
Students in this group have 3 things that stand out:
1. All of them have parents that live together. 2. None of them have had past class failures. 3. All of them plan on seeking higher education.
In [16]:
<span class="o">%%</span>R score15
school sex age address famsize Pstatus Medu Fedu Mjob Fjob
4 GP F 15 U GT3 T 4 2 health services
6 GP M 16 U LE3 T 4 3 services other
9 GP M 15 U LE3 A 3 2 services other
10 GP M 15 U GT3 T 3 4 other other
15 GP M 15 U GT3 A 2 2 other other
21 GP M 15 U GT3 T 4 3 teacher other
22 GP M 15 U GT3 T 4 4 health health
23 GP M 16 U LE3 T 4 2 teacher other
28 GP M 15 U GT3 T 4 2 health services
32 GP M 15 U GT3 T 4 4 services services
33 GP M 15 R GT3 T 4 3 teacher at_home
35 GP M 16 U GT3 T 3 2 other other
37 GP M 15 U LE3 T 4 3 teacher services
38 GP M 16 R GT3 A 4 4 other teacher
43 GP M 15 U GT3 T 4 4 services teacher
48 GP M 16 U GT3 T 4 3 health services
57 GP F 15 U GT3 A 4 3 services services
58 GP M 15 U GT3 T 4 4 teacher health
60 GP F 16 U GT3 T 4 2 services other
66 GP F 16 U LE3 T 4 3 teacher services
70 GP F 15 R LE3 T 3 1 other other
71 GP M 16 U GT3 T 3 1 other other
84 GP M 15 U LE3 T 2 2 services services
92 GP F 15 U GT3 T 4 3 services other
97 GP M 16 R GT3 T 4 3 services other
102 GP M 16 U GT3 T 4 4 services teacher
105 GP M 15 U GT3 A 3 4 services other
108 GP M 16 U GT3 T 3 3 services other
110 GP F 16 U LE3 T 4 4 health health
111 GP M 15 U LE3 A 4 4 teacher teacher
114 GP M 15 U LE3 T 4 2 teacher other
116 GP M 16 U GT3 T 4 4 teacher teacher
121 GP F 15 U GT3 T 1 2 at_home services
122 GP M 15 U GT3 T 2 2 services services
130 GP M 16 R GT3 T 4 4 teacher teacher
140 GP F 15 U GT3 T 4 4 teacher teacher
159 GP M 16 R GT3 T 2 2 at_home other
168 GP F 16 U GT3 T 4 2 health services
172 GP M 16 U GT3 T 1 0 other other
183 GP F 17 U GT3 T 2 4 services services
188 GP M 16 U LE3 T 2 1 other other
196 GP F 17 U LE3 T 2 4 services services
197 GP M 17 U GT3 T 4 4 services teacher
199 GP F 17 U GT3 T 4 4 services teacher
201 GP F 16 U GT3 T 4 3 health other
216 GP F 17 U LE3 T 3 2 other other
223 GP F 16 U GT3 T 2 3 services teacher
227 GP F 17 U GT3 T 3 2 other other
246 GP M 16 U GT3 T 2 1 other other
250 GP M 16 U GT3 T 0 2 other other
261 GP F 18 U GT3 T 4 3 services other
266 GP M 18 R LE3 A 3 4 other other
287 GP F 18 U GT3 T 2 2 at_home at_home
290 GP M 18 U LE3 A 4 4 teacher teacher
292 GP F 17 U GT3 T 4 3 health services
294 GP F 17 R LE3 T 3 1 services other
300 GP M 18 U LE3 T 4 4 teacher teacher
304 GP F 17 U GT3 T 3 2 health health
307 GP M 20 U GT3 A 3 2 services other
324 GP F 17 U GT3 T 3 1 services services
325 GP F 17 U LE3 T 0 2 at_home at_home
327 GP M 17 U GT3 T 3 3 other services
336 GP F 17 U GT3 T 3 4 services other
339 GP F 18 U LE3 T 3 3 services services
343 GP M 18 U LE3 T 3 4 services other
347 GP M 18 R GT3 T 4 3 teacher services
349 GP F 17 U GT3 T 4 3 health other
360 MS F 18 U LE3 T 1 1 at_home services
364 MS F 17 U LE3 T 4 4 at_home at_home
375 MS F 18 R LE3 T 4 4 other other
377 MS F 20 U GT3 T 4 2 health other
379 MS F 18 U GT3 T 3 3 other other
392 MS M 17 U LE3 T 3 1 services services
reason guardian traveltime studytime failures schoolsup famsup paid
4 home mother 1 3 0 no yes yes
6 reputation mother 1 2 0 no yes yes
9 home mother 1 2 0 no yes yes
10 home mother 1 2 0 no yes yes
15 home other 1 3 0 no yes no
21 reputation mother 1 2 0 no no no
22 other father 1 1 0 no yes yes
23 course mother 1 2 0 no no no
28 other mother 1 1 0 no no yes
32 reputation mother 2 2 0 no yes no
33 course mother 1 2 0 no yes no
35 home mother 1 1 0 no yes yes
37 home mother 1 3 0 no yes no
38 reputation mother 2 3 0 no yes no
43 course father 1 2 0 no yes no
48 reputation mother 1 4 0 no no no
57 reputation mother 1 2 0 no yes yes
58 reputation mother 1 2 0 no yes no
60 course mother 1 2 0 no yes no
66 course mother 3 2 0 no yes no
70 reputation father 2 4 0 no yes no
71 reputation father 2 4 0 no yes yes
84 home mother 2 2 0 no no yes
92 reputation mother 1 1 0 no no yes
97 reputation mother 2 1 0 yes yes no
102 other father 1 3 0 no yes no
105 course mother 1 2 0 no yes yes
108 home father 1 3 0 no yes no
110 other mother 1 3 0 no yes yes
111 course mother 1 1 0 no no no
114 course mother 1 1 0 no no no
116 course father 1 2 0 no yes no
121 course mother 1 2 0 no no no
122 home father 1 4 0 no yes yes
130 course mother 1 1 0 no no yes
140 course mother 2 1 0 no no no
159 course mother 3 1 0 no no no
168 home father 1 2 0 no no yes
172 reputation mother 2 2 0 no yes yes
183 reputation father 1 2 0 no yes no
188 course mother 1 2 0 no no yes
196 course father 1 2 0 no no no
197 home mother 1 1 0 no no no
199 home mother 2 1 1 no yes no
201 home mother 1 2 0 no yes no
216 reputation mother 2 2 0 no no yes
223 other mother 1 2 0 yes no no
227 course mother 1 2 0 no no no
246 course mother 3 1 0 no no no
250 other mother 1 1 0 no no yes
261 home father 1 2 0 no yes yes
266 reputation mother 2 2 0 no yes yes
287 other mother 1 3 0 no yes yes
290 reputation mother 1 2 0 no yes yes
292 reputation mother 1 3 0 no yes yes
294 reputation mother 2 4 0 no yes yes
300 home mother 1 1 0 no yes yes
304 reputation father 1 4 0 no yes yes
307 course other 1 1 0 no no no
324 course father 1 3 0 no yes no
325 home father 2 3 0 no no no
327 reputation mother 1 1 0 no no no
336 course mother 1 3 0 no no no
339 home mother 1 4 0 no yes no
343 home mother 1 2 0 no no no
347 course mother 1 3 0 no no no
349 reputation mother 1 3 0 no yes yes
360 course father 2 3 0 no no no
364 course mother 1 2 0 no yes yes
375 reputation mother 2 3 0 no no no
377 course other 2 3 2 no yes yes
379 home mother 1 2 0 no no yes
392 course mother 2 1 0 no no no
activities nursery higher internet romantic famrel freetime goout Dalc Walc
4 yes yes yes yes yes 3 2 2 1 1
6 yes yes yes yes no 5 4 2 1 2
9 no yes yes yes no 4 2 2 1 1
10 yes yes yes yes no 5 5 1 1 1
15 no yes yes yes yes 4 5 2 1 1
21 no yes yes yes no 4 4 1 1 1
22 no yes yes yes no 5 4 2 1 1
23 yes yes yes yes no 4 5 1 1 3
28 no yes yes yes no 2 2 4 2 4
32 yes yes yes yes no 4 3 1 1 1
33 yes yes yes yes yes 4 5 2 1 1
35 no no yes yes no 5 4 3 1 1
37 yes yes yes yes no 5 4 3 1 1
38 yes yes yes yes yes 2 4 3 1 1
43 yes yes yes yes no 4 3 3 1 1
48 yes yes yes yes no 4 2 2 1 1
57 yes yes yes yes no 4 3 2 1 1
58 yes yes yes no no 3 2 2 1 1
60 no yes yes yes no 4 2 3 1 1
66 yes yes yes yes no 5 4 3 1 2
70 no no yes yes no 4 4 2 2 3
71 no yes yes yes no 4 3 2 1 1
84 yes yes yes yes no 5 3 3 1 3
92 yes yes yes yes no 4 5 5 1 3
97 yes no yes yes no 3 3 3 1 1
102 yes yes yes yes yes 4 4 3 1 1
105 yes yes yes yes no 5 4 4 1 1
108 yes yes yes yes no 5 3 3 1 1
110 yes yes yes yes yes 5 4 5 1 1
111 yes yes yes yes no 5 5 3 1 1
114 no yes yes yes no 3 5 2 1 1
116 yes yes yes yes no 5 4 4 1 2
121 no no yes yes no 3 2 3 1 2
122 yes yes yes yes no 5 5 4 1 2
130 yes yes yes yes no 3 5 5 2 5
140 yes yes yes yes no 4 3 2 1 1
159 no no yes no no 4 2 2 1 2
168 no yes yes yes yes 4 2 3 1 1
172 yes yes yes yes yes 4 3 2 1 1
183 yes yes yes no no 5 4 2 2 3
188 yes yes yes yes yes 4 2 3 1 2
196 yes yes yes yes yes 4 3 2 1 1
197 no yes yes yes no 5 2 3 1 2
199 no yes yes yes no 4 2 4 2 3
201 yes yes yes yes no 4 3 5 1 5
216 no yes yes yes no 4 4 4 1 3
223 no yes yes yes no 2 3 1 1 1
227 yes no yes yes no 5 3 4 1 3
246 no yes yes yes no 4 3 3 1 1
250 no no yes yes no 4 3 2 2 4
261 no yes yes yes yes 3 1 2 1 3
266 yes yes yes yes no 4 2 5 3 4
287 no yes yes yes no 4 3 3 1 2
290 yes yes yes yes no 5 4 3 1 1
292 no yes yes yes no 4 2 2 1 2
294 no yes yes no no 3 1 2 1 1
300 no yes yes yes yes 1 4 2 2 2
304 yes no yes yes no 5 2 2 1 2
307 yes yes yes no no 5 5 3 1 1
324 no no yes yes no 3 4 3 2 3
325 no yes yes yes no 3 3 3 2 3
327 yes no yes yes no 4 3 5 3 5
336 no yes yes yes no 4 4 5 1 3
339 no yes yes yes no 5 3 3 1 1
343 yes yes yes yes yes 4 3 3 1 3
347 no yes yes yes yes 5 3 2 1 2
349 yes yes yes yes yes 4 4 3 1 3
360 no yes yes yes no 5 3 2 1 1
364 yes yes yes yes yes 2 3 4 1 1
375 no yes yes yes no 5 4 4 1 1
377 no no yes yes yes 5 4 3 1 1
379 no yes yes yes yes 4 1 3 1 2
392 no no yes yes no 2 4 5 3 4
health absences G1 G2 G3
4 5 2 15 14 15
6 5 10 15 15 15
9 1 0 16 18 19
10 5 0 14 15 15
15 3 0 14 16 16
21 1 0 13 14 15
22 5 0 12 15 15
23 5 2 15 15 16
28 1 4 15 16 15
32 5 0 17 16 17
33 5 0 17 16 16
35 5 0 12 14 15
37 4 2 15 16 18
38 5 7 15 16 15
43 5 2 19 18 18
48 2 4 19 19 20
57 1 0 14 15 15
58 5 4 14 15 15
60 5 2 15 16 16
66 1 2 16 15 15
70 3 12 16 16 16
71 5 0 13 15 15
84 4 4 15 15 15
92 1 4 16 17 18
97 4 2 11 15 15
102 4 0 16 17 17
105 1 0 16 18 18
108 5 2 16 18 18
110 4 4 14 15 16
111 4 6 18 19 19
114 3 10 18 19 19
116 5 2 15 15 16
121 1 2 16 15 15
122 5 6 16 14 15
130 4 8 18 18 18
140 5 0 16 16 15
159 3 2 17 15 15
168 3 0 14 15 16
172 3 2 13 15 16
183 5 0 16 17 17
188 5 0 15 15 15
196 5 0 14 15 15
197 5 4 17 15 16
199 2 24 18 18 18
201 2 2 16 16 16
216 1 2 14 15 15
223 3 2 16 16 17
227 3 10 16 15 15
246 4 6 18 18 18
250 5 0 13 15 15
261 2 21 17 18 18
266 1 13 17 17 17
287 2 5 18 18 19
290 2 9 15 13 15
292 3 0 15 15 15
294 3 6 18 18 18
300 1 5 16 15 16
304 5 0 17 17 18
307 5 0 17 18 18
324 5 1 12 14 15
325 2 0 16 15 15
327 5 3 14 15 16
336 5 16 16 15 15
339 1 7 16 15 17
343 5 11 16 15 15
347 4 9 16 15 16
349 4 0 13 15 15
360 4 0 18 16 16
364 1 0 16 15 15
375 1 0 19 18 19
377 3 4 15 14 15
379 1 0 15 15 15
392 2 3 14 16 16
Students Below 14¶
Create a training and test set for this group.
In [17]:
<span class="o">%%</span>R set.seed<span class="p">(</span><span class="m">123</span><span class="p">)</span> inTraining <span class="o"><-</span> createDataPartition<span class="p">(</span>score14<span class="o">$</span>G3<span class="p">,</span> p <span class="o">=</span> <span class="m">.75</span><span class="p">,</span> list <span class="o">=</span> <span class="kc">FALSE</span><span class="p">)</span> training <span class="o"><-</span> score14<span class="p">[</span> inTraining<span class="p">,]</span> testing <span class="o"><-</span> score14<span class="p">[</span><span class="o">-</span>inTraining<span class="p">,]</span>
Saturated Model¶
Below is a general model with all of our variables using the training set. This can help determine which predictors are statistically significant.
In [18]:
<span class="o">%%</span>R saturated14 <span class="o"><-</span> lm<span class="p">(</span>G3 <span class="o">~</span> . <span class="o">-</span>G1 <span class="o">-</span>G2<span class="p">,</span> data <span class="o">=</span> training<span class="p">)</span> summary<span class="p">(</span>saturated14<span class="p">)</span>
Call:
lm(formula = G3 ~ . - G1 - G2, data = training)
Residuals:
Min 1Q Median 3Q Max
-5.0532 -1.2297 -0.0758 1.5029 5.1073
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 10.740459 3.406452 3.153 0.00190 **
schoolMS -0.669689 0.564845 -1.186 0.23739
sexM 0.511533 0.416269 1.229 0.22079
age 0.029858 0.168227 0.177 0.85933
addressU 0.735560 0.458102 1.606 0.11016
famsizeLE3 0.013457 0.379180 0.035 0.97173
PstatusT 0.188910 0.558990 0.338 0.73581
Medu -0.008148 0.254834 -0.032 0.97453
Fedu 0.006450 0.222098 0.029 0.97686
Mjobhealth 1.584729 0.895228 1.770 0.07845 .
Mjobother -0.029289 0.528770 -0.055 0.95589
Mjobservices 0.545055 0.619720 0.880 0.38033
Mjobteacher -0.175333 0.787911 -0.223 0.82416
Fjobhealth -0.772045 1.062648 -0.727 0.46849
Fjobother -0.539129 0.757634 -0.712 0.47767
Fjobservices -0.479938 0.785370 -0.611 0.54193
Fjobteacher -0.079528 1.048411 -0.076 0.93962
reasonhome 0.688622 0.431180 1.597 0.11207
reasonother 0.321321 0.607912 0.529 0.59778
reasonreputation 0.435443 0.437953 0.994 0.32147
guardianmother 0.217571 0.432852 0.503 0.61585
guardianother 0.250323 0.754461 0.332 0.74045
traveltime 0.092044 0.260529 0.353 0.72429
studytime 0.411308 0.227115 1.811 0.07186 .
failures -0.806048 0.250153 -3.222 0.00152 **
schoolsupyes -0.491972 0.479693 -1.026 0.30650
famsupyes -0.298508 0.366403 -0.815 0.41636
paidyes 0.214191 0.373014 0.574 0.56656
activitiesyes 0.388324 0.358694 1.083 0.28048
nurseryyes -0.407032 0.407825 -0.998 0.31964
higheryes -0.816840 0.799668 -1.021 0.30845
internetyes -0.134818 0.453433 -0.297 0.76657
romanticyes 0.101779 0.360754 0.282 0.77818
famrel -0.016745 0.185782 -0.090 0.92829
freetime 0.050437 0.184857 0.273 0.78530
goout -0.462603 0.179052 -2.584 0.01060 *
Dalc 0.290830 0.240016 1.212 0.22727
Walc -0.025814 0.188709 -0.137 0.89135
health -0.056258 0.120458 -0.467 0.64106
absences -0.054226 0.020125 -2.694 0.00774 **
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 2.238 on 174 degrees of freedom
Multiple R-squared: 0.2508, Adjusted R-squared: 0.08291
F-statistic: 1.494 on 39 and 174 DF, p-value: 0.04302
Let’s use the step function to find a cut down version of Model 1 that removes uneccesary predictors.
In [19]:
<span class="o">%%</span>R step<span class="p">(</span>saturated14<span class="p">)</span>
Start: AIC=380.6
G3 ~ (school + sex + age + address + famsize + Pstatus + Medu +
Fedu + Mjob + Fjob + reason + guardian + traveltime + studytime +
failures + schoolsup + famsup + paid + activities + nursery +
higher + internet + romantic + famrel + freetime + goout +
Dalc + Walc + health + absences + G1 + G2) - G1 - G2
Df Sum of Sq RSS AIC
- Fjob 4 5.065 876.93 373.84
- guardian 2 1.346 873.21 376.93
- reason 3 13.248 885.11 377.82
- Fedu 1 0.004 871.87 378.60
- Medu 1 0.005 871.87 378.60
- famsize 1 0.006 871.87 378.60
- famrel 1 0.041 871.91 378.61
- Walc 1 0.094 871.96 378.62
- age 1 0.158 872.02 378.64
- freetime 1 0.373 872.24 378.69
- romantic 1 0.399 872.26 378.69
- internet 1 0.443 872.31 378.71
- Pstatus 1 0.572 872.44 378.74
- traveltime 1 0.625 872.49 378.75
- health 1 1.093 872.96 378.86
- paid 1 1.652 873.52 379.00
- famsup 1 3.326 875.19 379.41
- nursery 1 4.991 876.86 379.82
- higher 1 5.228 877.09 379.88
- schoolsup 1 5.271 877.14 379.89
- activities 1 5.873 877.74 380.03
- school 1 7.043 878.91 380.32
- Dalc 1 7.357 879.22 380.40
- Mjob 4 32.450 904.32 380.42
- sex 1 7.567 879.43 380.45
<none> 871.86 380.60
- address 1 12.919 884.78 381.74
- studytime 1 16.434 888.30 382.59
- goout 1 33.447 905.31 386.65
- absences 1 36.379 908.24 387.34
- failures 1 52.025 923.89 391.00
Step: AIC=373.84
G3 ~ school + sex + age + address + famsize + Pstatus + Medu +
Fedu + Mjob + reason + guardian + traveltime + studytime +
failures + schoolsup + famsup + paid + activities + nursery +
higher + internet + romantic + famrel + freetime + goout +
Dalc + Walc + health + absences
Df Sum of Sq RSS AIC
- guardian 2 0.947 877.88 370.07
- reason 3 13.191 890.12 371.03
- famsize 1 0.002 876.93 371.84
- Medu 1 0.014 876.94 371.84
- Fedu 1 0.027 876.96 371.84
- famrel 1 0.110 877.04 371.86
- Walc 1 0.378 877.31 371.93
- age 1 0.383 877.31 371.93
- freetime 1 0.400 877.33 371.93
- romantic 1 0.484 877.41 371.95
- traveltime 1 0.554 877.48 371.97
- Pstatus 1 0.674 877.60 372.00
- internet 1 0.705 877.63 372.01
- health 1 1.025 877.95 372.09
- paid 1 1.608 878.54 372.23
- famsup 1 3.436 880.37 372.67
- schoolsup 1 4.248 881.18 372.87
- higher 1 4.945 881.87 373.04
- nursery 1 5.473 882.40 373.17
- Mjob 4 30.591 907.52 373.17
- activities 1 6.427 883.36 373.40
- sex 1 6.927 883.86 373.52
- school 1 7.118 884.05 373.57
<none> 876.93 373.84
- Dalc 1 9.398 886.33 374.12
- address 1 12.765 889.69 374.93
- studytime 1 15.487 892.42 375.58
- goout 1 31.904 908.83 379.48
- absences 1 36.836 913.77 380.64
- failures 1 54.906 931.84 384.83
Step: AIC=370.07
G3 ~ school + sex + age + address + famsize + Pstatus + Medu +
Fedu + Mjob + reason + traveltime + studytime + failures +
schoolsup + famsup + paid + activities + nursery + higher +
internet + romantic + famrel + freetime + goout + Dalc +
Walc + health + absences
Df Sum of Sq RSS AIC
- reason 3 13.466 891.34 367.33
- Medu 1 0.001 877.88 368.07
- Fedu 1 0.002 877.88 368.07
- famsize 1 0.003 877.88 368.07
- famrel 1 0.099 877.98 368.09
- Walc 1 0.382 878.26 368.16
- freetime 1 0.520 878.40 368.19
- traveltime 1 0.521 878.40 368.19
- Pstatus 1 0.541 878.42 368.20
- romantic 1 0.548 878.42 368.20
- age 1 0.660 878.54 368.23
- internet 1 0.818 878.69 368.27
- health 1 1.034 878.91 368.32
- paid 1 1.537 879.41 368.44
- famsup 1 3.107 880.98 368.82
- schoolsup 1 4.288 882.16 369.11
- higher 1 4.696 882.57 369.21
- Mjob 4 29.905 907.78 369.24
- nursery 1 5.667 883.54 369.44
- activities 1 6.202 884.08 369.57
- sex 1 6.609 884.49 369.67
- school 1 7.736 885.61 369.94
<none> 877.88 370.07
- Dalc 1 9.348 887.22 370.33
- address 1 13.101 890.98 371.24
- studytime 1 17.000 894.88 372.17
- goout 1 32.920 910.80 375.95
- absences 1 35.981 913.86 376.66
- failures 1 57.841 935.72 381.72
Step: AIC=367.33
G3 ~ school + sex + age + address + famsize + Pstatus + Medu +
Fedu + Mjob + traveltime + studytime + failures + schoolsup +
famsup + paid + activities + nursery + higher + internet +
romantic + famrel + freetime + goout + Dalc + Walc + health +
absences
Df Sum of Sq RSS AIC
- Fedu 1 0.001 891.34 365.33
- Medu 1 0.012 891.35 365.33
- famsize 1 0.034 891.38 365.33
- famrel 1 0.165 891.51 365.36
- Pstatus 1 0.222 891.56 365.38
- freetime 1 0.312 891.65 365.40
- romantic 1 0.765 892.11 365.51
- Walc 1 0.804 892.15 365.52
- age 1 0.852 892.19 365.53
- internet 1 0.989 892.33 365.56
- health 1 1.151 892.49 365.60
- traveltime 1 1.173 892.52 365.61
- schoolsup 1 3.406 894.75 366.14
- higher 1 3.426 894.77 366.15
- famsup 1 3.993 895.34 366.28
- paid 1 4.108 895.45 366.31
- nursery 1 5.282 896.62 366.59
- Mjob 4 31.118 922.46 366.67
- activities 1 7.662 899.00 367.16
<none> 891.34 367.33
- school 1 8.631 899.97 367.39
- sex 1 8.987 900.33 367.47
- Dalc 1 9.983 901.33 367.71
- address 1 14.591 905.93 368.80
- studytime 1 17.603 908.95 369.51
- absences 1 28.273 919.62 372.01
- goout 1 31.410 922.75 372.74
- failures 1 59.037 950.38 379.05
Step: AIC=365.33
G3 ~ school + sex + age + address + famsize + Pstatus + Medu +
Mjob + traveltime + studytime + failures + schoolsup + famsup +
paid + activities + nursery + higher + internet + romantic +
famrel + freetime + goout + Dalc + Walc + health + absences
Df Sum of Sq RSS AIC
- Medu 1 0.014 891.36 363.33
- famsize 1 0.034 891.38 363.33
- famrel 1 0.167 891.51 363.37
- Pstatus 1 0.224 891.57 363.38
- freetime 1 0.313 891.66 363.40
- romantic 1 0.770 892.11 363.51
- Walc 1 0.808 892.15 363.52
- age 1 0.852 892.20 363.53
- internet 1 0.994 892.34 363.56
- health 1 1.155 892.50 363.60
- traveltime 1 1.174 892.52 363.61
- higher 1 3.434 894.78 364.15
- schoolsup 1 3.479 894.82 364.16
- famsup 1 4.105 895.45 364.31
- paid 1 4.110 895.45 364.31
- nursery 1 5.289 896.63 364.59
- Mjob 4 31.118 922.46 364.67
- activities 1 7.692 899.03 365.16
<none> 891.34 365.33
- school 1 8.660 900.00 365.39
- sex 1 8.986 900.33 365.47
- Dalc 1 9.983 901.33 365.71
- address 1 14.827 906.17 366.86
- studytime 1 17.708 909.05 367.53
- absences 1 28.845 920.19 370.14
- goout 1 31.423 922.77 370.74
- failures 1 60.614 951.96 377.40
Step: AIC=363.33
G3 ~ school + sex + age + address + famsize + Pstatus + Mjob +
traveltime + studytime + failures + schoolsup + famsup +
paid + activities + nursery + higher + internet + romantic +
famrel + freetime + goout + Dalc + Walc + health + absences
Df Sum of Sq RSS AIC
- famsize 1 0.027 891.38 361.34
- famrel 1 0.168 891.52 361.37
- Pstatus 1 0.233 891.59 361.38
- freetime 1 0.343 891.70 361.41
- romantic 1 0.761 892.12 361.51
- Walc 1 0.797 892.15 361.52
- age 1 0.892 892.25 361.54
- internet 1 0.982 892.34 361.56
- health 1 1.142 892.50 361.60
- traveltime 1 1.170 892.53 361.61
- schoolsup 1 3.478 894.83 362.16
- higher 1 3.506 894.86 362.17
- paid 1 4.104 895.46 362.31
- famsup 1 4.164 895.52 362.33
- nursery 1 5.298 896.65 362.60
- Mjob 4 31.633 922.99 362.79
- activities 1 7.679 899.04 363.16
<none> 891.36 363.33
- school 1 8.652 900.01 363.40
- sex 1 8.973 900.33 363.47
- Dalc 1 10.176 901.53 363.76
- address 1 14.821 906.18 364.86
- studytime 1 17.697 909.05 365.54
- absences 1 29.530 920.89 368.30
- goout 1 32.214 923.57 368.93
- failures 1 61.869 953.23 375.69
Step: AIC=361.34
G3 ~ school + sex + age + address + Pstatus + Mjob + traveltime +
studytime + failures + schoolsup + famsup + paid + activities +
nursery + higher + internet + romantic + famrel + freetime +
goout + Dalc + Walc + health + absences
Df Sum of Sq RSS AIC
- famrel 1 0.173 891.56 359.38
- Pstatus 1 0.261 891.65 359.40
- freetime 1 0.360 891.74 359.42
- romantic 1 0.754 892.14 359.52
- Walc 1 0.792 892.18 359.53
- age 1 0.890 892.27 359.55
- internet 1 0.972 892.36 359.57
- traveltime 1 1.147 892.53 359.61
- health 1 1.159 892.54 359.61
- schoolsup 1 3.481 894.86 360.17
- higher 1 3.512 894.90 360.18
- paid 1 4.133 895.52 360.33
- famsup 1 4.145 895.53 360.33
- nursery 1 5.441 896.83 360.64
- Mjob 4 31.713 923.10 360.82
- activities 1 7.657 899.04 361.17
<none> 891.38 361.34
- school 1 8.746 900.13 361.42
- sex 1 8.948 900.33 361.47
- Dalc 1 10.150 901.53 361.76
- address 1 14.989 906.37 362.90
- studytime 1 17.738 909.12 363.55
- absences 1 29.551 920.93 366.31
- goout 1 32.349 923.73 366.96
- failures 1 61.924 953.31 373.71
Step: AIC=359.38
G3 ~ school + sex + age + address + Pstatus + Mjob + traveltime +
studytime + failures + schoolsup + famsup + paid + activities +
nursery + higher + internet + romantic + freetime + goout +
Dalc + Walc + health + absences
Df Sum of Sq RSS AIC
- Pstatus 1 0.237 891.79 357.43
- freetime 1 0.320 891.88 357.45
- Walc 1 0.721 892.28 357.55
- romantic 1 0.759 892.32 357.56
- age 1 0.783 892.34 357.56
- internet 1 0.967 892.52 357.61
- traveltime 1 1.180 892.74 357.66
- health 1 1.302 892.86 357.69
- higher 1 3.509 895.07 358.22
- schoolsup 1 3.730 895.29 358.27
- famsup 1 4.043 895.60 358.34
- paid 1 4.088 895.64 358.36
- nursery 1 5.517 897.07 358.70
- Mjob 4 32.533 924.09 359.05
- activities 1 7.833 899.39 359.25
<none> 891.56 359.38
- school 1 8.591 900.15 359.43
- sex 1 8.785 900.34 359.47
- Dalc 1 10.553 902.11 359.89
- address 1 15.168 906.72 360.99
- studytime 1 17.751 909.31 361.60
- absences 1 29.446 921.00 364.33
- goout 1 33.672 925.23 365.31
- failures 1 61.777 953.33 371.71
Step: AIC=357.43
G3 ~ school + sex + age + address + Mjob + traveltime + studytime +
failures + schoolsup + famsup + paid + activities + nursery +
higher + internet + romantic + freetime + goout + Dalc +
Walc + health + absences
Df Sum of Sq RSS AIC
- freetime 1 0.393 892.19 355.53
- romantic 1 0.734 892.53 355.61
- Walc 1 0.790 892.58 355.62
- internet 1 0.856 892.65 355.64
- age 1 0.865 892.66 355.64
- traveltime 1 1.167 892.96 355.71
- health 1 1.328 893.12 355.75
- higher 1 3.510 895.30 356.27
- schoolsup 1 3.851 895.64 356.36
- famsup 1 3.904 895.70 356.37
- paid 1 4.337 896.13 356.47
- nursery 1 5.610 897.40 356.78
- Mjob 4 32.387 924.18 357.07
<none> 891.79 357.43
- activities 1 8.471 900.26 357.46
- school 1 8.555 900.35 357.48
- sex 1 9.120 900.91 357.61
- Dalc 1 10.434 902.23 357.92
- address 1 15.119 906.91 359.03
- studytime 1 17.625 909.42 359.62
- absences 1 31.137 922.93 362.78
- goout 1 33.963 925.76 363.43
- failures 1 61.749 953.54 369.76
Step: AIC=355.53
G3 ~ school + sex + age + address + Mjob + traveltime + studytime +
failures + schoolsup + famsup + paid + activities + nursery +
higher + internet + romantic + goout + Dalc + Walc + health +
absences
Df Sum of Sq RSS AIC
- romantic 1 0.743 892.93 353.71
- internet 1 0.873 893.06 353.74
- age 1 0.878 893.06 353.74
- Walc 1 0.943 893.13 353.75
- traveltime 1 1.099 893.28 353.79
- health 1 1.328 893.51 353.85
- higher 1 3.576 895.76 354.38
- famsup 1 3.807 895.99 354.44
- schoolsup 1 3.955 896.14 354.47
- paid 1 4.371 896.56 354.57
- nursery 1 5.809 897.99 354.92
- Mjob 4 32.890 925.08 355.27
<none> 892.19 355.53
- school 1 8.466 900.65 355.55
- activities 1 8.650 900.84 355.59
- sex 1 9.728 901.91 355.85
- Dalc 1 11.010 903.20 356.15
- address 1 14.924 907.11 357.08
- studytime 1 17.236 909.42 357.62
- absences 1 31.582 923.77 360.97
- goout 1 34.992 927.18 361.76
- failures 1 61.373 953.56 367.76
Step: AIC=353.71
G3 ~ school + sex + age + address + Mjob + traveltime + studytime +
failures + schoolsup + famsup + paid + activities + nursery +
higher + internet + goout + Dalc + Walc + health + absences
Df Sum of Sq RSS AIC
- Walc 1 0.768 893.70 351.89
- internet 1 0.822 893.75 351.90
- age 1 1.116 894.04 351.97
- traveltime 1 1.200 894.13 351.99
- health 1 1.440 894.37 352.05
- higher 1 3.931 896.86 352.65
- famsup 1 3.974 896.90 352.66
- schoolsup 1 4.136 897.06 352.69
- paid 1 4.382 897.31 352.75
- nursery 1 5.476 898.40 353.01
- school 1 8.246 901.17 353.67
<none> 892.93 353.71
- Mjob 4 34.369 927.30 353.79
- activities 1 9.014 901.94 353.86
- sex 1 9.236 902.16 353.91
- Dalc 1 10.926 903.86 354.31
- address 1 15.568 908.50 355.40
- studytime 1 17.880 910.81 355.95
- absences 1 30.856 923.78 358.98
- goout 1 35.702 928.63 360.10
- failures 1 62.751 955.68 366.24
Step: AIC=351.89
G3 ~ school + sex + age + address + Mjob + traveltime + studytime +
failures + schoolsup + famsup + paid + activities + nursery +
higher + internet + goout + Dalc + health + absences
Df Sum of Sq RSS AIC
- internet 1 0.832 894.53 350.09
- traveltime 1 1.178 894.87 350.17
- age 1 1.280 894.98 350.20
- health 1 1.674 895.37 350.29
- schoolsup 1 3.747 897.44 350.78
- famsup 1 3.907 897.60 350.82
- higher 1 4.045 897.74 350.86
- paid 1 4.107 897.80 350.87
- nursery 1 5.185 898.88 351.13
- school 1 8.093 901.79 351.82
<none> 893.70 351.89
- sex 1 8.529 902.23 351.92
- Mjob 4 34.632 928.33 352.03
- activities 1 9.147 902.84 352.07
- Dalc 1 10.831 904.53 352.47
- address 1 16.038 909.73 353.70
- studytime 1 18.926 912.62 354.37
- absences 1 33.120 926.82 357.68
- goout 1 44.792 938.49 360.36
- failures 1 64.389 958.09 364.78
Step: AIC=350.09
G3 ~ school + sex + age + address + Mjob + traveltime + studytime +
failures + schoolsup + famsup + paid + activities + nursery +
higher + goout + Dalc + health + absences
Df Sum of Sq RSS AIC
- traveltime 1 1.205 895.73 348.38
- age 1 1.372 895.90 348.42
- health 1 1.457 895.99 348.44
- paid 1 3.700 898.23 348.97
- famsup 1 3.862 898.39 349.01
- schoolsup 1 3.869 898.40 349.01
- higher 1 4.046 898.58 349.05
- nursery 1 5.122 899.65 349.31
- school 1 8.066 902.60 350.01
- sex 1 8.380 902.91 350.08
<none> 894.53 350.09
- Mjob 4 34.276 928.81 350.14
- activities 1 9.092 903.62 350.25
- Dalc 1 10.697 905.23 350.63
- address 1 15.359 909.89 351.73
- studytime 1 18.504 913.03 352.47
- absences 1 35.357 929.89 356.38
- goout 1 46.356 940.89 358.90
- failures 1 63.557 958.09 362.78
Step: AIC=348.38
G3 ~ school + sex + age + address + Mjob + studytime + failures +
schoolsup + famsup + paid + activities + nursery + higher +
goout + Dalc + health + absences
Df Sum of Sq RSS AIC
- age 1 1.129 896.86 346.65
- health 1 1.463 897.20 346.73
- paid 1 3.902 899.64 347.31
- famsup 1 3.993 899.73 347.33
- schoolsup 1 4.111 899.84 347.36
- higher 1 4.645 900.38 347.48
- nursery 1 4.855 900.59 347.53
- school 1 7.129 902.86 348.07
- Mjob 4 33.484 929.22 348.23
<none> 895.73 348.38
- sex 1 8.708 904.44 348.45
- activities 1 9.307 905.04 348.59
- Dalc 1 10.420 906.15 348.85
- address 1 14.186 909.92 349.74
- studytime 1 18.379 914.11 350.72
- absences 1 35.641 931.37 354.73
- goout 1 46.348 942.08 357.17
- failures 1 62.368 958.10 360.78
Step: AIC=346.65
G3 ~ school + sex + address + Mjob + studytime + failures + schoolsup +
famsup + paid + activities + nursery + higher + goout + Dalc +
health + absences
Df Sum of Sq RSS AIC
- health 1 1.494 898.36 345.00
- paid 1 3.929 900.79 345.58
- famsup 1 4.117 900.98 345.63
- nursery 1 5.106 901.97 345.86
- higher 1 5.181 902.04 345.88
- schoolsup 1 5.555 902.42 345.97
- school 1 6.043 902.91 346.08
- Mjob 4 32.895 929.76 346.35
<none> 896.86 346.65
- sex 1 8.662 905.52 346.70
- activities 1 8.875 905.74 346.75
- Dalc 1 10.389 907.25 347.11
- address 1 13.763 910.63 347.91
- studytime 1 19.266 916.13 349.19
- absences 1 34.513 931.37 352.73
- goout 1 45.430 942.29 355.22
- failures 1 61.840 958.70 358.92
Step: AIC=345
G3 ~ school + sex + address + Mjob + studytime + failures + schoolsup +
famsup + paid + activities + nursery + higher + goout + Dalc +
absences
Df Sum of Sq RSS AIC
- paid 1 3.868 902.22 343.92
- famsup 1 4.211 902.57 344.00
- nursery 1 5.102 903.46 344.21
- higher 1 5.326 903.68 344.27
- schoolsup 1 5.371 903.73 344.28
- school 1 5.855 904.21 344.39
- Mjob 4 31.786 930.14 344.44
<none> 898.36 345.00
- sex 1 8.595 906.95 345.04
- activities 1 9.235 907.59 345.19
- Dalc 1 9.792 908.15 345.32
- address 1 14.177 912.53 346.35
- studytime 1 20.456 918.81 347.82
- absences 1 33.903 932.26 350.93
- goout 1 44.888 943.24 353.44
- failures 1 62.365 960.72 357.37
Step: AIC=343.92
G3 ~ school + sex + address + Mjob + studytime + failures + schoolsup +
famsup + activities + nursery + higher + goout + Dalc + absences
Df Sum of Sq RSS AIC
- famsup 1 2.669 904.89 342.55
- nursery 1 4.396 906.62 342.96
- higher 1 4.483 906.71 342.98
- Mjob 4 31.089 933.31 343.17
- school 1 5.625 907.85 343.25
- schoolsup 1 5.748 907.97 343.28
- sex 1 7.586 909.81 343.71
- activities 1 8.104 910.33 343.84
<none> 902.22 343.92
- Dalc 1 11.099 913.32 344.54
- address 1 13.441 915.67 345.09
- studytime 1 24.800 927.02 347.72
- absences 1 33.936 936.16 349.82
- goout 1 44.784 947.01 352.29
- failures 1 64.064 966.29 356.60
Step: AIC=342.55
G3 ~ school + sex + address + Mjob + studytime + failures + schoolsup +
activities + nursery + higher + goout + Dalc + absences
Df Sum of Sq RSS AIC
- Mjob 4 28.859 933.75 341.27
- nursery 1 4.102 909.00 341.52
- higher 1 4.575 909.47 341.63
- school 1 4.830 909.72 341.69
- schoolsup 1 6.597 911.49 342.11
- activities 1 8.247 913.14 342.50
<none> 904.89 342.55
- sex 1 9.493 914.39 342.79
- Dalc 1 10.634 915.53 343.05
- address 1 14.000 918.89 343.84
- studytime 1 23.656 928.55 346.08
- absences 1 33.952 938.85 348.44
- goout 1 45.351 950.24 351.02
- failures 1 62.076 966.97 354.75
Step: AIC=341.27
G3 ~ school + sex + address + studytime + failures + schoolsup +
activities + nursery + higher + goout + Dalc + absences
Df Sum of Sq RSS AIC
- higher 1 2.544 936.30 339.85
- nursery 1 3.300 937.05 340.03
- school 1 5.738 939.49 340.58
- activities 1 6.604 940.36 340.78
- schoolsup 1 7.273 941.03 340.93
- Dalc 1 7.334 941.09 340.95
<none> 933.75 341.27
- sex 1 12.155 945.91 342.04
- address 1 15.614 949.37 342.82
- studytime 1 18.323 952.08 343.43
- absences 1 32.306 966.06 346.55
- goout 1 36.907 970.66 347.57
- failures 1 57.539 991.29 352.07
Step: AIC=339.85
G3 ~ school + sex + address + studytime + failures + schoolsup +
activities + nursery + goout + Dalc + absences
Df Sum of Sq RSS AIC
- nursery 1 3.745 940.04 338.71
- school 1 5.372 941.67 339.08
- activities 1 6.023 942.32 339.23
- Dalc 1 6.560 942.86 339.35
- schoolsup 1 7.469 943.77 339.55
<none> 936.30 339.85
- sex 1 14.398 950.69 341.12
- address 1 15.218 951.51 341.30
- studytime 1 17.742 954.04 341.87
- absences 1 30.266 966.56 344.66
- goout 1 38.949 975.25 346.58
- failures 1 55.098 991.39 350.09
Step: AIC=338.71
G3 ~ school + sex + address + studytime + failures + schoolsup +
activities + goout + Dalc + absences
Df Sum of Sq RSS AIC
- school 1 5.025 945.07 337.85
- activities 1 6.277 946.32 338.13
- Dalc 1 6.681 946.72 338.22
- schoolsup 1 7.978 948.02 338.52
<none> 940.04 338.71
- address 1 13.750 953.79 339.82
- sex 1 15.573 955.61 340.22
- studytime 1 17.445 957.49 340.64
- absences 1 29.515 969.56 343.32
- goout 1 41.370 981.41 345.93
- failures 1 53.366 993.41 348.53
Step: AIC=337.85
G3 ~ sex + address + studytime + failures + schoolsup + activities +
goout + Dalc + absences
Df Sum of Sq RSS AIC
- Dalc 1 5.606 950.67 337.12
- schoolsup 1 6.666 951.73 337.35
- activities 1 8.239 953.31 337.71
<none> 945.07 337.85
- sex 1 15.633 960.70 339.36
- studytime 1 20.818 965.88 340.51
- address 1 21.896 966.96 340.75
- absences 1 26.956 972.02 341.87
- goout 1 42.577 987.64 345.28
- failures 1 52.706 997.77 347.46
Step: AIC=337.12
G3 ~ sex + address + studytime + failures + schoolsup + activities +
goout + absences
Df Sum of Sq RSS AIC
- activities 1 5.991 956.66 336.46
- schoolsup 1 7.160 957.83 336.72
<none> 950.67 337.12
- address 1 18.871 969.54 339.32
- studytime 1 20.347 971.02 339.65
- absences 1 23.952 974.62 340.44
- sex 1 24.752 975.42 340.62
- goout 1 37.787 988.46 343.46
- failures 1 49.046 999.72 345.88
Step: AIC=336.46
G3 ~ sex + address + studytime + failures + schoolsup + goout +
absences
Df Sum of Sq RSS AIC
- schoolsup 1 5.570 962.23 335.70
<none> 956.66 336.46
- address 1 15.417 972.08 337.88
- absences 1 22.411 979.07 339.42
- studytime 1 23.997 980.66 339.76
- sex 1 29.866 986.53 341.04
- goout 1 36.975 993.64 342.57
- failures 1 51.533 1008.20 345.69
Step: AIC=335.7
G3 ~ sex + address + studytime + failures + goout + absences
Df Sum of Sq RSS AIC
<none> 962.23 335.70
- address 1 13.323 975.56 336.64
- absences 1 21.649 983.88 338.46
- studytime 1 24.962 987.20 339.18
- goout 1 34.506 996.74 341.24
- sex 1 35.079 997.31 341.36
- failures 1 52.412 1014.65 345.05
Call:
lm(formula = G3 ~ sex + address + studytime + failures + goout +
absences, data = training)
Coefficients:
(Intercept) sexM addressU studytime failures goout
10.25165 0.86537 0.60566 0.45018 -0.64907 -0.37654
absences
-0.03561
Model 2¶
Model 2 will be equivalent to the output of the step function.
In [20]:
<span class="o">%%</span>R
model2 <span class="o"><-</span> lm<span class="p">(</span>formula <span class="o">=</span> G3 <span class="o">~</span> sex <span class="o">+</span> address <span class="o">+</span> studytime <span class="o">+</span> failures <span class="o">+</span> goout <span class="o">+</span>
absences<span class="p">,</span> data <span class="o">=</span> training<span class="p">)</span>
In [21]:
<span class="o">%%</span>R
subs <span class="o"><-</span> regsubsets<span class="p">(</span>G3 <span class="o">~</span> sex <span class="o">+</span> address <span class="o">+</span> studytime <span class="o">+</span> failures <span class="o">+</span> goout <span class="o">+</span> absences<span class="p">,</span> data <span class="o">=</span> training<span class="p">)</span>
df <span class="o"><-</span> data.frame<span class="p">(</span>est <span class="o">=</span> c<span class="p">(</span>summary<span class="p">(</span>subs<span class="p">)</span><span class="o">$</span>adjr2<span class="p">,</span>
summary<span class="p">(</span>subs<span class="p">)</span><span class="o">$</span>bic<span class="p">),</span>
x <span class="o">=</span> rep<span class="p">(</span><span class="m">1</span><span class="o">:</span><span class="m">6</span><span class="p">,</span> <span class="m">6</span><span class="p">),</span>
type <span class="o">=</span> rep<span class="p">(</span>c<span class="p">(</span><span class="s">"adjr2"</span><span class="p">,</span> <span class="s">"bic"</span><span class="p">),</span>
each <span class="o">=</span> <span class="m">6</span><span class="p">))</span>
qplot<span class="p">(</span>x<span class="p">,</span> est<span class="p">,</span> data <span class="o">=</span> df<span class="p">,</span> geom <span class="o">=</span> <span class="s">"line"</span><span class="p">)</span> <span class="o">+</span>
theme_bw<span class="p">()</span> <span class="o">+</span> facet_grid<span class="p">(</span>type <span class="o">~</span> .<span class="p">,</span> scales <span class="o">=</span> <span class="s">"free_y"</span><span class="p">)</span>
In [22]:
<span class="o">%%</span>R summary<span class="p">(</span>model2<span class="p">)</span>
Call:
lm(formula = G3 ~ sex + address + studytime + failures + goout +
absences, data = training)
Residuals:
Min 1Q Median 3Q Max
-5.2007 -1.3576 -0.1115 1.6244 4.4124
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 10.25165 0.68067 15.061 < 2e-16 ***
sexM 0.86537 0.31502 2.747 0.006544 **
addressU 0.60566 0.35776 1.693 0.091972 .
studytime 0.45018 0.19427 2.317 0.021464 *
failures -0.64907 0.19330 -3.358 0.000935 ***
goout -0.37654 0.13820 -2.725 0.006990 **
absences -0.03561 0.01650 -2.158 0.032075 *
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 2.156 on 207 degrees of freedom
Multiple R-squared: 0.1732, Adjusted R-squared: 0.1492
F-statistic: 7.226 on 6 and 207 DF, p-value: 5.128e-07
Model 3¶
Model 3 will be our final model.
In [23]:
<span class="o">%%</span>R model3 <span class="o"><-</span> lm<span class="p">(</span>formula <span class="o">=</span> G3 <span class="o">~</span> sex <span class="o">+</span> failures<span class="p">,</span> data <span class="o">=</span> training<span class="p">)</span> summary<span class="p">(</span>model3<span class="p">)</span>
Call:
lm(formula = G3 ~ sex + failures, data = training)
Residuals:
Min 1Q Median 3Q Max
-5.3655 -1.3655 -0.0253 1.6345 3.8171
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 10.3655 0.2136 48.520 < 2e-16 ***
sexM 0.6599 0.3088 2.137 0.0337 *
failures -0.8424 0.1949 -4.323 2.37e-05 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 2.235 on 211 degrees of freedom
Multiple R-squared: 0.09429, Adjusted R-squared: 0.08571
F-statistic: 10.98 on 2 and 211 DF, p-value: 2.898e-05
ANOVA¶
We can now compare the 3 models we made using ANOVA.
In [24]:
<span class="o">%%</span>R anova<span class="p">(</span>saturated14<span class="p">,</span>model2<span class="p">,</span>model3<span class="p">)</span>
Analysis of Variance Table
Model 1: G3 ~ (school + sex + age + address + famsize + Pstatus + Medu +
Fedu + Mjob + Fjob + reason + guardian + traveltime + studytime +
failures + schoolsup + famsup + paid + activities + nursery +
higher + internet + romantic + famrel + freetime + goout +
Dalc + Walc + health + absences + G1 + G2) - G1 - G2
Model 2: G3 ~ sex + address + studytime + failures + goout + absences
Model 3: G3 ~ sex + failures
Res.Df RSS Df Sum of Sq F Pr(>F)
1 174 871.86
2 207 962.23 -33 -90.368 0.5465 0.978867
3 211 1054.04 -4 -91.806 4.5805 0.001532 **
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
In this case, ANOVA isn’t very useful since the strongest predictors from the original model have been cut out. By comparing models graphically, it’s easier to get an idea of what’s going on.
By removing the strong predictors of the original model, single predictors become less important and holistic models become more accurate. Below, we see that Model 1 performs the best on the test set.
This gives insight into how we should approach these students early on. One indicator will not make or break a child, but the overall profile can still be a strong indicator.
In [25]:
<span class="o">%%</span>R
<span class="c1">#Models</span>
final1 <span class="o"><-</span> lm<span class="p">(</span>G3 <span class="o">~</span> . <span class="o">-</span>G1 <span class="o">-</span>G2<span class="p">,</span> data<span class="o">=</span>testing<span class="p">)</span>
final2 <span class="o"><-</span> lm<span class="p">(</span>G3 <span class="o">~</span> sex <span class="o">+</span> address <span class="o">+</span> studytime <span class="o">+</span> failures <span class="o">+</span> goout <span class="o">+</span> absences<span class="p">,</span> data<span class="o">=</span> testing<span class="p">)</span>
final3 <span class="o"><-</span> lm<span class="p">(</span>G3 <span class="o">~</span> sex <span class="o">+</span> failures<span class="p">,</span> data<span class="o">=</span>testing<span class="p">)</span>
<span class="c1">#Graphs</span>
plot1 <span class="o"><-</span> qplot<span class="p">(</span>G3<span class="p">,</span> predict<span class="p">(</span>final1<span class="p">),</span> data <span class="o">=</span> testing<span class="p">,</span> geom <span class="o">=</span> <span class="s">"point"</span><span class="p">,</span> position <span class="o">=</span> <span class="s">"jitter"</span><span class="p">,</span>
alpha<span class="o">=</span><span class="m">.8</span><span class="p">,</span> main<span class="o">=</span><span class="s">"Model 1"</span><span class="p">)</span> <span class="o">+</span>
geom_abline<span class="p">(</span>intercept<span class="o">=</span><span class="m">0</span><span class="p">,</span> slope<span class="o">=</span><span class="m">1</span><span class="p">)</span> <span class="o">+</span>
theme<span class="p">(</span>legend.position<span class="o">=</span><span class="s">"none"</span><span class="p">)</span>
plot2 <span class="o"><-</span> qplot<span class="p">(</span>G3<span class="p">,</span> predict<span class="p">(</span>final2<span class="p">),</span> data <span class="o">=</span> testing<span class="p">,</span> geom <span class="o">=</span> <span class="s">"point"</span><span class="p">,</span> position <span class="o">=</span> <span class="s">"jitter"</span><span class="p">,</span>
alpha<span class="o">=</span><span class="m">.8</span><span class="p">,</span> main<span class="o">=</span><span class="s">"Model 2"</span><span class="p">)</span> <span class="o">+</span>
geom_abline<span class="p">(</span>intercept<span class="o">=</span><span class="m">0</span><span class="p">,</span> slope<span class="o">=</span><span class="m">1</span><span class="p">)</span> <span class="o">+</span>
theme<span class="p">(</span>legend.position<span class="o">=</span><span class="s">"none"</span><span class="p">)</span>
plot3 <span class="o"><-</span> qplot<span class="p">(</span>G3<span class="p">,</span> predict<span class="p">(</span>final3<span class="p">),</span> data <span class="o">=</span> testing<span class="p">,</span> geom <span class="o">=</span> <span class="s">"point"</span><span class="p">,</span> position <span class="o">=</span> <span class="s">"jitter"</span><span class="p">,</span>
alpha<span class="o">=</span><span class="m">.8</span><span class="p">,</span> main<span class="o">=</span><span class="s">"Model 3"</span><span class="p">)</span> <span class="o">+</span>
geom_abline<span class="p">(</span>intercept<span class="o">=</span><span class="m">0</span><span class="p">,</span>slope<span class="o">=</span><span class="m">1</span><span class="p">)</span> <span class="o">+</span>
theme<span class="p">(</span>legend.position<...To leave a comment for the author, please follow the link and comment on their blog: NYC Data Science Academy » R.
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.