R for Statistical Learning
Introduction
About This Book
Organization
Who?
Caveat Emptor
Conventions
0.0.1
Mathematics
0.0.2
Code
Acknowledgements
License
I Prerequisites
1
Overview
2
Probability Review
2.1
Probability Models
2.2
Probability Axioms
2.3
Probability Rules
2.4
Random Variables
2.4.1
Distributions
2.4.2
Discrete Random Variables
2.4.3
Continuous Random Variables
2.4.4
Several Random Variables
2.5
Expectations
2.6
Likelihood
2.7
Videos
2.8
References
3
R
, RStudio, RMarkdown
3.1
Videos
3.2
Template
4
Modeling Basics in
R
4.1
Visualization for Regression
4.2
The
lm()
Function
4.3
Hypothesis Testing
4.4
Prediction
4.5
Unusual Observations
4.6
Adding Complexity
4.6.1
Interactions
4.6.2
Polynomials
4.6.3
Transformations
4.7
rmarkdown
II Regression
5
Overview
6
Linear Models
6.1
Assesing Model Accuracy
6.2
Model Complexity
6.3
Test-Train Split
6.4
Adding Flexibility to Linear Models
6.5
Choosing a Model
7
\(k\)
-Nearest Neighbors
7.1
Parametric versus Non-Parametric Models
7.2
Local Approaches
7.2.1
Neighbors
7.2.2
Neighborhoods
7.3
\(k\)
-Nearest Neighbors
7.4
Tuning Parameters versus Model Parameters
7.5
KNN in
R
7.6
Choosing
\(k\)
7.7
Linear versus Non-Linear
7.8
Scaling Data
7.9
Curse of Dimensionality
7.10
Train Time versus Test Time
7.11
Interpretability
7.12
Data Example
7.13
rmarkdown
8
Bias–Variance Tradeoff
8.1
Reducible and Irreducible Error
8.2
Bias-Variance Decomposition
8.3
Simulation
8.4
Estimating Expected Prediction Error
8.5
rmarkdown
III Classification
9
Overview
9.1
Visualization for Classification
9.2
A Simple Classifier
9.3
Metrics for Classification
9.4
rmarkdown
10
Logistic Regression
10.1
Linear Regression
10.2
Bayes Classifier
10.3
Logistic Regression with
glm()
10.4
ROC Curves
10.5
Multinomial Logistic Regression
10.6
rmarkdown
11
Generative Models
11.1
Linear Discriminant Analysis
11.2
Quadratic Discriminant Analysis
11.3
Naive Bayes
11.4
Discrete Inputs
11.5
rmarkdown
12
k-Nearest Neighbors
12.1
Binary Data Example
12.2
Categorical Data
12.3
External Links
12.4
rmarkdown
IV Unsupervised Learning
13
Overview
13.1
Methods
13.1.1
Principal Component Analysis
13.1.2
\(k\)
-Means Clustering
13.1.3
Hierarchical Clustering
13.2
Examples
13.2.1
US Arrests
13.2.2
Simulated Data
13.2.3
Iris Data
13.3
External Links
13.4
RMarkdown
14
Principal Component Analysis
15
k-Means
16
Mixture Models
17
Hierarchical Clustering
V In Practice
18
Overview
19
Supervised Learning Overview
Bayes Classifier
The Bias-Variance Tradeoff
The Test-Train Split
Classification Methods
Discriminative versus Generative Methods
Parametric and Non-Parametric Methods
Tuning Parameters
Cross-Validation
Curse of Dimensionality
No-Free-Lunch Theorem
19.1
External Links
19.2
RMarkdown
20
Resampling
20.1
Validation-Set Approach
20.2
Cross-Validation
20.3
Test Data
20.4
Bootstrap
20.5
Which
\(K\)
?
20.6
Summary
20.7
External Links
20.8
rmarkdown
21
The
caret
Package
21.1
Classification
21.1.1
Tuning
21.2
Regression
21.2.1
Methods
21.3
External Links
21.4
rmarkdown
22
Subset Selection
22.1
AIC, BIC, and Cp
22.1.1
leaps
Package
22.1.2
Best Subset
22.1.3
Stepwise Methods
22.2
Validated RMSE
22.3
External Links
22.4
RMarkdown
VI The Modern Era
23
Overview
24
Regularization
24.1
Ridge Regression
24.2
Lasso
24.3
broom
24.4
Simulated Data,
\(p > n\)
24.5
External Links
24.6
rmarkdown
25
Elastic Net
25.1
Regression
25.2
Classification
25.3
External Links
25.4
rmarkdown
26
Trees
26.1
Classification Trees
26.2
Regression Trees
26.3
rpart
Package
26.4
External Links
26.5
rmarkdown
27
Ensemble Methods
27.1
Regression
27.1.1
Tree Model
27.1.2
Linear Model
27.1.3
Bagging
27.1.4
Random Forest
27.1.5
Boosting
27.1.6
Results
27.2
Classification
27.2.1
Tree Model
27.2.2
Logistic Regression
27.2.3
Bagging
27.2.4
Random Forest
27.2.5
Boosting
27.2.6
Results
27.3
Tuning
27.3.1
Random Forest and Bagging
27.3.2
Boosting
27.4
Tree versus Ensemble Boundaries
27.5
External Links
27.6
rmarkdown
28
Artificial Neural Networks
VII Appendix
29
Overview
30
Non-Linear Models
31
Regularized Discriminant Analysis
31.1
Sonar Data
31.2
RDA
31.3
RDA with Grid Search
31.4
RDA with Random Search Search
31.5
Comparison to Elastic Net
31.6
Results
31.7
External Links
31.8
RMarkdown
32
Support Vector Machines
© 2017 - 2019 David Dalpiaz
R for Statistical Learning
Chapter 23
Overview