SVM, Precision & Recall.
I completed the final honors exercise today. Here’s a confusion matrix of my support vector machine model model. This chart is read diagonally left to right, and then diagonally right to left. In other words, we start with the diagonal top left to bottom right:
- Top Left (True Positives): the model correctly predicted rain in 98 cases.
- Bottom Right (True Negatives): the model correctly predicted that it would not rain in 445 cases.
Then we read the chart going from top right to bottom left:
- Top Right (False Negatives): the model incorrectly predicted ‘No Rain’ in 84 cases where it actually rained.
- Bottom Left (False Positives): the model incorrectly predict ‘Rain’ in 28 cases where it did not actually rain.
Key takeaways:
- Useful metrics that come from this chart include:
- Precision: This is a metric that tells us how many predictions actually happened out of all the predictions for rain. e.g. in the example above:
- Precision for ‘Rain’ predictions: The model predicted rain cases of 126 total (i.e. 98 + 28) but only 98 were actually true rain cases in the test data. This gives a precision rate of 77.8%.
- Sensitivity (or Recall): This is a metric that tells us what proportion of predictions were correct vs. wrong. e.g. in the example above:
- Sensitivity for ‘Rain’: The model correctly predicted rain cases of 98 rain cases out of a total of 182 (98 + 84) actual rain occurrences. This gives a sensitivity ratio of 53.8%.
- Precision: This is a metric that tells us how many predictions actually happened out of all the predictions for rain. e.g. in the example above:
Update on where I am with the honors labs:
Linear Regression – completed.KNN – completedDecision Trees – completedLogistic Regression – completedSVM – completed