Example 3.5

[crossxwill]

The example below was supposed to show K=5 is better than K=1. However, the test error for K=1 is much lower than K=5.

The test set sample’s sale price is $176K and the neighbor’s prices, from closest to farthest, are: $175K, $128K, $100K, $120K, $125K. Using K = 1, the model would miss the true house price by $0.9K. This illustrates the concept of overfitting introduced in Section 1.2.1; the model is too aggressively using patterns in the training set to make predictions on new data points. For this model, increasing the number of neighbors might help alleviate the issue. Averaging all K = 5 points to make a prediction substantially cuts the error to $-46.4K.