We now split our data to the train-validation sets (for grid search) and the test set (for inference).
X = df.drop(columns=['class',
'image']).copy().to_numpy()y = df['class'].to_numpy()
X_trainval, X_test, y_trainval, y_test = train_test_split(
X, y, test_size=0.25, random_state=143, stratify=y
)
We define our hyperparameters and cross-validation strategy for the pipeline and fit the model.
rng = np.random.RandomState(143)skfold = StratifiedKFold(n_splits=5, shuffle=True, random_state=rng)
pipeline = Pipeline([('scaler', StandardScaler()),
('classifier', LogisticRegression())])
param_grid = [
{
'classifier': [KNeighborsClassifier()],
'classifier__n_neighbors': [3, 5, 10, 15]
},
{
'classifier': [LogisticRegression(solver='saga', max_iter=10_000)],
'classifier__penalty': ['l1', 'l2'],
'classifier__C': [0.1, 1, 10]
},
{
'classifier': [SVC(random_state=rng)],
'classifier__kernel': ['linear', 'rbf'],
'classifier__C': [0.1, 1, 10]
},
{
'classifier': [RandomForestClassifier(random_state=rng)],
'classifier__n_estimators': [100, 150, 200],
'classifier__max_depth': [3, 5, 7]
},
{
'classifier': [GradientBoostingClassifier(random_state=rng)],
'classifier__n_estimators': [100, 150, 200],
'classifier__max_depth': [3, 5, 7]
}
]
gs = GridSearchCV(pipeline,
param_grid=param_grid,
cv=skfold,
verbose=1)
gs.fit(X_trainval, y_trainval)
This will provide us an optimal model that classifies the leaves most accurately. The next step is to evaluate this model on the test set.
