6.1. Exercises

True or False: Scoring with Cross-Validation

Scoring and Cross Validation

Instructions:
Running a coding exercise for the first time could take a bit of time for everything to load. Be patient, it could take a few minutes.

When you see ____ in a coding exercise, replace it with what you assume to be the correct code. Run it and see if you obtain the desired output. Submit your code to validate if you were correct.

Make sure you remove the hash (#) symbol in the coding portions of this question. We have commented them so that the line won’t execute and you can test your code after each step.

Let’s bring back the Pokémon dataset that we saw in exercise 15.

We’ve built our pipeline and looked at the classification reports but this time we want to do cross-validation and look at the scores from cross-validation of not just accuracy, but precision and recall as well.

Tasks:

• Build a pipeline containing the column transformer and an SVC model and set class_weight="balanced" in the SVM classifier. Name this pipeline main_pipe.
• Perform cross-validation using cross-validate on the training split using the scoring measures accuracy, precision and recall.
• Save the results in a dataframe named multi_scores.

Hint 1

• Are you coding main_pipe as make_pipeline(preprocessor, SVC()).
• Are you specifying scoring = ['accuracy', 'precision', 'recall'] in your cross validation function?
• Are you calling cross_validate on main_pipe, X_train, and y_train?
• Are you specifying return_train_score=True in cross_validate?
• Are you saving the result in a dataframe?

Fully worked solution: