Which practice helps prevent data leakage during model evaluation?

Which practice helps prevent data leakage during model evaluation?

• A. Use k-fold cross-validation without a separate test set.
• B. Keep train, validation, and test sets separate and perform leakage checks.
• C. Use the test set to tune hyperparameters.
• D. Randomly shuffle labels before training.

Answer: (B)

Separating data into distinct training, validation, and test sets and checking for leakage protects the integrity of the evaluation. Data leakage happens when information from the evaluation data leaks into the training process, causing the model to appear to perform better than it would on truly new data. With this setup, you train only on the training data, use the validation set to tune hyperparameters or select features, and keep the test set untouched for the final performance estimate. Leakage checks make sure that any preprocessing steps—like scaling, imputation, or feature selection—are learned from the training data alone and then applied to validation and test data. For example, you calculate scaling parameters using only training data and apply them to the rest, and you perform imputation or feature selection without peeking at the test data. If you skip a separate test set or allow preprocessing or tuning to use test information, you risk optimistic results that won’t generalize. Maintaining separate sets and routinely checking for leakage gives a trustworthy, realistic view of how the model will perform in the real world.

Separating data into distinct training, validation, and test sets and checking for leakage protects the integrity of the evaluation. Data leakage happens when information from the evaluation data leaks into the training process, causing the model to appear to perform better than it would on truly new data.

With this setup, you train only on the training data, use the validation set to tune hyperparameters or select features, and keep the test set untouched for the final performance estimate. Leakage checks make sure that any preprocessing steps—like scaling, imputation, or feature selection—are learned from the training data alone and then applied to validation and test data. For example, you calculate scaling parameters using only training data and apply them to the rest, and you perform imputation or feature selection without peeking at the test data.

If you skip a separate test set or allow preprocessing or tuning to use test information, you risk optimistic results that won’t generalize. Maintaining separate sets and routinely checking for leakage gives a trustworthy, realistic view of how the model will perform in the real world.