Toggle Menu
Prev Up Next

scikit-learn 1.3.2
Other versions

Please cite us if you use the software.

# `sklearn.model_selection`.cross_validate

sklearn.model_selection.cross_validate(estimator, X, y=None, *, groups=None, scoring=None, cv=None, n_jobs=None, verbose=0, fit_params=None, pre_dispatch='2*n_jobs', return_train_score=False, return_estimator=False, return_indices=False, error_score=nan)

  Evaluate metric(s) by cross-validation and also record fit/score times.

  Read more in the User Guide.

  Parameters:
    estimator estimator object implementing ‘fit’

      The object to use to fit the data.

    X{array-like, sparse matrix} of shape (n_samples, n_features)

      The data to fit. Can be for example a list, or an array.

    y array-like of shape (n_samples,) or (n_samples, n_outputs), default=None

      The target variable to try to predict in the case of supervised learning.

    groups array-like of shape (n_samples,), default=None

      Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., `GroupKFold`).

    scoring str, callable, list, tuple, or dict, default=None

      Strategy to evaluate the performance of the cross-validated model on the test set.

      If `scoring` represents a single score, one can use:

      - a single string (see The scoring parameter: defining model evaluation rules);

      - a callable (see Defining your scoring strategy from metric functions) that returns a single value.

      If `scoring` represents multiple scores, one can use:

      - a list or tuple of unique strings;

      - a callable returning a dictionary where the keys are the metric names and the values are the metric scores;

      - a dictionary with metric names as keys and callables a values.

      See Specifying multiple metrics for evaluation for an example.

    cv int, cross-validation generator or an iterable, default=None

      Determines the cross-validation splitting strategy. Possible inputs for cv are:

      - None, to use the default 5-fold cross validation,

      - int, to specify the number of folds in a `(Stratified)KFold`,

      - CV splitter,

      - An iterable yielding (train, test) splits as arrays of indices.

      For int/None inputs, if the estimator is a classifier and `y` is either binary or multiclass, `StratifiedKFold` is used. In all other cases, `KFold` is used. These splitters are instantiated with `shuffle=False` so the splits will be the same across calls.

      Refer User Guide for the various cross-validation strategies that can be used here.

      Changed in version 0.22: `cv` default value if None changed from 3-fold to 5-fold.

    n_jobs int, default=None

      Number of jobs to run in parallel. Training the estimator and computing the score are parallelized over the cross-validation splits. `None` means 1 unless in a `joblib.parallel_backend` context. `-1` means using all processors. See Glossary for more details.

    verbose int, default=0

      The verbosity level.

    fit_params dict, default=None

      Parameters to pass to the fit method of the estimator.

    pre_dispatch int or str, default=’2*n_jobs’

      Controls the number of jobs that get dispatched during parallel execution. Reducing this number can be useful to avoid an explosion of memory consumption when more jobs get dispatched than CPUs can process. This parameter can be:

      > - An int, giving the exact number of total jobs that are spawned

      > - A str, giving an expression as a function of n_jobs, as in ‘2*n_jobs’

    return_train_score bool, default=False

      Whether to include train scores. Computing training scores is used to get insights on how different parameter settings impact the overfitting/underfitting trade-off. However computing the scores on the training set can be computationally expensive and is not strictly required to select the parameters that yield the best generalization performance.

      New in version 0.19.

      Changed in version 0.21: Default value was changed from `True` to `False`

    return_estimator bool, default=False

      Whether to return the estimators fitted on each split.

      New in version 0.20.

    return_indices bool, default=False

      Whether to return the train-test indices selected for each split.

      New in version 1.3.

    error_score‘raise’ or numeric, default=np.nan

      Value to assign to the score if an error occurs in estimator fitting. If set to ‘raise’, the error is raised. If a numeric value is given, FitFailedWarning is raised.

      New in version 0.20.

  Returns:
    scores dict of float arrays of shape (n_splits,)

      Array of scores of the estimator for each run of the cross validation.

      A dict of arrays containing the score/time arrays for each scorer is returned. The possible keys for this `dict` are:

      > `test_score`

      >   The score array for test scores on each cv split. Suffix `_score` in `test_score` changes to a specific metric like `test_r2` or `test_auc` if there are multiple scoring metrics in the scoring parameter.

      > `train_score`

      >   The score array for train scores on each cv split. Suffix `_score` in `train_score` changes to a specific metric like `train_r2` or `train_auc` if there are multiple scoring metrics in the scoring parameter. This is available only if `return_train_score` parameter is `True`.

      > `fit_time`

      >   The time for fitting the estimator on the train set for each cv split.

      > `score_time`

      >   The time for scoring the estimator on the test set for each cv split. (Note time for scoring on the train set is not included even if `return_train_score` is set to `True`

      > `estimator`

      >   The estimator objects for each cv split. This is available only if `return_estimator` parameter is set to `True`.

      > `indices`

      >   The train/test positional indices for each cv split. A dictionary is returned where the keys are either `"train"` or `"test"` and the associated values are a list of integer-dtyped NumPy arrays with the indices. Available only if `return_indices=True`.

  See also

  `cross_val_score`

    Run cross-validation for single metric evaluation.

  `cross_val_predict`

    Get predictions from each split of cross-validation for diagnostic purposes.

  `sklearn.metrics.make_scorer`

    Make a scorer from a performance metric or loss function.

  Examples

      >>> from sklearn import datasets, linear_model
      >>> from sklearn.model_selection import cross_validate
      >>> from sklearn.metrics import make_scorer
      >>> from sklearn.metrics import confusion_matrix
      >>> from sklearn.svm import LinearSVC
      >>> diabetes = datasets.load_diabetes()
      >>> X = diabetes.data[:150]
      >>> y = diabetes.target[:150]
      >>> lasso = linear_model.Lasso()

  Single metric evaluation using `cross_validate`

      >>> cv_results = cross_validate(lasso, X, y, cv=3)
      >>> sorted(cv_results.keys())
      ['fit_time', 'score_time', 'test_score']
      >>> cv_results['test_score']
      array([0.3315057 , 0.08022103, 0.03531816])

  Multiple metric evaluation using `cross_validate` (please refer the `scoring` parameter doc for more information)

      >>> scores = cross_validate(lasso, X, y, cv=3,
      ...                         scoring=('r2', 'neg_mean_squared_error'),
      ...                         return_train_score=True)
      >>> print(scores['test_neg_mean_squared_error'])
      [-3635.5... -3573.3... -6114.7...]
      >>> print(scores['train_r2'])
      [0.28009951 0.3908844  0.22784907]

## Examples using `sklearn.model_selection.cross_validate`

Categorical Feature Support in Gradient Boosting

Categorical Feature Support in Gradient Boosting

Combine predictors using stacking

Combine predictors using stacking

Time-related feature engineering

Time-related feature engineering

Quantile regression

Quantile regression

Common pitfalls in the interpretation of coefficients of linear models

Common pitfalls in the interpretation of coefficients of linear models

Class Likelihood Ratios to measure classification performance

Class Likelihood Ratios to measure classification performance

Plotting Cross-Validated Predictions

Plotting Cross-Validated Predictions

Comparing Target Encoder with Other Encoders

Comparing Target Encoder with Other Encoders