
.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/release_highlights/plot_release_highlights_0_23_0.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_release_highlights_plot_release_highlights_0_23_0.py>`
        to download the full example code or to run this example in your browser via Binder

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_release_highlights_plot_release_highlights_0_23_0.py:


========================================
Release Highlights for scikit-learn 0.23
========================================

.. currentmodule:: sklearn

We are pleased to announce the release of scikit-learn 0.23! Many bug fixes
and improvements were added, as well as some new key features. We detail
below a few of the major features of this release. **For an exhaustive list of
all the changes**, please refer to the :ref:`release notes <changes_0_23>`.

To install the latest version (with pip)::

    pip install --upgrade scikit-learn

or with conda::

    conda install -c conda-forge scikit-learn

.. GENERATED FROM PYTHON SOURCE LINES 25-36

Generalized Linear Models, and Poisson loss for gradient boosting
-----------------------------------------------------------------
Long-awaited Generalized Linear Models with non-normal loss functions are now
available. In particular, three new regressors were implemented:
:class:`~sklearn.linear_model.PoissonRegressor`,
:class:`~sklearn.linear_model.GammaRegressor`, and
:class:`~sklearn.linear_model.TweedieRegressor`. The Poisson regressor can be
used to model positive integer counts, or relative frequencies. Read more in
the :ref:`User Guide <Generalized_linear_regression>`. Additionally,
:class:`~sklearn.ensemble.HistGradientBoostingRegressor` supports a new
'poisson' loss as well.

.. GENERATED FROM PYTHON SOURCE LINES 36-55

.. code-block:: default


    import numpy as np
    from sklearn.model_selection import train_test_split
    from sklearn.linear_model import PoissonRegressor
    from sklearn.ensemble import HistGradientBoostingRegressor

    n_samples, n_features = 1000, 20
    rng = np.random.RandomState(0)
    X = rng.randn(n_samples, n_features)
    # positive integer target correlated with X[:, 5] with many zeros:
    y = rng.poisson(lam=np.exp(X[:, 5]) / 2)
    X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=rng)
    glm = PoissonRegressor()
    gbdt = HistGradientBoostingRegressor(loss="poisson", learning_rate=0.01)
    glm.fit(X_train, y_train)
    gbdt.fit(X_train, y_train)
    print(glm.score(X_test, y_test))
    print(gbdt.score(X_test, y_test))





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    0.35776189065725794
    0.42425183539869415




.. GENERATED FROM PYTHON SOURCE LINES 56-64

Rich visual representation of estimators
-----------------------------------------
Estimators can now be visualized in notebooks by enabling the
`display='diagram'` option. This is particularly useful to summarise the
structure of pipelines and other composite estimators, with interactivity to
provide detail.  Click on the example image below to expand Pipeline
elements.  See :ref:`visualizing_composite_estimators` for how you can use
this feature.

.. GENERATED FROM PYTHON SOURCE LINES 64-88

.. code-block:: default


    from sklearn import set_config
    from sklearn.pipeline import make_pipeline
    from sklearn.preprocessing import OneHotEncoder, StandardScaler
    from sklearn.impute import SimpleImputer
    from sklearn.compose import make_column_transformer
    from sklearn.linear_model import LogisticRegression

    set_config(display="diagram")

    num_proc = make_pipeline(SimpleImputer(strategy="median"), StandardScaler())

    cat_proc = make_pipeline(
        SimpleImputer(strategy="constant", fill_value="missing"),
        OneHotEncoder(handle_unknown="ignore"),
    )

    preprocessor = make_column_transformer(
        (num_proc, ("feat1", "feat3")), (cat_proc, ("feat0", "feat2"))
    )

    clf = make_pipeline(preprocessor, LogisticRegression())
    clf






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <style>#sk-container-id-5 {color: black;}#sk-container-id-5 pre{padding: 0;}#sk-container-id-5 div.sk-toggleable {background-color: white;}#sk-container-id-5 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-5 label.sk-toggleable__label-arrow:before {content: "▸";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-5 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-5 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-5 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-5 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-5 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-5 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: "▾";}#sk-container-id-5 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-5 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-5 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-5 div.sk-parallel-item::after {content: "";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-5 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 div.sk-serial::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-5 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-5 div.sk-item {position: relative;z-index: 1;}#sk-container-id-5 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-5 div.sk-item::before, #sk-container-id-5 div.sk-parallel-item::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-5 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-5 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-5 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-5 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-5 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-5 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-5 div.sk-label-container {text-align: center;}#sk-container-id-5 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-5 div.sk-text-repr-fallback {display: none;}</style><div id="sk-container-id-5" class="sk-top-container"><div class="sk-text-repr-fallback"><pre>Pipeline(steps=[(&#x27;columntransformer&#x27;,
                     ColumnTransformer(transformers=[(&#x27;pipeline-1&#x27;,
                                                      Pipeline(steps=[(&#x27;simpleimputer&#x27;,
                                                                       SimpleImputer(strategy=&#x27;median&#x27;)),
                                                                      (&#x27;standardscaler&#x27;,
                                                                       StandardScaler())]),
                                                      (&#x27;feat1&#x27;, &#x27;feat3&#x27;)),
                                                     (&#x27;pipeline-2&#x27;,
                                                      Pipeline(steps=[(&#x27;simpleimputer&#x27;,
                                                                       SimpleImputer(fill_value=&#x27;missing&#x27;,
                                                                                     strategy=&#x27;constant&#x27;)),
                                                                      (&#x27;onehotencoder&#x27;,
                                                                       OneHotEncoder(handle_unknown=&#x27;ignore&#x27;))]),
                                                      (&#x27;feat0&#x27;, &#x27;feat2&#x27;))])),
                    (&#x27;logisticregression&#x27;, LogisticRegression())])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class="sk-container" hidden><div class="sk-item sk-dashed-wrapped"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-15" type="checkbox" ><label for="sk-estimator-id-15" class="sk-toggleable__label sk-toggleable__label-arrow">Pipeline</label><div class="sk-toggleable__content"><pre>Pipeline(steps=[(&#x27;columntransformer&#x27;,
                     ColumnTransformer(transformers=[(&#x27;pipeline-1&#x27;,
                                                      Pipeline(steps=[(&#x27;simpleimputer&#x27;,
                                                                       SimpleImputer(strategy=&#x27;median&#x27;)),
                                                                      (&#x27;standardscaler&#x27;,
                                                                       StandardScaler())]),
                                                      (&#x27;feat1&#x27;, &#x27;feat3&#x27;)),
                                                     (&#x27;pipeline-2&#x27;,
                                                      Pipeline(steps=[(&#x27;simpleimputer&#x27;,
                                                                       SimpleImputer(fill_value=&#x27;missing&#x27;,
                                                                                     strategy=&#x27;constant&#x27;)),
                                                                      (&#x27;onehotencoder&#x27;,
                                                                       OneHotEncoder(handle_unknown=&#x27;ignore&#x27;))]),
                                                      (&#x27;feat0&#x27;, &#x27;feat2&#x27;))])),
                    (&#x27;logisticregression&#x27;, LogisticRegression())])</pre></div></div></div><div class="sk-serial"><div class="sk-item sk-dashed-wrapped"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-16" type="checkbox" ><label for="sk-estimator-id-16" class="sk-toggleable__label sk-toggleable__label-arrow">columntransformer: ColumnTransformer</label><div class="sk-toggleable__content"><pre>ColumnTransformer(transformers=[(&#x27;pipeline-1&#x27;,
                                     Pipeline(steps=[(&#x27;simpleimputer&#x27;,
                                                      SimpleImputer(strategy=&#x27;median&#x27;)),
                                                     (&#x27;standardscaler&#x27;,
                                                      StandardScaler())]),
                                     (&#x27;feat1&#x27;, &#x27;feat3&#x27;)),
                                    (&#x27;pipeline-2&#x27;,
                                     Pipeline(steps=[(&#x27;simpleimputer&#x27;,
                                                      SimpleImputer(fill_value=&#x27;missing&#x27;,
                                                                    strategy=&#x27;constant&#x27;)),
                                                     (&#x27;onehotencoder&#x27;,
                                                      OneHotEncoder(handle_unknown=&#x27;ignore&#x27;))]),
                                     (&#x27;feat0&#x27;, &#x27;feat2&#x27;))])</pre></div></div></div><div class="sk-parallel"><div class="sk-parallel-item"><div class="sk-item"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-17" type="checkbox" ><label for="sk-estimator-id-17" class="sk-toggleable__label sk-toggleable__label-arrow">pipeline-1</label><div class="sk-toggleable__content"><pre>(&#x27;feat1&#x27;, &#x27;feat3&#x27;)</pre></div></div></div><div class="sk-serial"><div class="sk-item"><div class="sk-serial"><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-18" type="checkbox" ><label for="sk-estimator-id-18" class="sk-toggleable__label sk-toggleable__label-arrow">SimpleImputer</label><div class="sk-toggleable__content"><pre>SimpleImputer(strategy=&#x27;median&#x27;)</pre></div></div></div><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-19" type="checkbox" ><label for="sk-estimator-id-19" class="sk-toggleable__label sk-toggleable__label-arrow">StandardScaler</label><div class="sk-toggleable__content"><pre>StandardScaler()</pre></div></div></div></div></div></div></div></div><div class="sk-parallel-item"><div class="sk-item"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-20" type="checkbox" ><label for="sk-estimator-id-20" class="sk-toggleable__label sk-toggleable__label-arrow">pipeline-2</label><div class="sk-toggleable__content"><pre>(&#x27;feat0&#x27;, &#x27;feat2&#x27;)</pre></div></div></div><div class="sk-serial"><div class="sk-item"><div class="sk-serial"><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-21" type="checkbox" ><label for="sk-estimator-id-21" class="sk-toggleable__label sk-toggleable__label-arrow">SimpleImputer</label><div class="sk-toggleable__content"><pre>SimpleImputer(fill_value=&#x27;missing&#x27;, strategy=&#x27;constant&#x27;)</pre></div></div></div><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-22" type="checkbox" ><label for="sk-estimator-id-22" class="sk-toggleable__label sk-toggleable__label-arrow">OneHotEncoder</label><div class="sk-toggleable__content"><pre>OneHotEncoder(handle_unknown=&#x27;ignore&#x27;)</pre></div></div></div></div></div></div></div></div></div></div><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-23" type="checkbox" ><label for="sk-estimator-id-23" class="sk-toggleable__label sk-toggleable__label-arrow">LogisticRegression</label><div class="sk-toggleable__content"><pre>LogisticRegression()</pre></div></div></div></div></div></div></div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 89-97

Scalability and stability improvements to KMeans
------------------------------------------------
The :class:`~sklearn.cluster.KMeans` estimator was entirely re-worked, and it
is now significantly faster and more stable. In addition, the Elkan algorithm
is now compatible with sparse matrices. The estimator uses OpenMP based
parallelism instead of relying on joblib, so the `n_jobs` parameter has no
effect anymore. For more details on how to control the number of threads,
please refer to our :ref:`parallelism` notes.

.. GENERATED FROM PYTHON SOURCE LINES 97-111

.. code-block:: default

    import scipy
    import numpy as np
    from sklearn.model_selection import train_test_split
    from sklearn.cluster import KMeans
    from sklearn.datasets import make_blobs
    from sklearn.metrics import completeness_score

    rng = np.random.RandomState(0)
    X, y = make_blobs(random_state=rng)
    X = scipy.sparse.csr_matrix(X)
    X_train, X_test, _, y_test = train_test_split(X, y, random_state=rng)
    kmeans = KMeans(n_init="auto").fit(X_train)
    print(completeness_score(kmeans.predict(X_test), y_test))





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    0.6560362663398502




.. GENERATED FROM PYTHON SOURCE LINES 112-126

Improvements to the histogram-based Gradient Boosting estimators
----------------------------------------------------------------
Various improvements were made to
:class:`~sklearn.ensemble.HistGradientBoostingClassifier` and
:class:`~sklearn.ensemble.HistGradientBoostingRegressor`. On top of the
Poisson loss mentioned above, these estimators now support :ref:`sample
weights <sw_hgbdt>`. Also, an automatic early-stopping criterion was added:
early-stopping is enabled by default when the number of samples exceeds 10k.
Finally, users can now define :ref:`monotonic constraints
<monotonic_cst_gbdt>` to constrain the predictions based on the variations of
specific features. In the following example, we construct a target that is
generally positively correlated with the first feature, with some noise.
Applying monotoinc constraints allows the prediction to capture the global
effect of the first feature, instead of fitting the noise.

.. GENERATED FROM PYTHON SOURCE LINES 126-169

.. code-block:: default

    import numpy as np
    from matplotlib import pyplot as plt
    from sklearn.model_selection import train_test_split

    # from sklearn.inspection import plot_partial_dependence
    from sklearn.inspection import PartialDependenceDisplay
    from sklearn.ensemble import HistGradientBoostingRegressor

    n_samples = 500
    rng = np.random.RandomState(0)
    X = rng.randn(n_samples, 2)
    noise = rng.normal(loc=0.0, scale=0.01, size=n_samples)
    y = 5 * X[:, 0] + np.sin(10 * np.pi * X[:, 0]) - noise

    gbdt_no_cst = HistGradientBoostingRegressor().fit(X, y)
    gbdt_cst = HistGradientBoostingRegressor(monotonic_cst=[1, 0]).fit(X, y)

    # plot_partial_dependence has been removed in version 1.2. From 1.2, use
    # PartialDependenceDisplay instead.
    # disp = plot_partial_dependence(
    disp = PartialDependenceDisplay.from_estimator(
        gbdt_no_cst,
        X,
        features=[0],
        feature_names=["feature 0"],
        line_kw={"linewidth": 4, "label": "unconstrained", "color": "tab:blue"},
    )
    # plot_partial_dependence(
    PartialDependenceDisplay.from_estimator(
        gbdt_cst,
        X,
        features=[0],
        line_kw={"linewidth": 4, "label": "constrained", "color": "tab:orange"},
        ax=disp.axes_,
    )
    disp.axes_[0, 0].plot(
        X[:, 0], y, "o", alpha=0.5, zorder=-1, label="samples", color="tab:green"
    )
    disp.axes_[0, 0].set_ylim(-3, 3)
    disp.axes_[0, 0].set_xlim(-1, 1)
    plt.legend()
    plt.show()




.. image-sg:: /auto_examples/release_highlights/images/sphx_glr_plot_release_highlights_0_23_0_001.png
   :alt: plot release highlights 0 23 0
   :srcset: /auto_examples/release_highlights/images/sphx_glr_plot_release_highlights_0_23_0_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 170-174

Sample-weight support for Lasso and ElasticNet
----------------------------------------------
The two linear regressors :class:`~sklearn.linear_model.Lasso` and
:class:`~sklearn.linear_model.ElasticNet` now support sample weights.

.. GENERATED FROM PYTHON SOURCE LINES 174-190

.. code-block:: default


    from sklearn.model_selection import train_test_split
    from sklearn.datasets import make_regression
    from sklearn.linear_model import Lasso
    import numpy as np

    n_samples, n_features = 1000, 20
    rng = np.random.RandomState(0)
    X, y = make_regression(n_samples, n_features, random_state=rng)
    sample_weight = rng.rand(n_samples)
    X_train, X_test, y_train, y_test, sw_train, sw_test = train_test_split(
        X, y, sample_weight, random_state=rng
    )
    reg = Lasso()
    reg.fit(X_train, y_train, sample_weight=sw_train)
    print(reg.score(X_test, y_test, sw_test))




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    0.999791942438998





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.706 seconds)


.. _sphx_glr_download_auto_examples_release_highlights_plot_release_highlights_0_23_0.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: binder-badge

      .. image:: images/binder_badge_logo.svg
        :target: https://mybinder.org/v2/gh/scikit-learn/scikit-learn/1.3.X?urlpath=lab/tree/notebooks/auto_examples/release_highlights/plot_release_highlights_0_23_0.ipynb
        :alt: Launch binder
        :width: 150 px



    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_release_highlights_0_23_0.py <plot_release_highlights_0_23_0.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_release_highlights_0_23_0.ipynb <plot_release_highlights_0_23_0.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
