.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "generated/gallery/data-shifts.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_generated_gallery_data-shifts.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_generated_gallery_data-shifts.py:


Data Shift Detection & Filtering
================================

Identifying data shifts/capacity changes in time series data

.. GENERATED FROM PYTHON SOURCE LINES 9-13

This example covers identifying data shifts/capacity changes in a time series
and extracting the longest time series segment free of these shifts, using
:py:func:`pvanalytics.quality.data_shifts.detect_data_shifts` and
:py:func:`pvanalytics.quality.data_shifts.get_longest_shift_segment_dates`.

.. GENERATED FROM PYTHON SOURCE LINES 13-20

.. code-block:: default


    import pvanalytics
    import pandas as pd
    import matplotlib.pyplot as plt
    from pvanalytics.quality import data_shifts as ds
    import pathlib








.. GENERATED FROM PYTHON SOURCE LINES 21-23

As an example, we load in a simulated pvlib AC power time series with a
single changepoint, occurring on October 28, 2015.

.. GENERATED FROM PYTHON SOURCE LINES 23-31

.. code-block:: default


    pvanalytics_dir = pathlib.Path(pvanalytics.__file__).parent
    data_shift_file = pvanalytics_dir / 'data' / 'pvlib_data_shift.csv'
    df = pd.read_csv(data_shift_file)
    df.index = pd.to_datetime(df['timestamp'])
    df['value'].plot()
    print("Changepoint at: " + str(df[df['label'] == 1].index[0]))




.. image-sg:: /generated/gallery/images/sphx_glr_data-shifts_001.png
   :alt: data shifts
   :srcset: /generated/gallery/images/sphx_glr_data-shifts_001.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Changepoint at: 2015-10-28 00:00:00




.. GENERATED FROM PYTHON SOURCE LINES 32-36

Now we run the data shift algorithm (with default parameters)
on the data stream, using
:py:func:`pvanalytics.quality.data_shifts.detect_data_shifts`. We plot the
predicted time series segments, based on algorithm results.

.. GENERATED FROM PYTHON SOURCE LINES 36-56

.. code-block:: default


    shift_mask = ds.detect_data_shifts(df['value'])
    shift_list = list(df[shift_mask].index)
    edges = [df.index[0]] + shift_list + [df.index[-1]]
    fig, ax = plt.subplots()
    for (st, ed) in zip(edges[:-1], edges[1:]):
        ax.plot(df.loc[st:ed, "value"])
    plt.show()

    # We zoom in around the changepoint to more closely show the data shift. Time
    # series segments pre- and post-shift are color-coded.

    edges = [pd.to_datetime("10-15-2015")] + shift_list + \
        [pd.to_datetime("11-15-2015")]
    fig, ax = plt.subplots()
    for (st, ed) in zip(edges[:-1], edges[1:]):
        ax.plot(df.loc[st:ed, "value"])
    plt.xticks(rotation=45)
    plt.show()




.. rst-class:: sphx-glr-horizontal


    *

      .. image-sg:: /generated/gallery/images/sphx_glr_data-shifts_002.png
         :alt: data shifts
         :srcset: /generated/gallery/images/sphx_glr_data-shifts_002.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /generated/gallery/images/sphx_glr_data-shifts_003.png
         :alt: data shifts
         :srcset: /generated/gallery/images/sphx_glr_data-shifts_003.png
         :class: sphx-glr-multi-img





.. GENERATED FROM PYTHON SOURCE LINES 57-61

We filter the time series by the detected changepoints, taking the longest
continuous segment free of data shifts, using
:py:func:`pvanalytics.quality.data_shifts.get_longest_shift_segment_dates`.
The trimmed time series is then plotted.

.. GENERATED FROM PYTHON SOURCE LINES 61-65

.. code-block:: default


    start_date, end_date = ds.get_longest_shift_segment_dates(df['value'])
    df['value'][start_date:end_date].plot()
    plt.show()



.. image-sg:: /generated/gallery/images/sphx_glr_data-shifts_004.png
   :alt: data shifts
   :srcset: /generated/gallery/images/sphx_glr_data-shifts_004.png
   :class: sphx-glr-single-img






.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  2.322 seconds)


.. _sphx_glr_download_generated_gallery_data-shifts.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: data-shifts.py <data-shifts.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: data-shifts.ipynb <data-shifts.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_