.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "generated/gallery/gaps/interpolated-periods.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_generated_gallery_gaps_interpolated-periods.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_generated_gallery_gaps_interpolated-periods.py:


Interpolated Data Periods
=========================

Identifying periods in a time series where the data has been
linearly interpolated.

.. GENERATED FROM PYTHON SOURCE LINES 10-15

Identifying periods where time series data has been linearly interpolated
and removing these periods may help to reduce noise when performing future
data analysis. This example shows how to use
:py:func:`pvanalytics.quality.gaps.interpolation_diff`, which identifies and
masks linearly interpolated periods.

.. GENERATED FROM PYTHON SOURCE LINES 15-22

.. code-block:: default


    import pvanalytics
    from pvanalytics.quality import gaps
    import matplotlib.pyplot as plt
    import pandas as pd
    import pathlib








.. GENERATED FROM PYTHON SOURCE LINES 23-33

First, we import the AC power data stream that we are going to check for
interpolated periods. The time series we download is a normalized AC power
time series from the PV Fleets Initiative, and is available via the DuraMAT
DataHub:
https://datahub.duramat.org/dataset/inverter-clipping-ml-training-set-real-data.
This data set has a Pandas DateTime index, with the min-max normalized
AC power time series represented in the 'value_normalized' column. There is
also an "interpolated_data_mask" column, where
interpolated periods are labeled as True, and all other data is labeled
as False. The data is sampled at 15-minute intervals.

.. GENERATED FROM PYTHON SOURCE LINES 33-47

.. code-block:: default


    pvanalytics_dir = pathlib.Path(pvanalytics.__file__).parent
    file = pvanalytics_dir / 'data' / 'ac_power_inv_2173_interpolated_data.csv'
    data = pd.read_csv(file, index_col=0, parse_dates=True)
    data = data.asfreq("15T")
    data['value_normalized'].plot()
    data.loc[data["interpolated_data_mask"], "value_normalized"].plot(ls='',
                                                                      marker='.')
    plt.legend(labels=["AC Power", "Interpolated Data"])
    plt.xlabel("Date")
    plt.ylabel("Normalized AC Power")
    plt.tight_layout()
    plt.show()




.. image-sg:: /generated/gallery/gaps/images/sphx_glr_interpolated-periods_001.png
   :alt: interpolated periods
   :srcset: /generated/gallery/gaps/images/sphx_glr_interpolated-periods_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 48-54

Now, we use :py:func:`pvanalytics.quality.gaps.interpolation_diff` to
identify linearly interpolated periods in the time series. We re-plot
the data with this mask. Please note that nighttime periods generally consist
of repeating 0 values; this means that these periods can be linearly
interpolated. Consequently, these periods are flagged by
:py:func:`pvanalytics.quality.gaps.interpolation_diff`.

.. GENERATED FROM PYTHON SOURCE LINES 54-64

.. code-block:: default

    detected_interpolated_data_mask = gaps.interpolation_diff(
        data['value_normalized'])
    data['value_normalized'].plot()
    data.loc[detected_interpolated_data_mask,
             "value_normalized"].plot(ls='', marker='.')
    plt.legend(labels=["AC Power", "Detected Interpolated Data"])
    plt.xlabel("Date")
    plt.ylabel("Normalized AC Power")
    plt.tight_layout()
    plt.show()



.. image-sg:: /generated/gallery/gaps/images/sphx_glr_interpolated-periods_002.png
   :alt: interpolated periods
   :srcset: /generated/gallery/gaps/images/sphx_glr_interpolated-periods_002.png
   :class: sphx-glr-single-img






.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  0.797 seconds)


.. _sphx_glr_download_generated_gallery_gaps_interpolated-periods.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: interpolated-periods.py <interpolated-periods.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: interpolated-periods.ipynb <interpolated-periods.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_