.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "generated/gallery/clipping.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_generated_gallery_clipping.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_generated_gallery_clipping.py:


Clipping Detection
===================

Identifying clipping periods using the PVAnalytics clipping module.

.. GENERATED FROM PYTHON SOURCE LINES 9-20

Identifying and removing clipping periods from AC power time series
data aids in generating more accurate degradation analysis results,
as using clipped data can lead to under-predicting degradation. In this
example, we show how to use
:py:func:`pvanalytics.features.clipping.geometric`
to mask clipping periods in an AC power time series. We use a
normalized time series example provided by the PV Fleets Initiative,
where clipping periods are labeled as True, and non-clipping periods are
labeled as False. This example is adapted from the DuraMAT DataHub
clipping data set:
https://datahub.duramat.org/dataset/inverter-clipping-ml-training-set-real-data

.. GENERATED FROM PYTHON SOURCE LINES 20-28

.. code-block:: default


    import pvanalytics
    from pvanalytics.features.clipping import geometric
    import matplotlib.pyplot as plt
    import pandas as pd
    import pathlib
    import numpy as np








.. GENERATED FROM PYTHON SOURCE LINES 29-31

First, read in the ac_power_inv_7539 example, and visualize a subset of the
clipping periods via the "label" mask column.

.. GENERATED FROM PYTHON SOURCE LINES 31-50

.. code-block:: default


    pvanalytics_dir = pathlib.Path(pvanalytics.__file__).parent
    ac_power_file_1 = pvanalytics_dir / 'data' / 'ac_power_inv_7539.csv'
    data = pd.read_csv(ac_power_file_1, index_col=0, parse_dates=True)
    data['label'] = data['label'].astype(bool)
    # This is the known frequency of the time series. You may need to infer
    # the frequency or set the frequency with your AC power time series.
    freq = "15T"

    data['value_normalized'].plot()
    data.loc[data['label'], 'value_normalized'].plot(ls='', marker='o')
    plt.legend(labels=["AC Power", "Labeled Clipping"],
               title="Clipped")
    plt.xticks(rotation=20)
    plt.xlabel("Date")
    plt.ylabel("Normalized AC Power")
    plt.tight_layout()
    plt.show()




.. image-sg:: /generated/gallery/images/sphx_glr_clipping_001.png
   :alt: clipping
   :srcset: /generated/gallery/images/sphx_glr_clipping_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 51-53

Now, use :py:func:`pvanalytics.features.clipping.geometric` to identify
clipping periods in the time series. Re-plot the data subset with this mask.

.. GENERATED FROM PYTHON SOURCE LINES 53-66

.. code-block:: default

    predicted_clipping_mask = geometric(ac_power=data['value_normalized'],
                                        freq=freq)
    data['value_normalized'].plot()
    data.loc[predicted_clipping_mask, 'value_normalized'].plot(ls='', marker='o')
    plt.legend(labels=["AC Power", "Detected Clipping"],
               title="Clipped")
    plt.xticks(rotation=20)
    plt.xlabel("Date")
    plt.ylabel("Normalized AC Power")
    plt.tight_layout()
    plt.show()





.. image-sg:: /generated/gallery/images/sphx_glr_clipping_002.png
   :alt: clipping
   :srcset: /generated/gallery/images/sphx_glr_clipping_002.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 67-69

Compare the filter results to the ground-truth labeled data side-by-side,
and generate an accuracy metric.

.. GENERATED FROM PYTHON SOURCE LINES 69-72

.. code-block:: default

    acc = 100 * np.sum(np.equal(data.label,
                                predicted_clipping_mask))/len(data.label)
    print("Overall model prediction accuracy: " + str(round(acc, 2)) + "%")




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Overall model prediction accuracy: 99.2%





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  0.436 seconds)


.. _sphx_glr_download_generated_gallery_clipping.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: clipping.py <clipping.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: clipping.ipynb <clipping.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_