.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "generated/gallery/day-night-masking/day-night-masking.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_generated_gallery_day-night-masking_day-night-masking.py: Day-Night Masking ================= Masking day-night periods using the PVAnalytics daytime module. .. GENERATED FROM PYTHON SOURCE LINES 9-17 Identifying and masking day-night periods in an AC power time series or irradiance time series can aid in future data analysis, such as detecting if a time series has daylight savings time or time shifts. Here, we use :py:func:`pvanalytics.features.daytime.power_or_irradiance` to mask day/night periods, as well as to estimate sunrise and sunset times in the data set. This function is particularly useful for cases where the time zone of a data stream is unknown or incorrect, as its outputs can be used to determine time zone. .. GENERATED FROM PYTHON SOURCE LINES 17-26 .. code-block:: default import pvanalytics from pvanalytics.features.daytime import power_or_irradiance import matplotlib.pyplot as plt import pandas as pd import pathlib import pvlib import numpy as np .. GENERATED FROM PYTHON SOURCE LINES 27-31 First, read in the 1-minute sampled AC power time series data, taken from the SERF East installation on the NREL campus. This sample is provided from the NREL PVDAQ database, and contains a column representing an AC power data stream. .. GENERATED FROM PYTHON SOURCE LINES 31-51 .. code-block:: default pvanalytics_dir = pathlib.Path(pvanalytics.__file__).parent ac_power_file = pvanalytics_dir / 'data' / 'serf_east_1min_ac_power.csv' data = pd.read_csv(ac_power_file, index_col=0, parse_dates=True) data = data.sort_index() # This is the known frequency of the time series. You may need to infer # the frequency or set the frequency with your AC power time series. freq = "1T" # These are the latitude-longitude coordinates associated with the # SERF East system. latitude = 39.742 longitude = -105.173 # Plot the time series. data['ac_power__752'].plot() plt.xlabel("Date") plt.ylabel("AC Power (kW)") plt.tight_layout() plt.show() .. image-sg:: /generated/gallery/day-night-masking/images/sphx_glr_day-night-masking_001.png :alt: day night masking :srcset: /generated/gallery/day-night-masking/images/sphx_glr_day-night-masking_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 52-55 It is critical to set all negative values in the AC power time series to 0 for :py:func:`pvanalytics.features.daytime.power_or_irradiance` to work properly. Negative erroneous data may affect daytime mask assignments. .. GENERATED FROM PYTHON SOURCE LINES 55-57 .. code-block:: default data.loc[data['ac_power__752'] < 0, 'ac_power__752'] = 0 .. GENERATED FROM PYTHON SOURCE LINES 58-60 Now, use :py:func:`pvanalytics.features.daytime.power_or_irradiance` to mask day periods in the time series. .. GENERATED FROM PYTHON SOURCE LINES 60-63 .. code-block:: default predicted_day_night_mask = power_or_irradiance(series=data['ac_power__752'], freq=freq) .. GENERATED FROM PYTHON SOURCE LINES 64-73 Function :py:func:`pvlib.solarposition.sun_rise_set_transit_spa` is used to get ground-truth sunrise and sunset times for each day at the site location, and a SPA-daytime mask is calculated based on these times. Data associated with SPA daytime periods is labeled as True, and data associated with SPA nighttime periods is labeled as False. SPA sunrise and sunset times are used here as a point of comparison to the :py:func:`pvanalytics.features.daytime.power_or_irradiance` outputs. SPA-based sunrise and sunset values are not needed to run :py:func:`pvanalytics.features.daytime.power_or_irradiance`. .. GENERATED FROM PYTHON SOURCE LINES 73-85 .. code-block:: default sunrise_sunset_df = pvlib.solarposition.sun_rise_set_transit_spa(data.index, latitude, longitude) data['sunrise_time'] = sunrise_sunset_df['sunrise'] data['sunset_time'] = sunrise_sunset_df['sunset'] data['daytime_mask'] = True data.loc[(data.index < data.sunrise_time) | (data.index > data.sunset_time), "daytime_mask"] = False .. GENERATED FROM PYTHON SOURCE LINES 86-89 Plot the AC power data stream with the mask output from :py:func:`pvanalytics.features.daytime.power_or_irradiance`, as well as the SPA-calculated sunrise and sunset .. GENERATED FROM PYTHON SOURCE LINES 89-105 .. code-block:: default data['ac_power__752'].plot() data.loc[predicted_day_night_mask, 'ac_power__752'].plot(ls='', marker='o') data.loc[~predicted_day_night_mask, 'ac_power__752'].plot(ls='', marker='o') sunrise_sunset_times = sunrise_sunset_df[['sunrise', 'sunset']].drop_duplicates() for sunrise, sunset in sunrise_sunset_times.itertuples(index=False): plt.axvline(x=sunrise, c="blue") plt.axvline(x=sunset, c="red") plt.legend(labels=["AC Power", "Daytime", "Nighttime", "SPA Sunrise", "SPA Sunset"]) plt.xlabel("Date") plt.ylabel("AC Power (kW)") plt.tight_layout() plt.show() .. image-sg:: /generated/gallery/day-night-masking/images/sphx_glr_day-night-masking_002.png :alt: day night masking :srcset: /generated/gallery/day-night-masking/images/sphx_glr_day-night-masking_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 106-109 Compare the predicted mask to the ground-truth SPA mask, to get the model accuracy. Also, compare sunrise and sunset times for the predicted mask compared to the ground truth sunrise and sunset times. .. GENERATED FROM PYTHON SOURCE LINES 109-127 .. code-block:: default acc = 100 * np.sum(np.equal(data.daytime_mask, predicted_day_night_mask))/len(data.daytime_mask) print("Overall model prediction accuracy: " + str(round(acc, 2)) + "%") # Generate predicted + SPA sunrise times for each day print("Sunrise Comparison:") print(pd.DataFrame({'predicted_sunrise': predicted_day_night_mask .index[predicted_day_night_mask] .to_series().resample("d").first(), 'pvlib_spa_sunrise': sunrise_sunset_df["sunrise"] .resample("d").first()})) # Generate predicted + SPA sunset times for each day print("Sunset Comparison:") print(pd.DataFrame({'predicted_sunset': predicted_day_night_mask .index[predicted_day_night_mask] .to_series().resample("d").last(), 'pvlib_spa_sunset': sunrise_sunset_df["sunrise"] .resample("d").last()})) .. rst-class:: sphx-glr-script-out .. code-block:: none Overall model prediction accuracy: 98.39% Sunrise Comparison: predicted_sunrise pvlib_spa_sunrise measured_on 2022-03-18 00:00:00-07:00 2022-03-18 06:11:00-07:00 2022-03-18 06:07:09.226592-07:00 2022-03-19 00:00:00-07:00 2022-03-19 06:14:00-07:00 2022-03-19 06:05:32.867153920-07:00 Sunset Comparison: predicted_sunset pvlib_spa_sunset measured_on 2022-03-18 00:00:00-07:00 2022-03-18 17:56:00-07:00 2022-03-18 06:07:09.226592-07:00 2022-03-19 00:00:00-07:00 2022-03-19 17:52:00-07:00 2022-03-19 06:05:32.867153920-07:00 .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 1.195 seconds) .. _sphx_glr_download_generated_gallery_day-night-masking_day-night-masking.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: day-night-masking.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: day-night-masking.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_