Opsd renewable ppls (#212)

* fix clustering of offwind-ac and offwind-dc in sites option

* add release nodes

* attach renewable assets by location (lat and lon) from OPSD register to network

* adapt default config to changes

* undo changes from a differen PR in cluster_network.py

* undo changes from a different PR, add release notes for this PR

* correct release notes

* add comments for relevant settings in add_electricity.py

* adjust configtable for electricity to OPSD renewable option and add estimate_renewable_capacities_from_capacitiy_stats

* reset cluster_network to HEAD

* add_electricity: Capacity is float

* config: add GB to OPSD_VRE_countries

* review and modify implementation

* update release notes

* Update envs/environment.yaml

Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>

Co-authored-by: martha.frysztacki <eb5194@iai-esm003.iai.kit.edu>
Co-authored-by: eb5194 <martha.frysztacki@kit.edu>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>

config.default.yaml

@@ -53,6 +53,8 @@ electricity:
   custom_powerplants: false # use pandas query strings here, e.g. Country in ['Germany']
   conventional_carriers: [nuclear, oil, OCGT, CCGT, coal, lignite, geothermal, biomass]
 
+  renewable_capacities_from_OPSD: [] # onwind, offwind, solar
+
   # estimate_renewable_capacities_from_capacity_stats:
   #   # Wind is the Fueltype in ppm.data.Capacity_stats, onwind, offwind-{ac,dc} the carrier in PyPSA-Eur
   #   Wind: [onwind, offwind-ac, offwind-dc]

doc/configtables/electricity.csv

@@ -1,16 +1,20 @@
-,Unit,Values,Description
-voltages,kV,"Any subset of {220., 300., 380.}","Voltage levels to consider when"
-co2limit,:math:`t_{CO_2-eq}/a`,float,"Cap on total annual system carbon dioxide emissions"
-co2base,:math:`t_{CO_2-eq}/a`,float,"Reference value of total annual system carbon dioxide emissions if relative emission reduction target is specified in ``{opts}`` wildcard."
-agg_p_nom_limits,--,file path,"Reference to ``.csv`` file specifying per carrier generator nominal capacity constraints for individual countries if ``'CCL'`` is in ``{opts}`` wildcard. Defaults to ``data/agg_p_nom_minmax.csv``."
-extendable_carriers,,,
--- Generator,--,"Any subset  of {'OCGT','CCGT'}","Places extendable conventional power plants (OCGT and/or CCGT) where gas power plants are located today without capacity limits."
--- StorageUnit,--,"Any subset of {'battery','H2'}","Adds extendable storage units (battery and/or hydrogen) at every node/bus after clustering without capacity limits and with zero initial capacity."
--- Store,--,"Any subset of {'battery','H2'}","Adds extendable storage units (battery and/or hydrogen) at every node/bus after clustering without capacity limits and with zero initial capacity."
--- Link,--,"Any subset of {'H2 pipeline'}","Adds extendable links (H2 pipelines only) at every connection where there are lines or HVDC links without capacity limits and with zero initial capacity. Hydrogen pipelines require hydrogen storage to be modelled as ``Store``."
-max_hours,,,
--- battery,h,float,"Maximum state of charge capacity of the battery in terms of hours at full output capacity ``p_nom``. Cf. `PyPSA documentation <https://pypsa.readthedocs.io/en/latest/components.html#storage-unit>`_."
--- H2,h,float,"Maximum state of charge capacity of the hydrogen storage in terms of hours at full output capacity ``p_nom``. Cf. `PyPSA documentation <https://pypsa.readthedocs.io/en/latest/components.html#storage-unit>`_."
-powerplants_filter,--,"use `pandas.query <https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.query.html>`_ strings here, e.g. Country not in ['Germany']","Filter query for the default powerplant database."
-custom_powerplants,--,"use `pandas.query <https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.query.html>`_ strings here, e.g. Country in ['Germany']","Filter query for the custom powerplant database."
-conventional_carriers,--,"Any subset of {nuclear, oil, OCGT, CCGT, coal, lignite, geothermal, biomass}","List of conventional power plants to include in the model from ``resources/powerplants.csv``."
+,Unit,Values,Description,
+voltages,kV,"Any subset of {220., 300., 380.}",Voltage levels to consider when,
+co2limit,:math:`t_{CO_2-eq}/a`,float,Cap on total annual system carbon dioxide emissions,
+co2base,:math:`t_{CO_2-eq}/a`,float,Reference value of total annual system carbon dioxide emissions if relative emission reduction target is specified in ``{opts}`` wildcard.,
+agg_p_nom_limits,--,file,path,Reference to ``.csv`` file specifying per carrier generator nominal capacity constraints for individual countries if ``'CCL'`` is in ``{opts}`` wildcard. Defaults to ``data/agg_p_nom_minmax.csv``.
+extendable_carriers,,,,
+--,Generator,--,"Any subset  of {'OCGT','CCGT'}",Places extendable conventional power plants (OCGT and/or CCGT) where gas power plants are located today without capacity limits.
+--,StorageUnit,--,"Any subset of {'battery','H2'}",Adds extendable storage units (battery and/or hydrogen) at every node/bus after clustering without capacity limits and with zero initial capacity.
+--,Store,--,"Any subset of {'battery','H2'}",Adds extendable storage units (battery and/or hydrogen) at every node/bus after clustering without capacity limits and with zero initial capacity.
+--,Link,--,Any subset of {'H2 pipeline'},Adds extendable links (H2 pipelines only) at every connection where there are lines or HVDC links without capacity limits and with zero initial capacity. Hydrogen pipelines require hydrogen storage to be modelled as ``Store``.
+max_hours,,,,
+--,battery,h,float,Maximum state of charge capacity of the battery in terms of hours at full output capacity ``p_nom``. Cf. `PyPSA documentation <https://pypsa.readthedocs.io/en/latest/components.html#storage-unit>`_.
+--,H2,h,float,Maximum state of charge capacity of the hydrogen storage in terms of hours at full output capacity ``p_nom``. Cf. `PyPSA documentation <https://pypsa.readthedocs.io/en/latest/components.html#storage-unit>`_.
+powerplants_filter,--,"use `pandas.query <https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.query.html>`_ strings here, e.g. Country not in ['Germany']",Filter query for the default powerplant database.,
+custom_powerplants,--,"use `pandas.query <https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.query.html>`_ strings here, e.g. Country in ['Germany']",Filter query for the custom powerplant database.,
+conventional_carriers,--,"Any subset of {nuclear, oil, OCGT, CCGT, coal, lignite, geothermal, biomass}",List of conventional power plants to include in the model from ``resources/powerplants.csv``.,
+renewable_capacities_from_OPSD,,[solar, onwind, offwind],List of carriers (offwind-ac and offwind-dc are included in offwind) whose capacities 'p_nom' are aligned to the `OPSD renewable power plant list <https://data.open-power-system-data.org/renewable_power_plants/>`_,
+,"Fueltype [ppm], e.g. “Wind”","list of fueltypes stings in PyPSA-EUR, eg. “[onwind, offwind-ac, offwind-dc]”",converts ppm Fueltype to PyPSA-EUR Fueltype,
+estimate_renewable_capacities_from_capacitiy_stats,,,,
+,"Fueltype [ppm], e.g. “Wind”","list of fueltypes stings in PyPSA-EUR, eg. “[onwind, offwind-ac, offwind-dc]”",converts ppm Fueltype to PyPSA-EUR Fueltype,

doc/release_notes.rst

@@ -34,6 +34,8 @@ Upcoming Release
 
 * Don't remove capital costs from lines and links, when imposing a line volume limit (wildcard ``lv``) or a line cost limit (``lc``). Previously, these were removed to move the expansion in direction of the limit.
 
+* Add renewable power plants from `OPSD <https://data.open-power-system-data.org/renewable_power_plants/2020-08-25>`_ to the network for specified technologies. This will  overwrite the capacities calculated from the heuristic approach in :func:`estimate_renewable_capacities()`.
+
 * Fix bug of clustering offwind-{ac,dc} sites in the option of high-resolution sites for renewables. Now, there are more sites for offwind-{ac,dc} available than network nodes. Before, they were clustered to the resolution of the network. (e.g. elec_s1024_37m.nc: 37 network nodes, 1024 sites)
 
 * Use `mamba` (https://github.com/mamba-org/mamba) for faster Travis CI builds (`#196 <https://github.com/PyPSA/pypsa-eur/pull/196>`_)

envs/environment.yaml

@@ -25,7 +25,7 @@ dependencies:
   - yaml
   - pytables
   - lxml
-  - powerplantmatching>=0.4.3
+  - powerplantmatching>=0.4.8
   - numpy<=1.19.0 # otherwise macos fails
 
   # Second order dependencies which should really be deps of atlite

scripts/add_electricity.py

@@ -25,6 +25,8 @@ Relevant Settings
         co2limit:
         extendable_carriers:
             Generator:
+        OPSD_VRES_countries:
+        include_renewable_capacities_from_OPSD:
         estimate_renewable_capacities_from_capacity_stats:
 
     load:
@@ -93,7 +95,8 @@ import pandas as pd
 import numpy as np
 import xarray as xr
 import geopandas as gpd
-import powerplantmatching as ppm
+import powerplantmatching as pm
+from powerplantmatching.export import map_country_bus
 
 from vresutils.costdata import annuity
 from vresutils.load import timeseries_opsd
@@ -304,7 +307,7 @@ def attach_conventional_generators(n, costs, ppl):
     ppl = (ppl.query('carrier in @carriers').join(costs, on='carrier')
            .rename(index=lambda s: 'C' + str(s)))
 
-    logger.info('Adding {} generators with capacities\n{}'
+    logger.info('Adding {} generators with capacities [MW] \n{}'
                 .format(len(ppl), ppl.groupby('carrier').p_nom.sum()))
 
     n.madd("Generator", ppl.index,
@@ -467,6 +470,39 @@ def attach_extendable_generators(n, costs, ppl):
                                       "Only OCGT, CCGT and nuclear are allowed at the moment.")
 
 
+
+def attach_OPSD_renewables(n):
+
+    available = ['DE', 'FR', 'PL', 'CH', 'DK', 'CZ', 'SE', 'GB']
+    tech_map = {'Onshore': 'onwind', 'Offshore': 'offwind', 'Solar': 'solar'}
+    countries = set(available) & set(n.buses.country)
+    techs = snakemake.config['electricity'].get('renewable_capacities_from_OPSD', [])
+    tech_map = {k: v for k, v in tech_map.items() if v in techs}
+
+    if not tech_map:
+        return
+
+    logger.info(f'Using OPSD renewable capacities in {", ".join(countries)} '
+                f'for technologies {", ".join(tech_map.values())}.')
+
+    df = pd.concat([pm.data.OPSD_VRE_country(c) for c in countries])
+    technology_b = ~df.Technology.isin(['Onshore', 'Offshore'])
+    df['Fueltype'] = df.Fueltype.where(technology_b, df.Technology)
+    df = df.query('Fueltype in @tech_map').powerplant.convert_country_to_alpha2()
+
+    for fueltype, carrier_like in tech_map.items():
+        gens = n.generators[lambda df: df.carrier.str.contains(carrier_like)]
+        buses = n.buses.loc[gens.bus.unique()]
+        gens_per_bus = gens.groupby('bus').p_nom.count()
+
+        caps = map_country_bus(df.query('Fueltype == @fueltype'), buses)
+        caps = caps.groupby(['bus']).Capacity.sum()
+        caps = caps / gens_per_bus.reindex(caps.index, fill_value=1)
+
+        n.generators.p_nom.update(gens.bus.map(caps).dropna())
+
+
+
 def estimate_renewable_capacities(n, tech_map=None):
     if tech_map is None:
         tech_map = (snakemake.config['electricity']
@@ -474,16 +510,25 @@ def estimate_renewable_capacities(n, tech_map=None):
 
     if len(tech_map) == 0: return
 
-    capacities = (ppm.data.Capacity_stats().powerplant.convert_country_to_alpha2()
+    capacities = (pm.data.Capacity_stats().powerplant.convert_country_to_alpha2()
                   [lambda df: df.Energy_Source_Level_2]
                   .set_index(['Fueltype', 'Country']).sort_index())
 
     countries = n.buses.country.unique()
 
+    if len(countries) == 0: return
+
+    logger.info('heuristics applied to distribute renewable capacities [MW] \n{}'
+                .format(capacities.query('Fueltype in @tech_map.keys() and Capacity >= 0.1')
+                        .groupby('Country').agg({'Capacity': 'sum'})))
+
     for ppm_fueltype, techs in tech_map.items():
         tech_capacities = capacities.loc[ppm_fueltype, 'Capacity']\
                                     .reindex(countries, fill_value=0.)
-        tech_i = n.generators.query('carrier in @techs').index
+        #tech_i = n.generators.query('carrier in @techs').index
+        tech_i = (n.generators.query('carrier in @techs')
+                  [n.generators.query('carrier in @techs')
+                   .bus.map(n.buses.country).isin(countries)].index)
         n.generators.loc[tech_i, 'p_nom'] = (
             (n.generators_t.p_max_pu[tech_i].mean() *
              n.generators.loc[tech_i, 'p_nom_max']) # maximal yearly generation
@@ -528,6 +573,8 @@ if __name__ == "__main__":
     attach_extendable_generators(n, costs, ppl)
 
     estimate_renewable_capacities(n)
+    attach_OPSD_renewables(n)
+
     add_nice_carrier_names(n)
 
     n.export_to_netcdf(snakemake.output[0])