Merge branch 'master' into multiyear

.github/workflows/ci.yaml

@@ -104,6 +104,6 @@ jobs:
           conda activate pypsa-eur
           conda list
           cp test/config.overnight.yaml config.yaml
-          snakemake -call solve_all_networks
+          snakemake -call
           cp test/config.myopic.yaml config.yaml
-          snakemake -call solve_all_networks
+          snakemake -call

README.md

@@ -20,14 +20,16 @@ all greenhouse gas emitters except waste management and land use.
 **WARNING**: PyPSA-Eur-Sec is under active development and has several
 [limitations](https://pypsa-eur-sec.readthedocs.io/en/latest/limitations.html) which
 you should understand before using the model. The github repository
-[issues](https://github.com/PyPSA/pypsa-eur-sec/issues) collects known
+[issues](https://github.com/PyPSA/pypsa-eur-sec/issues) collect known
-topics we are working on (please feel free to help or make suggestions). There is neither a full
+topics we are working on (please feel free to help or make suggestions).
-documentation nor a paper yet, but we hope to have a preprint out by mid-2022.
+The [documentation](https://pypsa-eur-sec.readthedocs.io/) remains somewhat
-You can find out more about the model capabilities in [a recent
+patchy.
-presentation at EMP-E](https://nworbmot.org/energy/brown-empe.pdf) or the
+You can find showcases of the model's capabilities in the preprint
-following [paper in Joule with a description of the industry
+[Benefits of a Hydrogen Network in Europe](https://arxiv.org/abs/2207.05816),
-sector](https://arxiv.org/abs/2109.09563). We cannot support this model if you
+a [paper in Joule with a description of the industry
-choose to use it.
+sector](https://arxiv.org/abs/2109.09563), or in [a 2021
+presentation at EMP-E](https://nworbmot.org/energy/brown-empe.pdf).
+We cannot support this model if you choose to use it.
 
 Please see the [documentation](https://pypsa-eur-sec.readthedocs.io/)
 for installation instructions and other useful information about the snakemake workflow.

config.default.yaml

@@ -235,6 +235,7 @@ sector:
   marginal_cost_storage: 0. #1e-4
   methanation: true
   helmeth: true
+  coal_cc: false
   dac: true
   co2_vent: true
   SMR: true

doc/index.rst

@@ -33,17 +33,20 @@ waste management, agriculture, forestry and land use.
 **WARNING**: PyPSA-Eur-Sec is under active development and has several
 `limitations <https://pypsa-eur-sec.readthedocs.io/en/latest/limitations.html>`_ which
 you should understand before using the model. The github repository
-`issues <https://github.com/PyPSA/pypsa-eur-sec/issues>`_ collects known
+`issues <https://github.com/PyPSA/pypsa-eur-sec/issues>`_ collect known
-topics we are working on (please feel free to help or make suggestions). There is neither a full
+topics we are working on (please feel free to help or make suggestions).
-documentation nor a paper yet, but we hope to have a preprint out by mid-2022.
+The `documentation <https://pypsa-eur-sec.readthedocs.io/>`_ remains somewhat
-We cannot support this model if you
+patchy.
-choose to use it.
+We cannot support this model if you choose to use it.
-
 
 .. note::
-    More about the current model capabilities and preliminary results
+    You can find showcases of the model's capabilities in the
-    can be found in `a recent presentation at EMP-E <https://nworbmot.org/energy/brown-empe.pdf>`_
+    preprint `Benefits of a Hydrogen Network in Europe
-    and the following `paper in Joule with a description of the industry sector <https://arxiv.org/abs/2109.09563>`_.
+    <https://arxiv.org/abs/2207.05816>`_, a `paper in Joule with a
+    description of the industry sector
+    <https://arxiv.org/abs/2109.09563>`_, or in `a 2021 presentation
+    at EMP-E <https://nworbmot.org/energy/brown-empe.pdf>`_.
+
 
 This diagram gives an overview of the sectors and the links between
 them:

doc/release_notes.rst

@@ -24,7 +24,7 @@ incorporates retrofitting options to hydrogen.
 * New rule ``build_gas_input_locations`` compiles the LNG import capacities
   (including planned projects from gem.wiki), pipeline entry capacities and
   local production capacities for each region of the model. These are the
-  regions where fossil gas can eventually enter the model. 
+  regions where fossil gas can eventually enter the model.
 
 * New rule ``cluster_gas_network`` that clusters the gas transmission network
   data to the model resolution. Cross-regional pipeline capacities are aggregated
@@ -47,8 +47,8 @@ incorporates retrofitting options to hydrogen.
   H2_retrofit_capacity_per_CH4`` units are made available as hydrogen pipeline
   capacity in the corresponding corridor. These repurposed hydrogen pipelines
   have lower costs than new hydrogen pipelines. Both new and repurposed pipelines
-  can be built simultaneously. The retrofitting option ``sector: H2_retrofit:`` also works 
+  can be built simultaneously. The retrofitting option ``sector: H2_retrofit:`` also works
-  with a copperplated methane infrastructure, i.e. when ``sector: gas_network: false``. 
+  with a copperplated methane infrastructure, i.e. when ``sector: gas_network: false``.
 
 * New hydrogen pipelines can now be built where there are already power or gas
   transmission routes. Previously, only the electricity transmission routes were
@@ -56,6 +56,8 @@ incorporates retrofitting options to hydrogen.
 
 **New features and functionality**
 
+* Units are assigned to the buses. These only provide a better understanding. The specifications of the units are not taken into account in the optimisation, which means that no automatic conversion of units takes place.
+
 * Option ``retrieve_sector_databundle`` to automatically retrieve and extract data bundle.
 
 * Add regionalised hydrogen salt cavern storage potentials from `Technical Potential of Salt Caverns for Hydrogen Storage in Europe <https://doi.org/10.20944/preprints201910.0187.v1>`_.
@@ -84,7 +86,7 @@ besides many performance improvements.
 
 This release is known to work with `PyPSA-Eur
 <https://github.com/PyPSA/pypsa-eur>`_ Version 0.4.0, `Technology Data
-<https://github.com/PyPSA/technology-data>`_ Version 0.3.0 and 
+<https://github.com/PyPSA/technology-data>`_ Version 0.3.0 and
 `PyPSA <https://github.com/PyPSA/PyPSA>`_ Version 0.18.0.
 
 Please note that the data bundle has also been updated.
@@ -202,19 +204,19 @@ Please note that the data bundle has also been updated.
   A function ``helper.override_component_attrs`` was added that loads this data
   and can pass the overridden component attributes into ``pypsa.Network()``.
 
-* Add various parameters to ``config.default.yaml`` which were previously hardcoded inside the scripts 
+* Add various parameters to ``config.default.yaml`` which were previously hardcoded inside the scripts
   (e.g. energy reference years, BEV settings, solar thermal collector models, geomap colours).
 
 * Removed stale industry demand rules ``build_industrial_energy_demand_per_country``
   and ``build_industrial_demand``. These are superseded with more regionally resolved rules.
 
 * Use simpler and shorter ``gdf.sjoin()`` function to allocate industrial sites
-  from the Hotmaps database to onshore regions.  
+  from the Hotmaps database to onshore regions.
   This change also fixes a bug:
   The previous version allocated sites to the closest bus,
   but at country borders (where Voronoi cells are distorted by the borders),
   this had resulted in e.g. a Spanish site close to the French border
-  being wrongly allocated to the French bus if the bus center was closer. 
+  being wrongly allocated to the French bus if the bus center was closer.
 
 * Retrofitting rule is now only triggered if endogeneously optimised.
 
@@ -225,7 +227,7 @@ Please note that the data bundle has also been updated.
 * Improve legibility of ``config.default.yaml`` and remove unused options.
 
 * Use the country-specific time zone mappings from ``pytz`` rather than a manual mapping.
-  
+
 * A function ``add_carrier_buses()`` was added to the ``prepare_network`` rule to reduce code duplication.
 
 * In the ``prepare_network`` rule the cost and potential adjustment was moved into an

scripts/copy_config.py

@@ -1,5 +1,6 @@
 
 from shutil import copy
+import yaml
 
 files = {
     "config.yaml": "config.yaml",
@@ -14,5 +15,16 @@ if __name__ == '__main__':
         from helper import mock_snakemake
         snakemake = mock_snakemake('copy_config')
 
+    basepath = snakemake.config['summary_dir'] + '/' + snakemake.config['run'] + '/configs/'
+
     for f, name in files.items():
-        copy(f,snakemake.config['summary_dir'] + '/' + snakemake.config['run'] + '/configs/' + name)
+        copy(f, basepath + name)
+
+    with open(basepath + 'config.snakemake.yaml', 'w') as yaml_file:
+        yaml.dump(
+            snakemake.config,
+            yaml_file,
+            default_flow_style=False,
+            allow_unicode=True,
+            sort_keys=False
+        )

scripts/helper.py

@@ -58,6 +58,7 @@ def mock_snakemake(rulename, **wildcards):
     import os
     from pypsa.descriptors import Dict
     from snakemake.script import Snakemake
+    from packaging.version import Version, parse
 
     script_dir = Path(__file__).parent.resolve()
     assert Path.cwd().resolve() == script_dir, \
@@ -67,7 +68,8 @@ def mock_snakemake(rulename, **wildcards):
         if os.path.exists(p):
             snakefile = p
             break
-    workflow = sm.Workflow(snakefile, overwrite_configfiles=[])
+    kwargs = dict(rerun_triggers=[]) if parse(sm.__version__) > Version("7.7.0") else {}
+    workflow = sm.Workflow(snakefile, overwrite_configfiles=[], **kwargs)
     workflow.include(snakefile)
     workflow.global_resources = {}
     rule = workflow.get_rule(rulename)

scripts/prepare_sector_network.py

@@ -385,10 +385,13 @@ def add_carrier_buses(n, carrier, nodes=None):
 
     n.add("Carrier", carrier)
 
+    unit = "MWh_LHV" if carrier == "gas" else "MWh_th"
+
     n.madd("Bus",
         nodes,
         location=location,
-        carrier=carrier
+        carrier=carrier,
+        unit=unit
     )
 
     #capital cost could be corrected to e.g. 0.2 EUR/kWh * annuity and O&M
@@ -439,6 +442,7 @@ def patch_electricity_network(n):
     update_wind_solar_costs(n, costs)
     n.loads["carrier"] = "electricity"
     n.buses["location"] = n.buses.index
+    n.buses["unit"] = "MWh_el"
     # remove trailing white space of load index until new PyPSA version after v0.18.
     n.loads.rename(lambda x: x.strip(), inplace=True)
     n.loads_t.p_set.rename(lambda x: x.strip(), axis=1, inplace=True)
@@ -455,7 +459,8 @@ def add_co2_tracking(n, options):
     n.add("Bus",
         "co2 atmosphere",
         location="EU",
-        carrier="co2"
+        carrier="co2",
+        unit="t_co2"
     )
 
     # can also be negative
@@ -471,7 +476,8 @@ def add_co2_tracking(n, options):
     n.madd("Bus",
         spatial.co2.nodes,
         location=spatial.co2.locations,
-        carrier="co2 stored"
+        carrier="co2 stored",
+        unit="t_co2"
     )
 
     n.madd("Store",
@@ -703,7 +709,8 @@ def insert_electricity_distribution_grid(n, costs):
     n.madd("Bus",
         nodes + " low voltage",
         location=nodes,
-        carrier="low voltage"
+        carrier="low voltage",
+        unit="MWh_el"
     )
 
     n.madd("Link",
@@ -770,7 +777,8 @@ def insert_electricity_distribution_grid(n, costs):
     n.madd("Bus",
         nodes + " home battery",
         location=nodes,
-        carrier="home battery"
+        carrier="home battery",
+        unit="MWh_el"
     )
 
     n.madd("Store",
@@ -845,7 +853,8 @@ def add_storage_and_grids(n, costs):
     n.madd("Bus",
         nodes + " H2",
         location=nodes,
-        carrier="H2"
+        carrier="H2",
+        unit="MWh_LHV"
     )
 
     n.madd("Link",
@@ -1051,7 +1060,8 @@ def add_storage_and_grids(n, costs):
     n.madd("Bus",
         nodes + " battery",
         location=nodes,
-        carrier="battery"
+        carrier="battery",
+        unit="MWh_el"
     )
 
     n.madd("Store",
@@ -1118,6 +1128,24 @@ def add_storage_and_grids(n, costs):
             lifetime=costs.at['helmeth', 'lifetime']
         )
 
+    if options.get('coal_cc'):
+
+        n.madd("Link",
+            spatial.nodes,
+            suffix=" coal CC",
+            bus0=spatial.coal.nodes,
+            bus1=spatial.nodes,
+            bus2="co2 atmosphere",
+            bus3="co2 stored",
+            marginal_cost=costs.at['coal', 'efficiency'] * costs.at['coal', 'VOM'], #NB: VOM is per MWel
+            capital_cost=costs.at['coal', 'efficiency'] * costs.at['coal', 'fixed'] + costs.at['biomass CHP capture', 'fixed'] * costs.at['coal', 'CO2 intensity'], #NB: fixed cost is per MWel
+            p_nom_extendable=True,
+            carrier="coal",
+            efficiency=costs.at['coal', 'efficiency'],
+            efficiency2=costs.at['coal', 'CO2 intensity'] * (1 - costs.at['biomass CHP capture','capture_rate']),
+            efficiency3=costs.at['coal', 'CO2 intensity'] * costs.at['biomass CHP capture','capture_rate'],
+            lifetime=costs.at['coal','lifetime']
+        )
 
     if options['SMR']:
 
@@ -1181,7 +1209,8 @@ def add_land_transport(n, costs):
             nodes,
             location=nodes,
             suffix=" EV battery",
-            carrier="Li ion"
+            carrier="Li ion",
+            unit="MWh_el"
         )
 
         p_set = electric_share * (transport[nodes] + cycling_shift(transport[nodes], 1) + cycling_shift(transport[nodes], 2)) / 3
@@ -1255,7 +1284,8 @@ def add_land_transport(n, costs):
             n.madd("Bus",
                 spatial.oil.nodes,
                 location=spatial.oil.locations,
-                carrier="oil"
+                carrier="oil",
+                unit="MWh_LHV"
             )
 
         ice_efficiency = options['transport_internal_combustion_efficiency']
@@ -1363,7 +1393,8 @@ def add_heat(n, costs):
         n.madd("Bus",
             nodes[name] + f" {name} heat",
             location=nodes[name],
-            carrier=name + " heat"
+            carrier=name + " heat",
+            unit="MWh_th"
         )
 
         ## Add heat load
@@ -1420,7 +1451,8 @@ def add_heat(n, costs):
             n.madd("Bus",
                 nodes[name] + f" {name} water tanks",
                 location=nodes[name],
-                carrier=name + " water tanks"
+                carrier=name + " water tanks",
+                unit="MWh_th"
             )
 
             n.madd("Link",
@@ -1449,9 +1481,6 @@ def add_heat(n, costs):
                                "for 'decentral' and 'central' separately.")
                 tes_time_constant_days = options["tes_tau"] if name_type == "decentral" else 180.
 
-            # conversion from EUR/m^3 to EUR/MWh for 40 K diff and 1.17 kWh/m^3/K
-            capital_cost = costs.at[name_type + ' water tank storage', 'fixed'] / 0.00117 / 40
-
             n.madd("Store",
                 nodes[name] + f" {name} water tanks",
                 bus=nodes[name] + f" {name} water tanks",
@@ -1459,7 +1488,7 @@ def add_heat(n, costs):
                 e_nom_extendable=True,
                 carrier=name + " water tanks",
                 standing_loss=1 - np.exp(- 1 / 24 / tes_time_constant_days),
-                capital_cost=capital_cost,
+                capital_cost=costs.at[name_type + ' water tank storage', 'fixed'],
                 lifetime=costs.at[name_type + ' water tank storage', 'lifetime']
             )
 
@@ -1725,13 +1754,15 @@ def add_biomass(n, costs):
     n.madd("Bus",
         spatial.gas.biogas,
         location=spatial.gas.locations,
-        carrier="biogas"
+        carrier="biogas",
+        unit="MWh_LHV"
     )
 
     n.madd("Bus",
         spatial.biomass.nodes,
         location=spatial.biomass.locations,
-        carrier="solid biomass"
+        carrier="solid biomass",
+        unit="MWh_LHV"
     )
 
     n.madd("Store",
@@ -1842,7 +1873,8 @@ def add_industry(n, costs):
     n.madd("Bus",
         spatial.biomass.industry,
         location=spatial.biomass.locations,
-        carrier="solid biomass for industry"
+        carrier="solid biomass for industry",
+        unit="MWh_LHV"
     )
 
     if options["biomass_transport"]:
@@ -1884,7 +1916,8 @@ def add_industry(n, costs):
     n.madd("Bus",
         spatial.gas.industry,
         location=spatial.gas.locations,
-        carrier="gas for industry")
+        carrier="gas for industry",
+        unit="MWh_LHV")
 
     gas_demand = industrial_demand.loc[nodes, "methane"] / 8760.
 
@@ -1940,7 +1973,8 @@ def add_industry(n, costs):
             nodes,
             suffix=" H2 liquid",
             carrier="H2 liquid",
-            location=nodes
+            location=nodes,
+            unit="MWh_LHV"
         )
 
         n.madd("Link",
@@ -1998,7 +2032,8 @@ def add_industry(n, costs):
         n.madd("Bus",
             spatial.oil.nodes,
             location=spatial.oil.locations,
-            carrier="oil"
+            carrier="oil",
+            unit="MWh_LHV"
         )
 
     if "oil" not in n.stores.carrier.unique():
@@ -2112,7 +2147,8 @@ def add_industry(n, costs):
     n.add("Bus",
         "process emissions",
         location="EU",
-        carrier="process emissions"
+        carrier="process emissions",
+        unit="t_co2"
     )
 
     # this should be process emissions fossil+feedstock
@@ -2297,7 +2333,7 @@ if __name__ == "__main__":
             simpl='',
             opts="",
             clusters="37",
-            lv=1.0,
+            lv=1.5,
             sector_opts='Co2L0-168H-T-H-B-I-solar3-dist1',
             planning_horizons="2020",
         )