Merge branch 'master' into exclude_shared_resources_master

config/config.default.yaml

@@ -40,17 +40,15 @@ scenario:
   simpl:
   - ''
   ll:
-  - v1.5
+  - vopt
   clusters:
   - 37
   - 128
   - 256
-  - 512
-  - 1024
   opts:
   - ''
   sector_opts:
-  - Co2L0-3H-T-H-B-I-A-dist1
+  - ''
   planning_horizons:
   # - 2020
   # - 2030
@@ -555,7 +553,7 @@ sector:
   - nearshore    # within 50 km of sea
     # - offshore
   ammonia: false
-  min_part_load_fischer_tropsch: 0.7
+  min_part_load_fischer_tropsch: 0.5
   min_part_load_methanolisation: 0.3
   min_part_load_methanation: 0.3
   use_fischer_tropsch_waste_heat: true

config/test/config.electricity.yaml

@@ -16,7 +16,7 @@ scenario:
   clusters:
   - 5
   opts:
-  - Co2L-24h
+  - ''
 
 countries: ['BE']
 
@@ -25,6 +25,7 @@ snapshots:
   end: "2013-03-08"
 
 electricity:
+  co2limit_enable: true
   co2limit: 100.e+6
 
   extendable_carriers:
@@ -64,6 +65,8 @@ renewable:
 
 clustering:
   exclude_carriers: ["OCGT", "offwind-ac", "coal"]
+  temporal:
+    resolution_elec: 24h
 
 lines:
   dynamic_line_rating:

config/test/config.myopic.yaml

@@ -19,7 +19,7 @@ scenario:
   clusters:
   - 5
   sector_opts:
-  - 24h-T-H-B-I-A-dist1
+  - ''
   planning_horizons:
   - 2030
   - 2040
@@ -35,7 +35,6 @@ sector:
   central_heat_vent: true
 
 electricity:
-  co2limit: 100.e+6
 
   extendable_carriers:
     Generator: [OCGT]
@@ -70,6 +69,10 @@ renewable:
   solar:
     cutout: be-03-2013-era5
 
+clustering:
+  temporal:
+    resolution_sector: 24h
+
 industry:
   St_primary_fraction:
     2030: 0.6

config/test/config.overnight.yaml

@@ -18,7 +18,7 @@ scenario:
   clusters:
   - 5
   sector_opts:
-  - CO2L0-24h-T-H-B-I-A-dist1
+  - ''
   planning_horizons:
   - 2030
 
@@ -29,7 +29,6 @@ snapshots:
   end: "2013-03-08"
 
 electricity:
-  co2limit: 100.e+6
 
   extendable_carriers:
     Generator: [OCGT]
@@ -64,6 +63,10 @@ renewable:
   solar:
     cutout: be-03-2013-era5
 
+clustering:
+  temporal:
+    resolution_sector: 24h
+
 sector:
   gas_network: true
   H2_retrofit: true

config/test/config.perfect.yaml

@@ -19,7 +19,7 @@ scenario:
   clusters:
   - 5
   sector_opts:
-  - 8760h-T-H-B-I-A-dist1
+  - ''
   planning_horizons:
   - 2030
   - 2040
@@ -32,7 +32,6 @@ snapshots:
   end: "2013-03-08"
 
 electricity:
-  co2limit: 100.e+6
 
   extendable_carriers:
     Generator: [OCGT]
@@ -71,6 +70,10 @@ renewable:
   solar:
     cutout: be-03-2013-era5
 
+clustering:
+  temporal:
+    resolution_sector: 8760h
+
 industry:
   St_primary_fraction:
     2020: 0.8

config/test/config.scenarios.yaml

@@ -20,7 +20,7 @@ scenario:
   clusters:
   - 5
   opts:
-  - Co2L-24H
+  - ''
 
 countries: ['BE']
 

doc/configtables/electricity.csv

@@ -2,7 +2,7 @@
 voltages,kV,"Any subset of {220., 300., 380.}",Voltage levels to consider
 gaslimit_enable,bool,true or false,Add an overall absolute gas limit configured in ``electricity: gaslimit``.
 gaslimit,MWhth,float or false,Global gas usage limit
-co2limit_enable,bool,true or false,Add an overall absolute carbon-dioxide emissions limit configured in ``electricity: co2limit``.
+co2limit_enable,bool,true or false,Add an overall absolute carbon-dioxide emissions limit configured in ``electricity: co2limit`` in :mod:`prepare_network`. **Warning:** This option should currently only be used with electricity-only networks, not for sector-coupled networks..
 co2limit,:math:`t_{CO_2-eq}/a`,float,Cap on total annual system carbon dioxide emissions
 co2base,:math:`t_{CO_2-eq}/a`,float,Reference value of total annual system carbon dioxide emissions if relative emission reduction target is specified in ``{opts}`` wildcard.
 agg_p_nom_limits,file,path,Reference to ``.csv`` file specifying per carrier generator nominal capacity constraints for individual countries if ``'CCL'`` is in ``{opts}`` wildcard. Defaults to ``data/agg_p_nom_minmax.csv``.

doc/configtables/sector-opts.csv

@@ -1,5 +1,5 @@
 Trigger, Description, Definition, Status
-``nH``, i.e. ``2H``-``6H``, "Resample the time-resolution by averaging over every ``n`` snapshots, ``prepare_network``: `average_every_nhours() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L110>`_ and its `caller <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L146>`__)", In active use
+``nH``, i.e. ``2h``-``6h``, "Resample the time-resolution by averaging over every ``n`` snapshots, ``prepare_network``: `average_every_nhours() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L110>`_ and its `caller <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L146>`__)", In active use
 ``Co2L``, Add an overall absolute carbon-dioxide emissions limit configured in ``electricity: co2limit``. If a float is appended an overall emission limit relative to the emission level given in ``electricity: co2base`` is added (e.g. ``Co2L0.05`` limits emissisions to 5% of what is given in ``electricity: co2base``), ``prepare_network``: `add_co2limit() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L19>`_ and its `caller <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L154>`__, In active use
 ``carrier+{c|p|m}factor``,"Alter the capital cost (``c``), installable potential (``p``) or marginal costs (``m``) of a carrier by a factor. Example: ``solar+c0.5`` reduces the capital cost of solar to 50\% of original values.", ``prepare_network``, In active use
 ``T``,Add land transport sector,,In active use

doc/introduction.rst

@@ -35,7 +35,7 @@ For instance, an invocation to
 
 .. code:: bash
 
-    .../pypsa-eur % snakemake -call results/networks/elec_s_128_ec_lvopt_Co2L-3H.nc
+    .../pypsa-eur % snakemake -call results/networks/elec_s_128_ec_lvopt_.nc
 
 follows this dependency graph
 
@@ -50,7 +50,7 @@ preceding rules which another rule takes as input data.
 
 .. note::
     The dependency graph was generated using
-    ``snakemake --dag results/networks/elec_s_128_ec_lvopt_Co2L-3H.nc -F | sed -n "/digraph/,/}/p" | dot -Tpng -o doc/img/intro-workflow.png``
+    ``snakemake --dag results/networks/elec_s_128_ec_lvopt_.nc -F | sed -n "/digraph/,/}/p" | dot -Tpng -o doc/img/intro-workflow.png``
 
 For the use of ``snakemake``, it makes sense to familiarize yourself quickly
 with the `basic tutorial

doc/release_notes.rst

@@ -10,6 +10,8 @@ Release Notes
 Upcoming Release
 ================
 
+* Bump minimum ``powerplantmatching`` version to v0.5.15.
+
 * Add floating wind technology for water depths below 60m
 
 * Add config ``run: shared_resources: exclude:`` to specify additional files
@@ -228,6 +230,8 @@ Upcoming Release
 
 * Fix custom busmap read in `cluster_network`.
 
+* Add `nodal_supply_energy` to `make_summary`.
+
 * Data on existing renewable capacities is now consistently taken from powerplantmatching (instead of being retrieved separately); the dataset has also been updated to include 2023 values.
 
 * Added shapes to .nc file for different stages of the network object in `base_network`, `simplify_network`, and `cluster_network`; the `build_bus_regions` rule is now integrated into the `base_network` rule.
@@ -238,6 +242,8 @@ Upcoming Release
 
 * Clarify suffix usage in `add_existing_baseyear`.
 
+* The ``{sector_opts}`` wildcard is now not used by default. All scenario definitions are now done in the ``config.yaml`` file.
+
 PyPSA-Eur 0.10.0 (19th February 2024)
 =====================================
 

doc/tutorial.rst

@@ -32,7 +32,7 @@ configuration, execute
 .. code:: bash
     :class: full-width
 
-    snakemake -call results/test-elec/networks/elec_s_6_ec_lcopt_Co2L-24H.nc --configfile config/test/config.electricity.yaml
+    snakemake -call results/test-elec/networks/elec_s_6_ec_lcopt_.nc --configfile config/test/config.electricity.yaml
 
 This configuration is set to download a reduced cutout via the rule :mod:`retrieve_cutout`.
 For more information on the data dependencies of PyPSA-Eur, continue reading :ref:`data`.
@@ -114,9 +114,9 @@ clustered down to 6 buses and every 24 hours aggregated to one snapshot. The com
 
 .. code:: bash
 
-    snakemake -call results/test-elec/networks/elec_s_6_ec_lcopt_Co2L-24H.nc --configfile config/test/config.electricity.yaml
+    snakemake -call results/test-elec/networks/elec_s_6_ec_lcopt_.nc --configfile config/test/config.electricity.yaml
 
-orders ``snakemake`` to run the rule :mod:`solve_network` that produces the solved network and stores it in ``results/networks`` with the name ``elec_s_6_ec_lcopt_Co2L-24H.nc``:
+orders ``snakemake`` to run the rule :mod:`solve_network` that produces the solved network and stores it in ``results/networks`` with the name ``elec_s_6_ec_lcopt_.nc``:
 
 .. literalinclude:: ../rules/solve_electricity.smk
    :start-at: rule solve_network:
@@ -133,7 +133,7 @@ This triggers a workflow of multiple preceding jobs that depend on each rule's i
         node[shape=box, style=rounded, fontname=sans,                 fontsize=10, penwidth=2];
         edge[penwidth=2, color=grey];
             0[label = "solve_network", color = "0.38 0.6 0.85", style="rounded"];
-            1[label = "prepare_network\nll: copt\nopts: Co2L-24H", color = "0.53 0.6 0.85", style="rounded"];
+            1[label = "prepare_network\nll: copt", color = "0.53 0.6 0.85", style="rounded"];
             2[label = "add_extra_components", color = "0.01 0.6 0.85", style="rounded"];
             3[label = "cluster_network\nclusters: 6", color = "0.03 0.6 0.85", style="rounded"];
             4[label = "simplify_network\nsimpl: ", color = "0.42 0.6 0.85", style="rounded"];
@@ -268,7 +268,7 @@ For example, you can explore the evolution of the PyPSA networks by running
 #. ``snakemake resources/networks/elec.nc -call --configfile config/test/config.electricity.yaml``
 #. ``snakemake resources/networks/elec_s.nc -call --configfile config/test/config.electricity.yaml``
 #. ``snakemake resources/networks/elec_s_6.nc -call --configfile config/test/config.electricity.yaml``
-#. ``snakemake resources/networks/elec_s_6_ec_lcopt_Co2L-24H.nc -call --configfile config/test/config.electricity.yaml``
+#. ``snakemake resources/networks/elec_s_6_ec_lcopt_.nc -call --configfile config/test/config.electricity.yaml``
 
 To run all combinations of wildcard values provided in the ``config/config.yaml`` under ``scenario:``,
 you can use the collection rule ``solve_elec_networks``.
@@ -306,6 +306,6 @@ Jupyter Notebooks).
 
     import pypsa
 
-    n = pypsa.Network("results/networks/elec_s_6_ec_lcopt_Co2L-24H.nc")
+    n = pypsa.Network("results/networks/elec_s_6_ec_lcopt_.nc")
 
 For inspiration, read the `examples section in the PyPSA documentation <https://pypsa.readthedocs.io/en/latest/examples-basic.html>`__.

doc/wildcards.rst

@@ -101,7 +101,7 @@ The ``{opts}`` wildcard
 The ``{opts}`` wildcard is used for electricity-only studies. It triggers
 optional constraints, which are activated in either :mod:`prepare_network` or
 the :mod:`solve_network` step. It may hold multiple triggers separated by ``-``,
-i.e. ``Co2L-3H`` contains the ``Co2L`` trigger and the ``3H`` switch. There are
+i.e. ``Co2L-3h`` contains the ``Co2L`` trigger and the ``3h`` switch. There are
 currently:
 
 
@@ -121,7 +121,7 @@ The ``{sector_opts}`` wildcard
 
   # Co2Lx specifies the CO2 target in x% of the 1990 values; default will give default (5%);
   # Co2L0p25 will give 25% CO2 emissions; Co2Lm0p05 will give 5% negative emissions
-  # xH is the temporal resolution; 3H is 3-hourly, i.e. one snapshot every 3 hours
+  # xH is the temporal resolution; 3h is 3-hourly, i.e. one snapshot every 3 hours
   # single letters are sectors: T for land transport, H for building heating,
   # B for biomass supply, I for industry, shipping and aviation,
   # A for agriculture, forestry and fishing

envs/environment.yaml

@@ -25,7 +25,7 @@ dependencies:
 - yaml
 - pytables
 - lxml
-- powerplantmatching>=0.5.13
+- powerplantmatching>=0.5.15
 - numpy
 - pandas>=2.1
 - geopandas>=0.11.0

rules/postprocess.smk

@@ -199,6 +199,7 @@ rule make_summary:
         energy=RESULTS + "csvs/energy.csv",
         supply=RESULTS + "csvs/supply.csv",
         supply_energy=RESULTS + "csvs/supply_energy.csv",
+        nodal_supply_energy=RESULTS + "csvs/nodal_supply_energy.csv",
         prices=RESULTS + "csvs/prices.csv",
         weighted_prices=RESULTS + "csvs/weighted_prices.csv",
         market_values=RESULTS + "csvs/market_values.csv",

scripts/build_energy_totals.py

@@ -142,6 +142,7 @@ def build_eurostat(input_eurostat, countries, nprocesses=1, disable_progressbar=
         "Domestic navigation": "Domestic Navigation",
         "International maritime bunkers": "Bunkers",
         "UK": "GB",
+        "EL": "GR",
     }
     columns_rename = {"Total": "Total all products"}
     df.rename(index=index_rename, columns=columns_rename, inplace=True)

scripts/make_summary.py

@@ -413,6 +413,85 @@ def calculate_supply_energy(n, label, supply_energy):
     return supply_energy
 
 
+def calculate_nodal_supply_energy(n, label, nodal_supply_energy):
+    """
+    Calculate the total energy supply/consumption of each component at the
+    buses aggregated by carrier and node.
+    """
+
+    bus_carriers = n.buses.carrier.unique()
+
+    for i in bus_carriers:
+        bus_map = n.buses.carrier == i
+        bus_map.at[""] = False
+
+        for c in n.iterate_components(n.one_port_components):
+            items = c.df.index[c.df.bus.map(bus_map).fillna(False)]
+
+            if len(items) == 0:
+                continue
+
+            s = (
+                pd.concat(
+                    [
+                        (
+                            c.pnl.p[items]
+                            .multiply(n.snapshot_weightings.generators, axis=0)
+                            .sum()
+                            .multiply(c.df.loc[items, "sign"])
+                        ),
+                        c.df.loc[items][["bus", "carrier"]],
+                    ],
+                    axis=1,
+                )
+                .groupby(by=["bus", "carrier"])
+                .sum()[0]
+            )
+            s = pd.concat([s], keys=[c.list_name])
+            s = pd.concat([s], keys=[i])
+
+            nodal_supply_energy = nodal_supply_energy.reindex(
+                s.index.union(nodal_supply_energy.index)
+            )
+            nodal_supply_energy.loc[s.index, label] = s
+
+        for c in n.iterate_components(n.branch_components):
+            for end in [col[3:] for col in c.df.columns if col[:3] == "bus"]:
+                items = c.df.index[c.df["bus" + str(end)].map(bus_map).fillna(False)]
+
+                if (len(items) == 0) or c.pnl["p" + end].empty:
+                    continue
+
+                s = (
+                    pd.concat(
+                        [
+                            (
+                                (-1)
+                                * c.pnl["p" + end][items]
+                                .multiply(n.snapshot_weightings.generators, axis=0)
+                                .sum()
+                            ),
+                            c.df.loc[items][["bus0", "carrier"]],
+                        ],
+                        axis=1,
+                    )
+                    .groupby(by=["bus0", "carrier"])
+                    .sum()[0]
+                )
+
+                s.index = s.index.map(lambda x: (x[0], x[1] + end))
+                s = pd.concat([s], keys=[c.list_name])
+                s = pd.concat([s], keys=[i])
+
+                nodal_supply_energy = nodal_supply_energy.reindex(
+                    s.index.union(nodal_supply_energy.index)
+                )
+
+                nodal_supply_energy.loc[s.index, label] = s
+
+    return nodal_supply_energy
+
+
 def calculate_metrics(n, label, metrics):
     metrics_list = [
         "line_volume",
@@ -637,6 +716,7 @@ def make_summaries(networks_dict):
         "energy",
         "supply",
         "supply_energy",
+        "nodal_supply_energy",
         "prices",
         "weighted_prices",
         "price_statistics",

scripts/prepare_sector_network.py

@@ -3578,7 +3578,7 @@ if __name__ == "__main__":
             opts="",
             clusters="37",
             ll="v1.0",
-            sector_opts="CO2L0-24H-T-H-B-I-A-dist1",
+            sector_opts="CO2L0-24h-T-H-B-I-A-dist1",
             planning_horizons="2030",
         )