diff --git a/doc/release_notes.rst b/doc/release_notes.rst index c216fd35..0e96f013 100644 --- a/doc/release_notes.rst +++ b/doc/release_notes.rst @@ -10,12 +10,71 @@ Release Notes Upcoming Release ================ -* PyPSA-EUR now supports the simultaneous execution of multiple scenarios. For - this purpose, a scenarios.yaml file has been introduced which contains - customizable scenario names with corresponding configuration overrides. To - enable it, set the ``run: scenarios:`` key to ``True`` and define the scenario - names to run under ``run: name:`` in the configuration file. The latter must - be a subset of toplevel keys in the scenario file. +* Added new scenario management that supports the simultaneous execution of + multiple scenarios with a single ``snakemake`` call. For this purpose, a + ``scenarios.yaml`` file is introduced which contains customizable scenario + names with configuration overrides. To enable it, set the ``run: scenarios: + true`` and define the list of scenario names to run under ``run: name:`` in + the configuration file. The latter must be a subset of toplevel keys in the + scenario file. + + - To get started, a scenarios template file ``config/scenarios.template.yaml`` + is included in the repository, which is copied to ``config/scenarios.yaml`` + on first use. + + - The scenario file can be changed via ``run: scenarios: file:``. + + - If scenario management is activated with ``run: scenarios: enable: true``, a + new wildcard ``{run}`` is introduced. This means that the configuration + settings may depend on the new ``{run}`` wildcard. Therefore, a new + ``config_provider()`` function is used in the ``Snakefile`` and ``.smk`` + files, which takes wildcard values into account. The calls to the ``config`` + object have been reduced in ``.smk`` files since there is no awareness of + wildcard values outside rule definitions. + + - The scenario files can also be programmatically created using the template + script ``config/create_scenarios.py``. This script can be run with + ``snakemake -j1 create_scenarios`` and creates the scenarios file referenced + under ``run: scenarios: file:``. + + - The setting ``run: name: all`` will run all scenarios in + ``config/scenarios.yaml``. Otherwise, it will run those passed as list in + ``run: name:`` as long as ``run: scenarios: enable: true``. + + - The setting ``run: shared_resources:`` indicates via a boolean whether the + resources should be encapsulated by the ``run: name:``. The special setting + ``run: shared_resources: base`` shares resources until ``add_electricity`` + that do not contain wildcards other than ``{"technology", "year", + "scope"}``. + + - Added new configuration options for all ``{opts}`` and ``{sector_opts}`` + wildcard values to create a unique configuration file (``config.yaml``) per + PyPSA network file. This is done with the help of a new function + ``update_config_from_wildcards()`` which parses configuration settings from + wildcards and updates the ``snakemake.config`` object. These updated + configuration settings are used in the scripts rather than directly parsed + values from ``snakemake.wildcards``. + + - The cost data was moved from ``data/costs_{year}.csv`` to + ``resources/costs_{year}.csv`` since it depends on configuration settings. + The ``retrieve_cost_data`` rule was changed to calling a Python script. + + - Moved time clustering settings to ``clustering: temporal:`` from + ``snapshots:`` so that the latter is only used to define the + ``pandas.DatetimeIndex`` which simplifies the scenario management. + + - Collection rules get a new wildcard ``run=config["run"]["name"]`` so they + can collect outputs across different scenarios. + + - **Warning:** One caveat remains for the scenario management with myopic or + perfect foresight pathway optimisation. The first investment period must be + shared across all scenarios. The reason is that the ``wildcard_constraints`` + defined for the rule ``add_existing_baseyear`` do not accept wildcard-aware + input functions (cf. + `https://github.com/snakemake/snakemake/issues/2703`_). + +* The outputs of the rule ``retrieve_gas_infrastructure_data`` no longer + marked as ``protected()`` as the download size is small. * Improved representation of industry transition pathways. A new script was added to interpolate industry sector ratios from today's status quo to future diff --git a/rules/validate.smk b/rules/validate.smk index ec2d87cb..60a6ca50 100644 --- a/rules/validate.smk +++ b/rules/validate.smk @@ -17,10 +17,7 @@ rule build_electricity_production: The data is used for validation of the optimization results. """ params: - snapshots=lambda w: { - k: config_provider("snapshots", k)(w) - for k in ["start", "end", "inclusive"] - }, + snapshots=config_provider("snapshots"), countries=config_provider("countries"), output: resources("historical_electricity_production.csv"),