bjoern/pypsa-eur
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Merge pull request #640 from PyPSA/configs-in-one-folder

Browse Source 
Configs in one folder
This commit is contained in:
Fabian Neumann 2023-04-21 16:09:29 +02:00 committed by GitHub
commit 2b069e3031
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
40 changed files with 92 additions and 88 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
.github/workflows/ci.yaml vendored
View File

@ -103,9 +103,9 @@ jobs:
run: | run: |
conda activate pypsa-eur conda activate pypsa-eur
conda list conda list
snakemake -call solve_elec_networks --configfile test/config.electricity.yaml --rerun-triggers=mtime snakemake -call solve_elec_networks --configfile config/test/config.electricity.yaml --rerun-triggers=mtime
snakemake -call all --configfile test/config.overnight.yaml --rerun-triggers=mtime snakemake -call all --configfile config/test/config.overnight.yaml --rerun-triggers=mtime
snakemake -call all --configfile test/config.myopic.yaml --rerun-triggers=mtime snakemake -call all --configfile config/test/config.myopic.yaml --rerun-triggers=mtime
- name: Upload artifacts - name: Upload artifacts
uses: actions/upload-artifact@v3 uses: actions/upload-artifact@v3

1
.syncignore-receive
View File

@ -18,3 +18,4 @@ cutouts
data data
benchmarks benchmarks
*.nc *.nc
configs

6
Snakefile
View File

@ -14,11 +14,11 @@ from snakemake.utils import min_version
min_version("7.7") min_version("7.7")
if not exists("config.yaml"): if not exists("config/config.yaml"):
copyfile("config.default.yaml", "config.yaml") copyfile("config/config.default.yaml", "config/config.yaml")
configfile: "config.yaml" configfile: "config/config.yaml"
COSTS = f"data/costs_{config['costs']['year']}.csv" COSTS = f"data/costs_{config['costs']['year']}.csv"

0
config.default.yaml → config/config.default.yaml Executable file → Normal file
View File

0
test/config.electricity.yaml → config/test/config.electricity.yaml Executable file → Normal file
View File

0
test/config.myopic.yaml → config/test/config.myopic.yaml
View File

0
test/config.overnight.yaml → config/test/config.overnight.yaml
View File

56
doc/configuration.rst
View File

@ -9,14 +9,14 @@
Configuration Configuration
########################################## ##########################################
PyPSA-Eur has several configuration options which are documented in this section and are collected in a ``config.yaml`` file located in the root directory. Users should copy the provided default configuration (``config.default.yaml``) and amend their own modifications and assumptions in the user-specific configuration file (``config.yaml``); confer installation instructions at :ref:`defaultconfig`. PyPSA-Eur has several configuration options which are documented in this section and are collected in a ``config/config.yaml`` file located in the root directory. Users should copy the provided default configuration (``config/config.default.yaml``) and amend their own modifications and assumptions in the user-specific configuration file (``config/config.yaml``); confer installation instructions at :ref:`defaultconfig`.
.. _toplevel_cf: .. _toplevel_cf:
Top-level configuration Top-level configuration
======================= =======================
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:lines: 5-11,18-19,62,80-90 :lines: 5-11,18-19,62,80-90
@ -37,7 +37,7 @@ investment changes as more ambitious greenhouse-gas emission reduction targets a
The ``run`` section is used for running and storing scenarios with different configurations which are not covered by :ref:`wildcards`. It determines the path at which resources, networks and results are stored. Therefore the user can run different configurations within the same directory. If a run with a non-empty name should use cutouts shared across runs, set ``shared_cutouts`` to `true`. The ``run`` section is used for running and storing scenarios with different configurations which are not covered by :ref:`wildcards`. It determines the path at which resources, networks and results are stored. Therefore the user can run different configurations within the same directory. If a run with a non-empty name should use cutouts shared across runs, set ``shared_cutouts`` to `true`.
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: run: :start-at: run:
:end-before: foresight: :end-before: foresight:
@ -76,7 +76,7 @@ An exemplary dependency graph (starting from the simplification rules) then look
.. image:: img/scenarios.png .. image:: img/scenarios.png
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: scenario: :start-at: scenario:
:end-before: countries: :end-before: countries:
@ -93,7 +93,7 @@ An exemplary dependency graph (starting from the simplification rules) then look
Specifies the temporal range to build an energy system model for as arguments to `pandas.date_range <https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.date_range.html>`_ Specifies the temporal range to build an energy system model for as arguments to `pandas.date_range <https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.date_range.html>`_
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: snapshots: :start-at: snapshots:
:end-before: enable: :end-before: enable:
@ -110,7 +110,7 @@ Specifies the temporal range to build an energy system model for as arguments to
Switches for some rules and optional features. Switches for some rules and optional features.
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: enable: :start-at: enable:
:end-before: co2_budget: :end-before: co2_budget:
@ -125,7 +125,7 @@ Switches for some rules and optional features.
``electricity`` ``electricity``
=============== ===============
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: electricity: :start-at: electricity:
:end-before: atlite: :end-before: atlite:
@ -142,7 +142,7 @@ Switches for some rules and optional features.
Define and specify the ``atlite.Cutout`` used for calculating renewable potentials and time-series. All options except for ``features`` are directly used as `cutout parameters <https://atlite.readthedocs.io/en/latest/ref_api.html#cutout>`_. Define and specify the ``atlite.Cutout`` used for calculating renewable potentials and time-series. All options except for ``features`` are directly used as `cutout parameters <https://atlite.readthedocs.io/en/latest/ref_api.html#cutout>`_.
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: atlite: :start-at: atlite:
:end-before: renewable: :end-before: renewable:
@ -160,7 +160,7 @@ Define and specify the ``atlite.Cutout`` used for calculating renewable potentia
``onwind`` ``onwind``
---------- ----------
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: renewable: :start-at: renewable:
:end-before: offwind-ac: :end-before: offwind-ac:
@ -173,7 +173,7 @@ Define and specify the ``atlite.Cutout`` used for calculating renewable potentia
``offwind-ac`` ``offwind-ac``
-------------- --------------
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: offwind-ac: :start-at: offwind-ac:
:end-before: offwind-dc: :end-before: offwind-dc:
@ -186,7 +186,7 @@ Define and specify the ``atlite.Cutout`` used for calculating renewable potentia
``offwind-dc`` ``offwind-dc``
--------------- ---------------
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: offwind-dc: :start-at: offwind-dc:
:end-before: solar: :end-before: solar:
@ -199,7 +199,7 @@ Define and specify the ``atlite.Cutout`` used for calculating renewable potentia
``solar`` ``solar``
--------------- ---------------
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: solar: :start-at: solar:
:end-before: hydro: :end-before: hydro:
@ -212,7 +212,7 @@ Define and specify the ``atlite.Cutout`` used for calculating renewable potentia
``hydro`` ``hydro``
--------------- ---------------
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: hydro: :start-at: hydro:
:end-before: conventional: :end-before: conventional:
@ -234,7 +234,7 @@ with country specific values. Then, the values are read in and applied to all
generators of the given carrier in the given country. Note that the value(s) generators of the given carrier in the given country. Note that the value(s)
overwrite the existing values. overwrite the existing values.
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: conventional: :start-at: conventional:
:end-before: lines: :end-before: lines:
@ -247,7 +247,7 @@ overwrite the existing values.
``lines`` ``lines``
============= =============
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: lines: :start-at: lines:
:end-before: links: :end-before: links:
@ -262,7 +262,7 @@ overwrite the existing values.
``links`` ``links``
============= =============
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: links: :start-at: links:
:end-before: transformers: :end-before: transformers:
@ -277,7 +277,7 @@ overwrite the existing values.
``transformers`` ``transformers``
================ ================
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: transformers: :start-at: transformers:
:end-before: load: :end-before: load:
@ -292,7 +292,7 @@ overwrite the existing values.
``load`` ``load``
============= =============
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-after: type: :start-after: type:
:end-at: scaling_factor: :end-at: scaling_factor:
@ -307,7 +307,7 @@ overwrite the existing values.
``costs`` ``costs``
============= =============
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: costs: :start-at: costs:
:end-before: clustering: :end-before: clustering:
@ -323,7 +323,7 @@ overwrite the existing values.
``clustering`` ``clustering``
============== ==============
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: clustering: :start-at: clustering:
:end-before: solving: :end-before: solving:
@ -345,7 +345,7 @@ overwrite the existing values.
.. warning:: .. warning::
More comprehensive documentation for this segment will be released soon. More comprehensive documentation for this segment will be released soon.
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: energy: :start-at: energy:
:end-before: biomass: :end-before: biomass:
@ -362,7 +362,7 @@ overwrite the existing values.
.. warning:: .. warning::
More comprehensive documentation for this segment will be released soon. More comprehensive documentation for this segment will be released soon.
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: biomass: :start-at: biomass:
:end-before: solar_thermal: :end-before: solar_thermal:
@ -378,7 +378,7 @@ overwrite the existing values.
.. warning:: .. warning::
More comprehensive documentation for this segment will be released soon. More comprehensive documentation for this segment will be released soon.
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: solar_thermal: :start-at: solar_thermal:
:end-before: existing_capacities: :end-before: existing_capacities:
@ -394,7 +394,7 @@ overwrite the existing values.
.. warning:: .. warning::
More comprehensive documentation for this segment will be released soon. More comprehensive documentation for this segment will be released soon.
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: existing_capacities: :start-at: existing_capacities:
:end-before: sector: :end-before: sector:
@ -410,7 +410,7 @@ overwrite the existing values.
.. warning:: .. warning::
More comprehensive documentation for this segment will be released soon. More comprehensive documentation for this segment will be released soon.
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: sector: :start-at: sector:
:end-before: industry: :end-before: industry:
@ -426,7 +426,7 @@ overwrite the existing values.
.. warning:: .. warning::
More comprehensive documentation for this segment will be released soon. More comprehensive documentation for this segment will be released soon.
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: industry: :start-at: industry:
:end-before: costs: :end-before: costs:
@ -436,7 +436,7 @@ overwrite the existing values.
``solving`` ``solving``
============= =============
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: solving: :start-at: solving:
:end-before: plotting: :end-before: plotting:
@ -459,7 +459,7 @@ overwrite the existing values.
.. warning:: .. warning::
More comprehensive documentation for this segment will be released soon. More comprehensive documentation for this segment will be released soon.
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config/config.default.yaml
:language: yaml :language: yaml
:start-at: plotting: :start-at: plotting:

4
doc/costs.rst
View File

@ -9,7 +9,7 @@ Techno-Economic Assumptions
The database of cost assumptions is retrieved from the repository The database of cost assumptions is retrieved from the repository
`PyPSA/technology-data <https://github.com/pypsa/technology-data>`_ and then `PyPSA/technology-data <https://github.com/pypsa/technology-data>`_ and then
saved to a file ``data/costs_{year}.csv``. The ``config.yaml`` provides options saved to a file ``data/costs_{year}.csv``. The ``config/config.yaml`` provides options
to choose a reference year and use a specific version of the repository. to choose a reference year and use a specific version of the repository.
.. literalinclude:: ../config.default.yaml .. literalinclude:: ../config.default.yaml
@ -48,7 +48,7 @@ Modifying Assumptions
===================== =====================
Some cost assumptions (e.g. marginal cost and capital cost) can be directly Some cost assumptions (e.g. marginal cost and capital cost) can be directly
set in the ``config.yaml`` (cf. Section :ref:`costs_cf` in set in the ``config/config.yaml`` (cf. Section :ref:`costs_cf` in
:ref:`config`). To change cost assumptions in more detail, make a copy of :ref:`config`). To change cost assumptions in more detail, make a copy of
``data/costs_{year}.csv`` and reference the new cost file in the ``Snakefile``: ``data/costs_{year}.csv`` and reference the new cost file in the ``Snakefile``:

12
doc/foresight.rst
View File

@ -28,7 +28,7 @@ It does not affect the year for cost and technology assumptions, which is set se
costs: costs:
year: 2030 year: 2030
For running overnight scenarios, use in the ``config.yaml``: For running overnight scenarios, use in the ``config/config.yaml``:
.. code:: yaml .. code:: yaml
@ -44,7 +44,7 @@ Perfect foresight scenarios
Perfect foresight is currently under development and not yet implemented. Perfect foresight is currently under development and not yet implemented.
For running perfect foresight scenarios, in future versions you will be able to For running perfect foresight scenarios, in future versions you will be able to
set in the ``config.yaml``: set in the ``config/config.yaml``:
.. code:: yaml .. code:: yaml
@ -94,13 +94,13 @@ evolve with the myopic approach:
Configuration Configuration
-------------- --------------
For running myopic foresight transition scenarios, set in ``config.yaml``: For running myopic foresight transition scenarios, set in ``config/config.yaml``:
.. code:: yaml .. code:: yaml
foresight: myopic foresight: myopic
The following options included in the config.yaml file are relevant for the The following options included in the ``config/config.yaml`` file are relevant for the
myopic code. myopic code.
The ``{planning_horizons}`` wildcard indicates the year in which the network is The ``{planning_horizons}`` wildcard indicates the year in which the network is
@ -163,7 +163,7 @@ Options
The total carbon budget for the entire transition path can be indicated in the The total carbon budget for the entire transition path can be indicated in the
`sector_opts `sector_opts
<https://github.com/PyPSA/pypsa-eur-sec/blob/f13902510010b734c510c38c4cae99356f683058/config.default.yaml#L25>`_ <https://github.com/PyPSA/pypsa-eur-sec/blob/f13902510010b734c510c38c4cae99356f683058/config.default.yaml#L25>`_
in ``config.yaml``. The carbon budget can be split among the in ``config/config.yaml``. The carbon budget can be split among the
``planning_horizons`` following an exponential or beta decay. E.g. ``'cb40ex0'`` ``planning_horizons`` following an exponential or beta decay. E.g. ``'cb40ex0'``
splits a carbon budget equal to 40 Gt :math:`_{CO_2}` following an exponential splits a carbon budget equal to 40 Gt :math:`_{CO_2}` following an exponential
decay whose initial linear growth rate r is zero. They can also follow some decay whose initial linear growth rate r is zero. They can also follow some
@ -218,7 +218,7 @@ add_brownfield for the remaining planning_horizons.
``results/run_name/networks/prenetworks-brownfield``. ``results/run_name/networks/prenetworks-brownfield``.
Steps 2 and 3 are solved recursively for all the planning_horizons included in Steps 2 and 3 are solved recursively for all the planning_horizons included in
``config.yaml``. ``config/config.yaml``.
Rule overview Rule overview
-------------- --------------

14
doc/installation.rst
View File

@ -119,19 +119,19 @@ Handling Configuration Files
============================ ============================
PyPSA-Eur has several configuration options that must be specified in a PyPSA-Eur has several configuration options that must be specified in a
``config.yaml`` file located in the root directory. An example configuration ``config/config.yaml`` file located in the root directory. An example configuration
``config.default.yaml`` is maintained in the repository, which will be used to ``config/config.default.yaml`` is maintained in the repository, which will be used to
automatically create your customisable ``config.yaml`` on first use. More automatically create your customisable ``config/config.yaml`` on first use. More
details on the configuration options are in :ref:`config`. details on the configuration options are in :ref:`config`.
You can also use ``snakemake`` to specify another file, e.g. You can also use ``snakemake`` to specify another file, e.g.
``config.mymodifications.yaml``, to update the settings of the ``config.yaml``. ``config/config.mymodifications.yaml``, to update the settings of the ``config/config.yaml``.
.. code:: bash .. code:: bash
.../pypsa-eur % snakemake -call --configfile config.mymodifications.yaml .../pypsa-eur % snakemake -call --configfile config/config.mymodifications.yaml
.. warning:: .. warning::
Users are advised to regularly check their own ``config.yaml`` against changes Users are advised to regularly check their own ``config/config.yaml`` against changes
in the ``config.default.yaml`` when pulling a new version from the remote in the ``config/config.default.yaml`` when pulling a new version from the remote
repository. repository.

2
doc/introduction.rst
View File

@ -74,7 +74,7 @@ what data to retrieve and what files to produce. Details are explained in
:ref:`wildcards` and :ref:`scenario`. :ref:`wildcards` and :ref:`scenario`.
The model also has several further configuration options collected in the The model also has several further configuration options collected in the
``config.yaml`` file located in the root directory, which that are not part of ``config/config.yaml`` file located in the root directory, which that are not part of
the scenarios. Options are explained in :ref:`config`. the scenarios. Options are explained in :ref:`config`.
Folder Structure Folder Structure

5
doc/release_notes.rst
View File

@ -10,6 +10,9 @@ Release Notes
Upcoming Release Upcoming Release
================ ================
* **Important:** The configuration files are now located in the ``config`` directory. This counts for ``config.default.yaml``, ``config.yaml`` as well as the test configuration files which are now located in ``config/test``. Config files that are still in the root directory will be ignored.
* Bugfix: Correct typo in the CPLEX solver configuration in ``config.default.yaml``. * Bugfix: Correct typo in the CPLEX solver configuration in ``config.default.yaml``.
* Renamed script file from PyPSA-EUR ``build_load_data`` to ``build_electricity_demand``. * Renamed script file from PyPSA-EUR ``build_load_data`` to ``build_electricity_demand``.
@ -339,7 +342,7 @@ PyPSA-Eur 0.5.0 (27th July 2022)
* Network building is made deterministic by supplying a fixed random state to * Network building is made deterministic by supplying a fixed random state to
network clustering routines. network clustering routines.
* Clustering strategies for generator and bus attributes can now be specified directly in the ``config.yaml``. * Clustering strategies for generator and bus attributes can now be specified directly in the ``config/config.yaml``.
* Iterative solving with impedance updates is skipped if there are no expandable * Iterative solving with impedance updates is skipped if there are no expandable
lines. lines.

6
doc/retrieve.rst
View File

@ -42,7 +42,7 @@ The :ref:`tutorial` uses a smaller cutout than required for the full model (30 M
build_cutout: build_cutout:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`toplevel_cf` :ref:`toplevel_cf`
**Outputs** **Outputs**
@ -69,7 +69,7 @@ This rule, as a substitute for :mod:`build_natura_raster`, downloads an already
build_natura_raster: build_natura_raster:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`toplevel_cf` :ref:`toplevel_cf`
**Outputs** **Outputs**
@ -111,7 +111,7 @@ This rule downloads techno-economic assumptions from the `technology-data reposi
version: version:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`costs_cf` :ref:`costs_cf`
**Outputs** **Outputs**

4
doc/spatial_resolution.rst
View File

@ -11,7 +11,7 @@ Spatial resolution
The default nodal resolution of the model follows the electricity generation and transmission model `PyPSA-Eur <https://github.com/PyPSA/pypsa-eur>`_, which clusters down the electricity transmission substations in each European country based on the k-means algorithm (See `cluster_network <https://pypsa-eur.readthedocs.io/en/latest/simplification/cluster_network.html#rule-cluster-network>`_ for a complete explanation). This gives nodes which correspond to major load and generation centres (typically cities). The default nodal resolution of the model follows the electricity generation and transmission model `PyPSA-Eur <https://github.com/PyPSA/pypsa-eur>`_, which clusters down the electricity transmission substations in each European country based on the k-means algorithm (See `cluster_network <https://pypsa-eur.readthedocs.io/en/latest/simplification/cluster_network.html#rule-cluster-network>`_ for a complete explanation). This gives nodes which correspond to major load and generation centres (typically cities).
The total number of nodes for Europe is set in the ``config.yaml`` file under ``clusters``. The number of nodes can vary between 37, the number of independent countries / synchronous areas, and several hundred. With 200-300 nodes the model needs 100-150 GB RAM to solve with a commercial solver like Gurobi. The total number of nodes for Europe is set in the ``config/config.yaml`` file under ``clusters``. The number of nodes can vary between 37, the number of independent countries / synchronous areas, and several hundred. With 200-300 nodes the model needs 100-150 GB RAM to solve with a commercial solver like Gurobi.
Exemplary unsolved network clustered to 512 nodes: Exemplary unsolved network clustered to 512 nodes:
@ -21,7 +21,7 @@ Exemplary unsolved network clustered to 37 nodes:
.. image:: ../graphics/elec_s_37.png .. image:: ../graphics/elec_s_37.png
The total number of nodes for Europe is set in the config.yaml file under `clusters <https://github.com/PyPSA/pypsa-eur-sec/blob/3daff49c9999ba7ca7534df4e587e1d516044fc3/config.default.yaml#L20>`_. The number of nodes can vary between 37, the number of independent countries/synchronous areas, and several hundred. With 200-300 nodes, the model needs 100-150 GB RAM to solve with a commercial solver like Gurobi. The total number of nodes for Europe is set in the ``config/config.yaml`` file under `clusters <https://github.com/PyPSA/pypsa-eur-sec/blob/3daff49c9999ba7ca7534df4e587e1d516044fc3/config.default.yaml#L20>`_. The number of nodes can vary between 37, the number of independent countries/synchronous areas, and several hundred. With 200-300 nodes, the model needs 100-150 GB RAM to solve with a commercial solver like Gurobi.
Not all of the sectors are at the full nodal resolution, and some demand for some sectors is distributed to nodes using heuristics that need to be corrected. Some networks are copper-plated to reduce computational times. Not all of the sectors are at the full nodal resolution, and some demand for some sectors is distributed to nodes using heuristics that need to be corrected. Some networks are copper-plated to reduce computational times.
Here are some examples of how spatial resolution is set for different sectors in PyPSA-Eur-Sec: Here are some examples of how spatial resolution is set for different sectors in PyPSA-Eur-Sec:

2
doc/supply_demand.rst
View File

@ -189,7 +189,7 @@ higher costs and higher efficiency gains. They are added by step-wise
linearisation in form of two additional generations in linearisation in form of two additional generations in
the `prepare_sector_network.py <https://github.com/PyPSA/pypsa-eur-sec/blob/3daff49c9999ba7ca7534df4e587e1d516044fc3/scripts/prepare_sector_network.py#L1600>`_ script. the `prepare_sector_network.py <https://github.com/PyPSA/pypsa-eur-sec/blob/3daff49c9999ba7ca7534df4e587e1d516044fc3/scripts/prepare_sector_network.py#L1600>`_ script.
Settings in the config.yaml concerning the endogenously optimisation of building Settings in the ``config/config.yaml`` concerning the endogenously optimisation of building
renovation include `cost factor <https://github.com/PyPSA/pypsa-eur-sec/blob/3daff49c9999ba7ca7534df4e587e1d516044fc3/config.default.yaml#L223>`_, `interest rate <https://github.com/PyPSA/pypsa-eur-sec/blob/3daff49c9999ba7ca7534df4e587e1d516044fc3/config.default.yaml#L224>`_, `annualised cost <https://github.com/PyPSA/pypsa-eur-sec/blob/3daff49c9999ba7ca7534df4e587e1d516044fc3/config.default.yaml#L225>`_, `tax weighting <https://github.com/PyPSA/pypsa-eur-sec/blob/3daff49c9999ba7ca7534df4e587e1d516044fc3/config.default.yaml#L226>`_, and `construction index <https://github.com/PyPSA/pypsa-eur-sec/blob/3daff49c9999ba7ca7534df4e587e1d516044fc3/config.default.yaml#L227>`_. renovation include `cost factor <https://github.com/PyPSA/pypsa-eur-sec/blob/3daff49c9999ba7ca7534df4e587e1d516044fc3/config.default.yaml#L223>`_, `interest rate <https://github.com/PyPSA/pypsa-eur-sec/blob/3daff49c9999ba7ca7534df4e587e1d516044fc3/config.default.yaml#L224>`_, `annualised cost <https://github.com/PyPSA/pypsa-eur-sec/blob/3daff49c9999ba7ca7534df4e587e1d516044fc3/config.default.yaml#L225>`_, `tax weighting <https://github.com/PyPSA/pypsa-eur-sec/blob/3daff49c9999ba7ca7534df4e587e1d516044fc3/config.default.yaml#L226>`_, and `construction index <https://github.com/PyPSA/pypsa-eur-sec/blob/3daff49c9999ba7ca7534df4e587e1d516044fc3/config.default.yaml#L227>`_.
Further information are given in the study by Zeyen et al. : `Mitigating heat demand peaks in buildings in a highly renewable European energy system, (2021) <https://arxiv.org/abs/2012.01831>`_. Further information are given in the study by Zeyen et al. : `Mitigating heat demand peaks in buildings in a highly renewable European energy system, (2021) <https://arxiv.org/abs/2012.01831>`_.

6
doc/tutorial.rst
View File

@ -26,7 +26,7 @@ local machine. The tutorial will cover examples on how to configure and
customise the PyPSA-Eur model and run the ``snakemake`` workflow step by step customise the PyPSA-Eur model and run the ``snakemake`` workflow step by step
from network creation to the solved network. The configuration for the tutorial from network creation to the solved network. The configuration for the tutorial
is located at ``test/config.electricity.yaml``. It includes parts deviating from is located at ``test/config.electricity.yaml``. It includes parts deviating from
the default config file ``config.default.yaml``. To run the tutorial with this the default config file ``config/config.default.yaml``. To run the tutorial with this
configuration, execute configuration, execute
.. code:: bash .. code:: bash
@ -282,7 +282,7 @@ For example, you can explore the evolution of the PyPSA networks by running
#. ``snakemake resources/networks/elec_s_6.nc -call --configfile test/config.electricity.yaml`` #. ``snakemake resources/networks/elec_s_6.nc -call --configfile test/config.electricity.yaml``
#. ``snakemake resources/networks/elec_s_6_ec_lcopt_Co2L-24H.nc -call --configfile test/config.electricity.yaml`` #. ``snakemake resources/networks/elec_s_6_ec_lcopt_Co2L-24H.nc -call --configfile test/config.electricity.yaml``
To run all combinations of wildcard values provided in the ``config.yaml`` under ``scenario:``, To run all combinations of wildcard values provided in the ``config/config.yaml`` under ``scenario:``,
you can use the collection rule ``solve_elec_networks``. you can use the collection rule ``solve_elec_networks``.
.. code:: bash .. code:: bash
@ -290,7 +290,7 @@ you can use the collection rule ``solve_elec_networks``.
snakemake -call solve_elec_networks --configfile test/config.electricity.yaml snakemake -call solve_elec_networks --configfile test/config.electricity.yaml
If you now feel confident and want to tackle runs with larger temporal and If you now feel confident and want to tackle runs with larger temporal and
spatial scope, clean-up the repository and after modifying the ``config.yaml`` file spatial scope, clean-up the repository and after modifying the ``config/config.yaml`` file
target the collection rule ``solve_elec_networks`` again without providing the test target the collection rule ``solve_elec_networks`` again without providing the test
configuration file. configuration file.

4
doc/tutorial_sector.rst
View File

@ -29,7 +29,7 @@ Overnight Scenarios
Configuration Configuration
------------- -------------
The default configuration file (``config.default.yaml``) is set up for running The default configuration file (``config/config.default.yaml``) is set up for running
overnight scenarios. Running a sector-coupled model unlocks many further overnight scenarios. Running a sector-coupled model unlocks many further
configuration options. In the example below, we say that the gas network should configuration options. In the example below, we say that the gas network should
be added and spatially resolved. We also say that the existing gas network may be added and spatially resolved. We also say that the existing gas network may
@ -513,7 +513,7 @@ Scaling-Up
========== ==========
If you now feel confident and want to tackle runs with larger temporal, technological and If you now feel confident and want to tackle runs with larger temporal, technological and
spatial scope, clean-up the repository and after modifying the ``config.yaml`` file spatial scope, clean-up the repository and after modifying the ``config/config.yaml`` file
target the collection rule ``all`` again without providing the test target the collection rule ``all`` again without providing the test
configuration file. configuration file.

4
rules/postprocess.smk
View File

@ -37,7 +37,7 @@ rule copy_config:
params: params:
RDIR=RDIR, RDIR=RDIR,
output: output:
RESULTS + "configs/config.yaml", RESULTS + "config/config.yaml",
threads: 1 threads: 1
resources: resources:
mem_mb=1000, mem_mb=1000,
@ -51,7 +51,7 @@ rule copy_config:
rule copy_conda_env: rule copy_conda_env:
output: output:
RESULTS + "configs/environment.yaml", RESULTS + "config/environment.yaml",
threads: 1 threads: 1
resources: resources:
mem_mb=500, mem_mb=500,

2
rules/solve_myopic.smk
View File

@ -79,7 +79,7 @@ rule solve_sector_network_myopic:
network=RESULTS network=RESULTS
+ "prenetworks-brownfield/elec_s{simpl}_{clusters}_l{ll}_{opts}_{sector_opts}_{planning_horizons}.nc", + "prenetworks-brownfield/elec_s{simpl}_{clusters}_l{ll}_{opts}_{sector_opts}_{planning_horizons}.nc",
costs="data/costs_{planning_horizons}.csv", costs="data/costs_{planning_horizons}.csv",
config=RESULTS + "configs/config.yaml", config=RESULTS + "config/config.yaml",
output: output:
RESULTS RESULTS
+ "postnetworks/elec_s{simpl}_{clusters}_l{ll}_{opts}_{sector_opts}_{planning_horizons}.nc", + "postnetworks/elec_s{simpl}_{clusters}_l{ll}_{opts}_{sector_opts}_{planning_horizons}.nc",

4
rules/solve_overnight.smk
View File

@ -9,8 +9,8 @@ rule solve_sector_network:
network=RESULTS network=RESULTS
+ "prenetworks/elec_s{simpl}_{clusters}_l{ll}_{opts}_{sector_opts}_{planning_horizons}.nc", + "prenetworks/elec_s{simpl}_{clusters}_l{ll}_{opts}_{sector_opts}_{planning_horizons}.nc",
costs="data/costs_{}.csv".format(config["costs"]["year"]), costs="data/costs_{}.csv".format(config["costs"]["year"]),
config=RESULTS + "configs/config.yaml", config=RESULTS + "config/config.yaml",
#env=RDIR + 'configs/environment.yaml', #env=RDIR + 'config/environment.yaml',
output: output:
RESULTS RESULTS
+ "postnetworks/elec_s{simpl}_{clusters}_l{ll}_{opts}_{sector_opts}_{planning_horizons}.nc", + "postnetworks/elec_s{simpl}_{clusters}_l{ll}_{opts}_{sector_opts}_{planning_horizons}.nc",

2
scripts/_helpers.py
View File

@ -83,7 +83,7 @@ def load_network(import_name=None, custom_components=None):
custom_components : dict custom_components : dict
Dictionary listing custom components. Dictionary listing custom components.
For using ``snakemake.config['override_components']`` For using ``snakemake.config['override_components']``
in ``config.yaml`` define: in ``config/config.yaml`` define:
.. code:: yaml .. code:: yaml

2
scripts/add_electricity.py
View File

@ -41,7 +41,7 @@ Relevant Settings
length_factor: length_factor:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at :ref:`costs_cf`, Documentation of the configuration file ``config/config.yaml`` at :ref:`costs_cf`,
:ref:`electricity_cf`, :ref:`load_cf`, :ref:`renewable_cf`, :ref:`lines_cf` :ref:`electricity_cf`, :ref:`load_cf`, :ref:`renewable_cf`, :ref:`lines_cf`
Inputs Inputs

4
scripts/add_extra_components.py
View File

@ -27,7 +27,7 @@ Relevant Settings
Store: Store:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at :ref:`costs_cf`, Documentation of the configuration file ``config/config.yaml`` at :ref:`costs_cf`,
:ref:`electricity_cf` :ref:`electricity_cf`
Inputs Inputs
@ -44,7 +44,7 @@ Outputs
Description Description
----------- -----------
The rule :mod:`add_extra_components` attaches additional extendable components to the clustered and simplified network. These can be configured in the ``config.yaml`` at ``electricity: extendable_carriers:``. It processes ``networks/elec_s{simpl}_{clusters}.nc`` to build ``networks/elec_s{simpl}_{clusters}_ec.nc``, which in contrast to the former (depending on the configuration) contain with **zero** initial capacity The rule :mod:`add_extra_components` attaches additional extendable components to the clustered and simplified network. These can be configured in the ``config/config.yaml`` at ``electricity: extendable_carriers:``. It processes ``networks/elec_s{simpl}_{clusters}.nc`` to build ``networks/elec_s{simpl}_{clusters}_ec.nc``, which in contrast to the former (depending on the configuration) contain with **zero** initial capacity
- ``StorageUnits`` of carrier 'H2' and/or 'battery'. If this option is chosen, every bus is given an extendable ``StorageUnit`` of the corresponding carrier. The energy and power capacities are linked through a parameter that specifies the energy capacity as maximum hours at full dispatch power and is configured in ``electricity: max_hours:``. This linkage leads to one investment variable per storage unit. The default ``max_hours`` lead to long-term hydrogen and short-term battery storage units. - ``StorageUnits`` of carrier 'H2' and/or 'battery'. If this option is chosen, every bus is given an extendable ``StorageUnit`` of the corresponding carrier. The energy and power capacities are linked through a parameter that specifies the energy capacity as maximum hours at full dispatch power and is configured in ``electricity: max_hours:``. This linkage leads to one investment variable per storage unit. The default ``max_hours`` lead to long-term hydrogen and short-term battery storage units.

2
scripts/base_network.py
View File

@ -38,7 +38,7 @@ Relevant Settings
type: type:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`snapshots_cf`, :ref:`toplevel_cf`, :ref:`electricity_cf`, :ref:`load_cf`, :ref:`snapshots_cf`, :ref:`toplevel_cf`, :ref:`electricity_cf`, :ref:`load_cf`,
:ref:`lines_cf`, :ref:`links_cf`, :ref:`transformers_cf` :ref:`lines_cf`, :ref:`links_cf`, :ref:`transformers_cf`

2
scripts/build_bus_regions.py
View File

@ -14,7 +14,7 @@ Relevant Settings
countries: countries:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`toplevel_cf` :ref:`toplevel_cf`
Inputs Inputs

2
scripts/build_cutout.py
View File

@ -25,7 +25,7 @@ Relevant Settings
{cutout}: {cutout}:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`atlite_cf` :ref:`atlite_cf`
Inputs Inputs

2
scripts/build_electricity_demand.py
View File

@ -25,7 +25,7 @@ Relevant Settings
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`load_cf` :ref:`load_cf`
Inputs Inputs

2
scripts/build_hydro_profile.py
View File

@ -20,7 +20,7 @@ Relevant Settings
clip_min_inflow: clip_min_inflow:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`toplevel_cf`, :ref:`renewable_cf` :ref:`toplevel_cf`, :ref:`renewable_cf`
Inputs Inputs

2
scripts/build_natura_raster.py
View File

@ -18,7 +18,7 @@ Relevant Settings
cutout: cutout:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`renewable_cf` :ref:`renewable_cf`
Inputs Inputs

2
scripts/build_powerplants.py
View File

@ -21,7 +21,7 @@ Relevant Settings
custom_powerplants: custom_powerplants:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`electricity` :ref:`electricity`
Inputs Inputs

2
scripts/build_renewable_profiles.py
View File

@ -42,7 +42,7 @@ Relevant settings
resource: resource:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`snapshots_cf`, :ref:`atlite_cf`, :ref:`renewable_cf` :ref:`snapshots_cf`, :ref:`atlite_cf`, :ref:`renewable_cf`
Inputs Inputs

2
scripts/build_shapes.py
View File

@ -15,7 +15,7 @@ Relevant Settings
countries: countries:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`toplevel_cf` :ref:`toplevel_cf`
Inputs Inputs

2
scripts/build_ship_raster.py
View File

@ -19,7 +19,7 @@ Relevant Settings
cutout: cutout:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`renewable_cf` :ref:`renewable_cf`
Inputs Inputs

2
scripts/cluster_network.py
View File

@ -27,7 +27,7 @@ Relevant Settings
length_factor: length_factor:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`toplevel_cf`, :ref:`renewable_cf`, :ref:`solving_cf`, :ref:`lines_cf` :ref:`toplevel_cf`, :ref:`renewable_cf`, :ref:`solving_cf`, :ref:`lines_cf`
Inputs Inputs

4
scripts/copy_config.py
View File

@ -12,7 +12,7 @@ from shutil import copy
import yaml import yaml
files = { files = {
"config.yaml": "config.yaml", "config/config.yaml": "config.yaml",
"Snakefile": "Snakefile", "Snakefile": "Snakefile",
"scripts/solve_network.py": "solve_network.py", "scripts/solve_network.py": "solve_network.py",
"scripts/prepare_sector_network.py": "prepare_sector_network.py", "scripts/prepare_sector_network.py": "prepare_sector_network.py",
@ -24,7 +24,7 @@ if __name__ == "__main__":
snakemake = mock_snakemake("copy_config") snakemake = mock_snakemake("copy_config")
basepath = Path(f"results/{snakemake.params.RDIR}configs/") basepath = Path(f"results/{snakemake.params.RDIR}config/")
for f, name in files.items(): for f, name in files.items():
copy(f, basepath / name) copy(f, basepath / name)

2
scripts/prepare_links_p_nom.py
View File

@ -18,7 +18,7 @@ Relevant Settings
prepare_links_p_nom: prepare_links_p_nom:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`toplevel_cf` :ref:`toplevel_cf`
Inputs Inputs

2
scripts/prepare_network.py
View File

@ -34,7 +34,7 @@ Relevant Settings
max_hours: max_hours:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`costs_cf`, :ref:`electricity_cf` :ref:`costs_cf`, :ref:`electricity_cf`
Inputs Inputs

2
scripts/retrieve_databundle.py
View File

@ -23,7 +23,7 @@ The :ref:`tutorial` uses a smaller `data bundle <https://zenodo.org/record/35179
tutorial: tutorial:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`toplevel_cf` :ref:`toplevel_cf`
**Outputs** **Outputs**

2
scripts/simplify_network.py
View File

@ -40,7 +40,7 @@ Relevant Settings
name: name:
.. seealso:: .. seealso::
Documentation of the configuration file ``config.yaml`` at Documentation of the configuration file ``config/config.yaml`` at
:ref:`costs_cf`, :ref:`electricity_cf`, :ref:`renewable_cf`, :ref:`costs_cf`, :ref:`electricity_cf`, :ref:`renewable_cf`,
:ref:`lines_cf`, :ref:`links_cf`, :ref:`solving_cf` :ref:`lines_cf`, :ref:`links_cf`, :ref:`solving_cf`