013b705ee4
* Cluster first: build renewable profiles and add all assets after clustering * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * correction: pass landfall_lengths through functions * assign landfall_lenghts correctly * remove parameter add_land_use_constraint * fix network_dict * calculate distance to shoreline, remove underwater_fraction * adjust simplification parameter to exclude Crete from offshore wind connections * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove unused geth2015 hydro capacities * removing remaining traces of {simpl} wildcard * add release notes and update workflow graphics * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: lisazeyen <lisa.zeyen@web.de>
4.5 KiB
Introduction
Note
Find the introductory slides here.
Warning
The video only introduces the electricity-only part of PyPSA-Eur.
Workflow
The generation of the model is controlled by the open workflow management system Snakemake. In a nutshell, the Snakefile declares for each script in the scripts directory a rule which describes which files the scripts consume and produce (their corresponding input and output files). The snakemake tool then runs the scripts in the correct order according to the rules' input and output dependencies. Moreover, snakemake will track what parts of the workflow have to be regenerated when files or scripts were modified.
For instance, an invocation to
System Message: WARNING/2 (<stdin>, line 36)
Cannot analyze code. Pygments package not found.
.. code:: bash
.../pypsa-eur % snakemake -call results/networks/base_s_128_elec_lvopt_.nc
follows this dependency graph
to solve an electricity system model.
The blocks represent the individual rules which are required to create the file referenced in the command above. The arrows indicate the outputs from preceding rules which another rule takes as input data.
Note
The dependency graph was generated using snakemake --dag results/networks/base_s_128_elec_lvopt_.nc -F | sed -n "/digraph/,/}/p" | dot -Tpng -o doc/img/intro-workflow.png
For the use of snakemake, it makes sense to familiarize yourself quickly with the basic tutorial and then read carefully through the documentation of the command line interface, noting the arguments -j, -c, -f, -F, -n, -r, --dag and -t in particular.
Scenarios, Configuration and Modification
It is easy to run PyPSA-Eur for multiple scenarios using the wildcards feature of snakemake. Wildcards allow to generalise a rule to produce all files that follow a regular expression pattern, which defines a particular scenario. One can think of a wildcard as a parameter that shows up in the input/output file names and thereby determines which rules to run, what data to retrieve and what files to produce. Details are explained in :ref:`wildcards` and :ref:`scenario`.
System Message: ERROR/3 (<stdin>, line 66); backlink
Unknown interpreted text role "ref".System Message: ERROR/3 (<stdin>, line 66); backlink
Unknown interpreted text role "ref".The model also has several further configuration options collected in the config/config.default.yaml file located in the root directory, which that are not part of the scenarios. Options are explained in :ref:`config`.
System Message: ERROR/3 (<stdin>, line 76); backlink
Unknown interpreted text role "ref".Folder Structure
- scripts: Includes all the Python scripts executed by the snakemake rules.
- rules: Includes all the snakemake rules loaded in the Snakefile.
- envs: Includes all the conda environment specifications to run the workflow.
- data: Includes input data that is not produced by any snakemake rule.
- cutouts: Stores raw weather data cutouts from atlite.
- resources: Stores intermediate results of the workflow which can be picked up again by subsequent rules.
- results: Stores the solved PyPSA network data, summary files and plots.
- logs: Stores log files.
- benchmarks: Stores snakemake benchmarks.
- doc: Includes the documentation of PyPSA-Eur.
- graphics: Includes some graphics for the documentation of PyPSA-Eur.
System Requirements
Building the model with the scripts in this repository runs on a regular computer. But optimising for investment and operation decisions across many scenarios requires a strong interior-point solver like Gurobi or CPLEX with more memory. Open-source solvers like HiGHS <https://highs.dev> can also be used for smaller problems.