transfer descriptions from readme

doc/configtables/opts.csv

@@ -0,0 +1,6 @@
+Trigger, Description, Definition, Status
+``Co2L``, Add an overall Co2 emissions limit configured in ``electricity: co2limit``, ``prepare_network``: `add_co2limit() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L19>`_ and its `caller <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L154>`_, In active use
+``nH``; i.e. ``2H``-``6H``, Resample the time-resolution by averaging over every ``n`` snapshots, ``prepare_network``: `average_every_nhours() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L110>`_ and its `caller <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L146>`_), In active use
+``Ep``, Add cost for a Co2 price configured in ``costs: emission_prices: co2`` to ``marginal_cost`` of generators, ``prepare_network``: `add_emission_prices() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L24>`_ and its `caller <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L158>`_, Broken
+``BAU``, Add a per-``carrier`` minimal overall capacity; i.e. at least ``40GW`` of ``OCGT`` in Europe; configured in ``electricity: BAU_mincapacities``, ``solve_network``: `add_opts_constraints() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/solve_network.py#L66>`_, Untested
+``SAFE``, Add a capacity reserve margin of a certain fraction above the peak demand to which renewable generators and storage do *not* contribute. Ignores network., ``solve_network`` `add_opts_constraints() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/solve_network.py#L73>`_, Untested

doc/configtables/plotting.csv

@@ -1,15 +1,15 @@
 ,Unit,Values,Description
 map,,,
--- figsize,--,"[width, height]; e.g. [7, 7]","Figure size in inches."
+-- figsize,--,"[width, height]; e.g. [7,7]","Figure size in inches."
--- boundaries,--,E,TODO
+-- boundaries,°,"[x1,x2,y1,y2]","Boundaries of the map plots in degrees latitude (y) and longitude (x)"
--- p_nom,--,E,TODO
+-- p_nom,,,
--- -- bus_size_factor,--,E,TODO
+-- -- bus_size_factor,--,float,"Factor by which values determining bus sizes are scaled to fit well in the plot."
--- -- linewidth_factor,--,E,TODO
+-- -- linewidth_factor,--,float,"Factor by which values determining bus sizes are scaled to fit well in the plot."
-costs_max,--,E,TODO
+costs_max,bn Euro,float,"Upper y-axis limit in cost bar plots."
-costs_threshold,--,E,TODO
+costs_threshold,bn Euro,float,"Threshold below which technologies will not be shown in cost bar plots."
-energy_max,--,E,TODO
+energy_max,TWh,float,"Upper y-axis limit in energy bar plots."
-energy_min,--,E,TODO
+energy_min,TWh,float,"Lower y-axis limit in energy bar plots."
-energy_threshold,--,E,TODO
+energy_threshold,TWh,float,"Threshold below which technologies will not be shown in energy bar plots."
 vre_techs,--,E,TODO
 conv_techs,--,E,TODO
 storage_techs,--,E,TODO

doc/configuration.rst

@@ -2,7 +2,7 @@
 Configuration
 ##########################################
 
-by changing the default configuration in ``config.yaml``.
+PyPSA-Eur has several configuration options which are documented in this section and are collected in a ``config.yaml`` file located in the root directory. Users can amend their own modifications and assumptions by changing the default configuration provided in the configuration file (``config.yaml``).
 
 Top-level configuration
 =======================
@@ -29,7 +29,7 @@ Wildcards and ``scenario``
    :widths: 25,7,22,30
    :file: configtables/scenario.csv
    
-
+.. image:: img/scenarios.png
 
 ``snapshots``
 =============
@@ -156,6 +156,9 @@ Arguments to `pandas.date_range <https://pandas.pydata.org/pandas-docs/stable/re
    :widths: 25,7,22,30
    :file: configtables/costs.csv
 
+.. note::
+    To change cost assumptions in more detail (i.e. other than ``marginal_cost`` and ``capital_cost``), consider modifying cost assumptions directly in ``data/costs.csv`` as this is not yet supported through the config file.
+
 ``solving``
 =============
 

doc/index.rst

@@ -23,6 +23,7 @@ Documentation
 
 * :doc:`introduction`
 * :doc:`installation`
+* :doc:`wildcards`
 * :doc:`configuration`
 
 .. toctree::
@@ -32,6 +33,7 @@ Documentation
 
    introduction
    installation
+   wildcards
    configuration
 
 **Rules Overview**
@@ -100,4 +102,5 @@ Licence
 =======
 
 The code in PyPSA-Eur is released as free software under the `GPLv3
-<http://www.gnu.org/licenses/gpl-3.0.en.html>`_.
+<http://www.gnu.org/licenses/gpl-3.0.en.html>`_, see
+`LICENSE <https://github.com/PyPSA/pypsa-eur/blob/master/LICENSE.txt>`_.

doc/installation.rst

@@ -82,3 +82,10 @@ Not all data dependencies are shipped with the git repository, since git is not
 .. code:: bash
 
     .../pypsa-eur % rm -rf data/pypsa-eur-data-bundle.tar.xz pypsa-eur-cutouts.tar.xz
+
+Install a Solver
+================
+
+Default choice for the solver is Gurobi (freely available under academic license) or CPLEX. If you want to go fully opensource the CBC solver (https://projects.coin-or.org/Cbc) can be used. To install CBC run ::
+
+    conda install -c conda-forge coincbc

doc/introduction.rst

@@ -4,11 +4,40 @@
  Introduction
 ##########################################
 
-Section A
+Workflow
 =========
 
-Section B
+The generation of the model is controlled by the workflow management system
-=========
+`Snakemake <https://snakemake.bitbucket.io/>`_.
+In a nutshell, the ``Snakefile`` declares for each python script in the ``scripts`` directory a rule which describes which files the scripts consume and produce (their corresponding input and output files).
+The ``snakemake`` tool then runs the scripts in the correct order according to the rules' input/output dependencies.
+Moreover, it is able to track, what parts of the workflow have to be regenerated, when a data file or a script is modified/updated.
+
+For instance an invocation to
+
+.. code:: bash
+
+    .../pypsa-eur % snakemake networks/elec_s_128.nc
+
+follows this dependency graph:
+
+.. image:: img/workflow.png
+
+The **blocks** represent the individual rules which are required to create the file ``networks/elec_s_128.nc``. The **arrows** indicate the outputs from preceding rules which a particular rule takes as input data.
+
+.. note::
+    The dependency graph shown above was generated using
+    ``snakemake --dag networks/elec_s_128.nc | dot -Tpng > workflow.png``
+
+For the use of ``snakemake``, it makes sense to familiarize oneself quickly with its `basic tutorial <https://snakemake.readthedocs.io/en/stable/tutorial/basics.html>`_ and then read carefully through the section `Executing Snakemake <https://snakemake.readthedocs.io/en/stable/executable.html>`_, noting the arguments ``-n``, ``-r``, but also ``--dag``, ``-R`` and ``-t``.
+
+Modification
+============
+
+TODO: wildcards modification
+
+The model has several configuration options collected in the ``config.yaml`` file
+located in the root directory.
 
 System Requirements
 ===================

doc/preparation.rst

@@ -2,6 +2,31 @@
 Preparing Networks
 ##########################################
 
+In detail this means it has to run the independent scripts,
+- `build_shapes` to generate GeoJSON files with country, exclusive economic zones and nuts3 shapes
+- `build_cutout` to prepare smaller weather data portions from ERA5 for cutout `europe-2013-era5` and SARAH for cutout `europe-2013-sarah`.
+
+With these and the externally extracted `ENTSO-E online map topology`, it can build the PyPSA basis model
+- `base_network` stored at `networks/base.nc` with all `buses`, HVAC `lines` and HVDC `links`, and in
+- `build_bus_regions` determine the Voronoi cell of each substation.
+
+Then it hands these over to the scripts for generating renewable and hydro feedin data,
+- `build_hydro_profile` for the hourly hydro energy availability,
+- `build_renewable_potentials` for the landuse/natura2000 constrained installation potentials for PV and wind,
+- `build_renewable_profiles` for the PV and wind hourly capacity factors in each Voronoi cell.
+- `build_powerplants` uses [powerplantmatching](https://github.com/FRESNA/powerplantmatching) to determine today's thermal power plant capacities and then locates the closest substation for each powerplant.
+
+The central rule `add_electricity` then ties all the different data inputs together to a detailed PyPSA model stored in `networks/elec.nc`, containing:
+
+- Today's transmission topology and capacities (optionally including lines which are under construction according to the config settings `lines: under_construction` and `links: under_construction`)
+- Today's thermal and hydro generation capacities (for the technologies listed in the config setting `electricity: conventional_carriers`)
+- Today's load time-series (upsampled according to population and gross domestic product)
+
+It further adds extendable `generators` and `storage_units` with *zero* capacity for
+- wind and pv installations with today's locational, hourly wind and solar pv capacity factors (but **no** capacities)
+- long-term hydrogen and short-term battery storage units (if listed in `electricity: extendable_carriers`)
+- additional open-cycle gas turbines (if `OCGT` is listed in `electricity: extendable_carriers`)
+
 .. each rule description should have a list of parameters
 .. from the config.yaml that affect this rule.
 

doc/simplification.rst

@@ -4,6 +4,14 @@
 Simplifying Networks
 ##########################################
 
+The additional rules prepare approximations of the full model, in which generation, storage and transmission capacities can be co-optimized
+- `simplify_network` transforms the transmission grid to a 380 kV-only equivalent network, while
+- `cluster_network` uses a kmeans based clustering technique to partition the network into a certain number of zones and then reduce the network to a representation with one bus per zone.
+
+The simplification and clustering steps are described in detail in the paper
+[The role of spatial scale in joint optimisations of generation and transmission for European highly renewable scenarios](https://arxiv.org/abs/1705.07617), 2017, [arXiv:1705.07617](https://arxiv.org/abs/1705.07617), [doi:10.1109/EEM.2017.7982024](https://doi.org/10.1109/EEM.2017.7982024).
+
+
 Simplify Network
 ================
 

doc/solving.rst

@@ -2,6 +2,13 @@
 Solving Networks
 ##########################################
 
+After generating the network it can be solved by using 'solve_all_elec_networks'. This runs the following rules:
+- 'cluster_network'
+- 'prepare_network'
+- 'solve_all_elec_networks'
+- 'solve_network'
+
+
 Solve Network
 =============
 

doc/wildcards.rst

@@ -0,0 +1,54 @@
+#########
+Wildcards
+#########
+
+Detailed explanations of how wildcards work in ``snakemake`` can be found in the `relevant section of the documentation <https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#wildcards>`_.
+
+The ``simpl`` wildcard
+======================
+
+The ``ll`` wildcard
+===================
+
+The ``clusters`` wildcard
+=========================
+
+The ``network`` wildcard
+========================
+
+
+The ``opts`` wildcard
+=====================
+
+The ``opts`` wildcard triggers optional constraints, which are activated in either ``prepare_network`` or the ``solve_network`` step. It may hold multiple triggers separated by ``-``, i.e. ``Co2L-3H`` contains the ``Co2L`` trigger and the ``3H`` switch. There are currently:
+
+
+.. csv-table::
+   :header-rows: 1
+   :widths: 10,20,10,10
+   :file: configtables/opts.csv
+
+
+The ``country`` wildcard
+========================
+
+The rules ``make_summary`` (generating summaries of all or a subselection of the solved networks) and ``plot_p_nom_max`` (for plotting the cumulative generation potentials for renewable technologies) can be narrowed to individual countries using the ``country`` wildcard.
+
+If ``country = all``, then the rule acts on the network for all countries defined in ``config.yaml``. If otherwise ``country = DE`` or another country code, then the network is narrowed to buses of this country for the rule. For example to get a summary of the energy generated in Germany (in the solution for Europe) use:
+
+
+.. code:: bash
+
+    snakemake results/summaries/elec_s_all_lall_Co2L-3H_DE
+
+The ``cutout`` wildcard
+=======================
+
+The ``technology`` wildcard
+===========================
+
+The ``attr`` wildcard
+=====================
+
+The ``ext`` wildcard
+====================