Separate retrieval and building of cutouts, natura raster into distinct configurations each. (#136)

* Separate config switches for retrieving and building cutout, natura raster.

* Include new switches in tutorial and test configs.

* Update release_notes.rst

* Update toplevel.csv

* Update tutorial.rst

* Update configuration.rst

Snakefile

@@ -135,7 +135,8 @@ if config['enable'].get('build_cutout', False):
         benchmark: "benchmarks/build_cutout_{cutout}"
         # group: 'feedin_preparation'
         script: "scripts/build_cutout.py"
-else:
+
+if config['enable'].get('retrieve_cutout', True):
     rule retrieve_cutout:
         output: directory(expand("cutouts/{cutouts}", **config['atlite'])),
         log: "logs/retrieve_cutout.log"
@@ -150,7 +151,8 @@ if config['enable'].get('build_natura_raster', False):
         output: "resources/natura.tiff"
         log: "logs/build_natura_raster.log"
         script: "scripts/build_natura_raster.py"
-else:
+
+if config['enable'].get('retrieve_natura_raster', True):
     rule retrieve_natura_raster:
         output: "resources/natura.tiff"
         log: "logs/retrieve_natura_raster.log"

config.default.yaml

@@ -25,7 +25,9 @@ enable:
   prepare_links_p_nom: false
   retrieve_databundle: true
   build_cutout: false
+  retrieve_cutout: true
   build_natura_raster: false
+  retrieve_natura_raster: true
 
 electricity:
   voltages: [220., 300., 380.]

config.tutorial.yaml

@@ -24,7 +24,9 @@ enable:
   prepare_links_p_nom: false
   retrieve_databundle: true
   build_cutout: false
+  retrieve_cutout: true
   build_natura_raster: false
+  retrieve_natura_raster: true
 
 electricity:
   voltages: [220., 300., 380.]

doc/configtables/toplevel.csv

@@ -10,5 +10,7 @@ focus_weights,--,"Keys should be two-digit country codes (e.g. DE) and values sh
 enable,,,
 -- prepare_links_p_nom,bool,"{true, false}","Switch to retrieve current HVDC projects from `Wikipedia <https://en.wikipedia.org/wiki/List_of_HVDC_projects>`_"
 -- retrieve_databundle,bool,"{true, false}","Switch to retrieve databundle from zenodo via the rule :mod:`retrieve_databundle` or whether to keep a custom databundle located in the corresponding folder."
--- build_cutout,bool,"{true, false}","Switch to build cutouts via the rule :mod:`build_cutout` or whether to retrieve the data from zenodo with :mod:`retrieve_cutout`."
--- build_natura_raster,bool,"{true, false}","Switch to create raster ``natura.tiff`` via the rule :mod:`build_natura_raster` or whether to retrieve ``natura.tiff`` from zenodo with :mod:`retrieve_natura_raster`."
+-- build_cutout,bool,"{true, false}","Switch to enable the building of cutouts via the rule :mod:`build_cutout`."
+-- retrieve_cutout,bool,"{true, false}","Switch to enable the retrieval of cutouts from zenodo with :mod:`retrieve_cutout`."
+-- build_natura_raster,bool,"{true, false}","Switch to enable the creation of the raster ``natura.tiff`` via the rule :mod:`build_natura_raster`."
+-- retrieve_natura_raster,bool,"{true, false}","Switch to enable the retrieval of ``natura.tiff`` from zenodo with :mod:`retrieve_natura_raster`."

doc/configuration.rst

@@ -13,7 +13,7 @@ Top-level configuration
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 1-8,17,24-28
+   :lines: 1-8,17,24-30
 
 .. csv-table::
    :header-rows: 1
@@ -75,7 +75,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 30-48
+   :lines: 32-50
 
 .. csv-table::
    :header-rows: 1
@@ -92,7 +92,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 55-68
+   :lines: 57-70
 
 .. csv-table::
    :header-rows: 1
@@ -109,7 +109,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 70-87
+   :lines: 72-89
 
 .. csv-table::
    :header-rows: 1
@@ -121,7 +121,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 70,88-100
+   :lines: 72,90-102
 
 .. csv-table::
    :header-rows: 1
@@ -133,7 +133,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 70,101-114
+   :lines: 72,103-116
 
 .. csv-table::
    :header-rows: 1
@@ -145,7 +145,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 70,115-134
+   :lines: 72,117-136
 
 .. csv-table::
    :header-rows: 1
@@ -157,7 +157,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 70,135-141
+   :lines: 72,137-143
 
 .. csv-table::
    :header-rows: 1
@@ -171,7 +171,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 143-150
+   :lines: 145-152
 
 .. csv-table::
    :header-rows: 1
@@ -185,7 +185,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 152-155
+   :lines: 154-157
 
 .. csv-table::
    :header-rows: 1
@@ -199,7 +199,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 157-160
+   :lines: 159-162
 
 .. csv-table::
    :header-rows: 1
@@ -213,7 +213,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 162-163
+   :lines: 164-165
 
 .. csv-table::
    :header-rows: 1
@@ -227,7 +227,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 165-177
+   :lines: 167-179
 
 .. csv-table::
    :header-rows: 1
@@ -249,7 +249,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 179-187
+   :lines: 181-189
 
 .. csv-table::
    :header-rows: 1
@@ -261,7 +261,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 179,188-204
+   :lines: 181,190-206
 
 .. csv-table::
    :header-rows: 1
@@ -275,7 +275,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 206-340
+   :lines: 208-342
 
 .. csv-table::
    :header-rows: 1

doc/release_notes.rst

@@ -7,6 +7,8 @@ PyPSA-Eur 0.2.0 (TBD)
 
 * The optimization is now performed using the ``pyomo=False`` setting in the :func:`pypsa.lopf.network_lopf`. This speeds up the solving process significantly and consumes much less memory. The inclusion of additional constraints were adjusted to the new implementation. They are all passed to the :func:`network_lopf` function via the ``extra_functionality`` argument. The rule ``trace_solve_network`` was integrated into the rule :mod:`solve_network` and can be activated via configuration with ``solving: options: track_iterations: true``. The charging and discharging capacities of batteries modelled as store-link combination are now coupled (`#116 <https://github.com/PyPSA/pypsa-eur/pull/116>`_).
 
+* ``snakemake`` rules for retrieving cutouts and the natura raster can now be disabled independently from their respective rules to build them; via ``config.*yaml`` (`#136 <https://github.com/PyPSA/pypsa-eur/pull/136>`_).
+
 
 PyPSA-Eur 0.1.0 (9th January 2020)
 ==================================

doc/tutorial.rst

@@ -50,35 +50,35 @@ It is also possible to allow less or more carbon-dioxide emissions. Here, we lim
 
 .. literalinclude:: ../config.tutorial.yaml
    :language: yaml
-   :lines: 31
+   :lines: 33
 
 PyPSA-Eur also includes a database of existing conventional powerplants.
 We can select which types of powerplants we like to be included with fixed capacities:
 
 .. literalinclude:: ../config.tutorial.yaml
    :language: yaml
-   :lines: 45
+   :lines: 47
 
 To accurately model the temporal and spatial availability of renewables such as wind and solar energy, we rely on historical weather data.
 It is advisable to adapt the required range of coordinates to the selection of countries.
 
 .. literalinclude:: ../config.tutorial.yaml
    :language: yaml
-   :lines: 47-55
+   :lines: 49-57
 
 We can also decide which weather data source should be used to calculate potentials and capacity factor time-series for each carrier.
 For example, we may want to use the ERA-5 dataset for solar and not the default SARAH-2 dataset.
 
 .. literalinclude:: ../config.tutorial.yaml
    :language: yaml
-   :lines: 57,100-101
+   :lines: 59,102-103
 
 Finally, it is possible to pick a solver. For instance, this tutorial uses the open-source solvers CBC and Ipopt and does not rely
 on the commercial solvers Gurobi or CPLEX (for which free academic licenses are available).
 
 .. literalinclude:: ../config.tutorial.yaml
    :language: yaml
-   :lines: 156,165-166
+   :lines: 158,167-168
 
 .. note::
 

test/config.test1.yaml

@@ -24,7 +24,9 @@ enable:
   prepare_links_p_nom: false
   retrieve_databundle: true
   build_cutout: false
+  retrieve_cutout: true
   build_natura_raster: false
+  retrieve_natura_raster: true
 
 electricity:
   voltages: [220., 300., 380.]