fix snakemake as --cores (-j) is now a mandatory argument

in 5.11.0 https://snakemake.readthedocs.io/en/stable/project_info/history.html
This commit is contained in:
Fabian Neumann 2020-03-17 14:55:30 +01:00
parent ccc56ce3c8
commit 0c6c23478b
6 changed files with 14 additions and 14 deletions

View File

@ -19,6 +19,6 @@ before_install:
script:
- cp ./test/config.test1.yaml ./config.yaml
- snakemake solve_all_elec_networks
- snakemake -j all solve_all_elec_networks
- rm -rf resources/*.nc resources/*.geojson resources/*.h5 networks results
# could repeat for more configurations in future

View File

@ -35,7 +35,7 @@ facilitate running multiple scenarios through a single command
.. code:: bash
snakemake solve_all_elec_networks
snakemake -j 1 solve_all_elec_networks
For each wildcard, a **list of values** is provided. The rule ``solve_all_elec_networks`` will trigger the rules for creating ``results/networks/elec_s{simpl}_{clusters}_ec_l{ll}_{opts}.nc`` for **all combinations** of the provided wildcard values as defined by Python's `itertools.product(...) <https://docs.python.org/2/library/itertools.html#itertools.product>`_ function that snakemake's `expand(...) function <https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#targets>`_ uses.

View File

@ -17,7 +17,7 @@ For instance an invocation to
.. code:: bash
.../pypsa-eur % snakemake networks/elec_s_128.nc
.../pypsa-eur % snakemake -j 1 networks/elec_s_128.nc
follows this dependency graph:
@ -29,7 +29,7 @@ The **blocks** represent the individual rules which are required to create the f
The dependency graph shown above was generated using
``snakemake --dag networks/elec_s_128.nc | dot -Tpng > workflow.png``
For the use of ``snakemake``, it makes sense to familiarize oneself quickly with its `basic tutorial <https://snakemake.readthedocs.io/en/stable/tutorial/basics.html>`_ and then read carefully through the section `Executing Snakemake <https://snakemake.readthedocs.io/en/stable/executable.html>`_, noting the arguments ``-n``, ``-r``, but also ``--dag``, ``-R`` and ``-t``.
For the use of ``snakemake``, it makes sense to familiarize oneself quickly with its `basic tutorial <https://snakemake.readthedocs.io/en/stable/tutorial/basics.html>`_ and then read carefully through the section `Executing Snakemake <https://snakemake.readthedocs.io/en/stable/executable.html>`_, noting the arguments ``-j``, ``-n``, ``-r``, but also ``--dag``, ``-R`` and ``-t``.
Scenarios, Configuration and Modification
=========================================

View File

@ -63,7 +63,7 @@ Release Process
* Upload code to `zenodo code repository <https://doi.org/10.5281/zenodo.3520875>`_ with `GNU GPL 3.0 <https://www.gnu.org/licenses/gpl-3.0.en.html>`_ license.
* Create pre-built networks for ``config.default.yaml`` by running ``snakemake extra_components_all_elec_networks``.
* Create pre-built networks for ``config.default.yaml`` by running ``snakemake -j 1 extra_components_all_elec_networks``.
* Upload pre-built networks to `zenodo data repository <https://doi.org/10.5281/zenodo.3601882>`_ with `CC BY 4.0 <https://creativecommons.org/licenses/by/4.0/>`_ license.

View File

@ -103,7 +103,7 @@ clustered down to 6 buses and every 24 hours aggregated to one snapshot. The com
.. code:: bash
.../pypsa-eur % snakemake results/networks/elec_s_6_ec_lcopt_Co2L-24H.nc
.../pypsa-eur % snakemake -j 1 results/networks/elec_s_6_ec_lcopt_Co2L-24H.nc
orders ``snakemake`` to run the script ``solve_network`` that produces the solved network and stores it in ``.../pypsa-eur/results/networks`` with the name ``elec_s_6_ec_lcopt_Co2L-24H.nc``:
@ -120,7 +120,7 @@ orders ``snakemake`` to run the script ``solve_network`` that produces the solve
.. warning::
On Windows the previous command may currently cause a ``MissingRuleException`` due to problems with output files in subfolders.
This is an `open issue <https://github.com/snakemake/snakemake/issues/46>`_ at `snakemake <https://snakemake.readthedocs.io/>`_.
Windows users should add the option ``--keep-target-files`` to the command or instead run ``snakemake solve_all_elec_networks``.
Windows users should add the option ``--keep-target-files`` to the command or instead run ``snakemake -j 1 solve_all_elec_networks``.
This triggers a workflow of multiple preceding jobs that depend on each rule's inputs and outputs:
@ -242,15 +242,15 @@ You can produce any output file occuring in the ``Snakefile`` by running
.. code:: bash
.../pypsa-eur % snakemake <output file>
.../pypsa-eur % snakemake -j 1 <output file>
For example, you can explore the evolution of the PyPSA networks by running
#. ``.../pypsa-eur % snakemake networks/base.nc``
#. ``.../pypsa-eur % snakemake networks/elec.nc``
#. ``.../pypsa-eur % snakemake networks/elec_s.nc``
#. ``.../pypsa-eur % snakemake networks/elec_s_6.nc``
#. ``.../pypsa-eur % snakemake networks/elec_s_6_ec_lcopt_Co2L-24H.nc``
#. ``.../pypsa-eur % snakemake -j 1 networks/base.nc``
#. ``.../pypsa-eur % snakemake -j 1 networks/elec.nc``
#. ``.../pypsa-eur % snakemake -j 1 networks/elec_s.nc``
#. ``.../pypsa-eur % snakemake -j 1 networks/elec_s_6.nc``
#. ``.../pypsa-eur % snakemake -j 1 networks/elec_s_6_ec_lcopt_Co2L-24H.nc``
There's a special rule: If you simply run

View File

@ -124,7 +124,7 @@ in Germany (in the solution for Europe) use:
.. code:: bash
snakemake results/summaries/elec_s_all_lall_Co2L-3H_DE
snakemake -j 1 results/summaries/elec_s_all_lall_Co2L-3H_DE
.. seealso::
:mod:`make_summary`, :mod:`plot_summary`, :mod:`plot_p_nom_max`