eaf30a9b65
* rewrite mocksnakemake for parsing real Snakefile * continue add function to scripts * going through all scripts, setting new mocksnakemake * fix plotting scripts * fix build_country_flh * fix build_country_flh II * adjust config files * fix make_summary for tutorial network * create dir also for output * incorporate suggestions * consistent import of mocksnakemake * consistent import of mocksnakemake II * Update scripts/_helpers.py Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com> * Update scripts/_helpers.py Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com> * Update scripts/_helpers.py Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com> * Update scripts/_helpers.py Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com> * Update scripts/plot_network.py Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com> * Update scripts/plot_network.py Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com> * Update scripts/retrieve_databundle.py Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com> * use pathlib for mocksnakemake * rename mocksnakemake into mock_snakemake * revert change in data * Update scripts/_helpers.py Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com> * remove setting logfile in mock_snakemake, use Path in configure_logging * fix fallback path and base_dir fix return type of make_io_accessable * reformulate mock_snakemake * incorporate suggestion, fix typos * mock_snakemake: apply absolute paths again, add assertion error *.py: make hard coded io path accessable for mock_snakemake * retrieve_natura_raster: use snakemake.output for fn_out * include suggestion * Apply suggestions from code review Co-Authored-By: Jonas Hörsch <jonas.hoersch@posteo.de> * linting, add return ad end of file * Update scripts/plot_p_nom_max.py Co-Authored-By: Jonas Hörsch <jonas.hoersch@posteo.de> * Update scripts/plot_p_nom_max.py fixes #112 Co-Authored-By: Jonas Hörsch <jonas.hoersch@posteo.de> * plot_p_nom_max: small correction * config.tutorial.yaml fix snapshots end * use techs instead of technology * revert try out from previous commit, complete replacing * change clusters -> clusts in plot_p_nom_max due to wildcard constraints of clusters * change clusters -> clusts in plot_p_nom_max due to wildcard constraints of clusters II
66 lines
2.3 KiB
Python
66 lines
2.3 KiB
Python
## Copyright 2019 Fabian Hofmann (FIAS)
|
|
"""
|
|
.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.3517935.svg
|
|
:target: https://doi.org/10.5281/zenodo.3517935
|
|
|
|
The data bundle (1.4 GB) contains common GIS datasets like NUTS3 shapes, EEZ shapes, CORINE Landcover, Natura 2000 and also electricity specific summary statistics like historic per country yearly totals of hydro generation, GDP and POP on NUTS3 levels and per-country load time-series.
|
|
|
|
This rule downloads the data bundle from `zenodo <https://doi.org/10.5281/zenodo.3517935>`_ and extracts it in the ``data`` sub-directory, such that all files of the bundle are stored in the ``data/bundle`` subdirectory.
|
|
|
|
The :ref:`tutorial` uses a smaller `data bundle <https://zenodo.org/record/3517921/files/pypsa-eur-tutorial-data-bundle.tar.xz>`_ than required for the full model (19 MB)
|
|
|
|
.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.3517921.svg
|
|
:target: https://doi.org/10.5281/zenodo.3517921
|
|
|
|
**Relevant Settings**
|
|
|
|
.. code:: yaml
|
|
|
|
tutorial:
|
|
|
|
.. seealso::
|
|
Documentation of the configuration file ``config.yaml`` at
|
|
:ref:`toplevel_cf`
|
|
|
|
**Outputs**
|
|
|
|
- ``cutouts/bundle``: input data collected from various sources
|
|
|
|
"""
|
|
|
|
import logging
|
|
logger = logging.getLogger(__name__)
|
|
from _helpers import progress_retrieve, configure_logging
|
|
|
|
from pathlib import Path
|
|
import tarfile
|
|
|
|
if __name__ == "__main__":
|
|
# Detect running outside of snakemake and mock snakemake for testing
|
|
if 'snakemake' not in globals():
|
|
from _helpers import mock_snakemake
|
|
snakemake = mock_snakemake('retrieve_databundle')
|
|
rootpath = '..'
|
|
else:
|
|
rootpath = '.'
|
|
configure_logging(snakemake) # TODO Make logging compatible with progressbar (see PR #102)
|
|
|
|
if snakemake.config['tutorial']:
|
|
url = "https://zenodo.org/record/3517921/files/pypsa-eur-tutorial-data-bundle.tar.xz"
|
|
else:
|
|
url = "https://zenodo.org/record/3517935/files/pypsa-eur-data-bundle.tar.xz"
|
|
|
|
# Save locations
|
|
tarball_fn = Path(f"{rootpath}/bundle.tar.xz")
|
|
to_fn = Path(f"{rootpath}/data")
|
|
|
|
logger.info(f"Downloading databundle from '{url}'.")
|
|
progress_retrieve(url, tarball_fn)
|
|
|
|
logger.info(f"Extracting databundle.")
|
|
tarfile.open(tarball_fn).extractall(to_fn)
|
|
|
|
tarball_fn.unlink()
|
|
|
|
logger.info(f"Databundle available in '{to_fn}'.")
|