Koen van Greevenbroek
90fb0b13da
Various bugfixes to make multiyear branch run
2024-02-29 12:48:16 +01:00
pre-commit-ci[bot]
8dfab45453
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2024-02-27 12:35:59 +00:00
toniseibold
5bcecc62c6
retrieve eurostat data is now outside of retrieve_sector_databundle
2024-02-27 13:32:07 +01:00
toniseibold
d363aeb57d
removing old eurostat data reports as an option, cleaning up code
2024-02-27 12:04:07 +01:00
Fabian Neumann
719d4dad23
Merge branch 'master' into scenario-management
2024-02-20 12:08:54 +01:00
Fabian Neumann
16e2286188
update copyright years to 2024
2024-02-19 16:21:48 +01:00
Fabian Neumann
b543bf37c9
retrieve.smk: do not protect gas network data
2024-02-17 20:48:46 +01:00
Fabian Neumann
e898781fda
use wildcard input functions where applicable
2024-02-17 18:35:26 +01:00
Fabian Neumann
16e42c8fe6
further progress and bugfixes
2024-02-16 11:17:00 +01:00
Fabian Neumann
c83db4e84e
add retrieve_cost_data.py script to handle config_provider and move to resources
...
since dependent on config setting
2024-02-12 16:54:33 +01:00
Fabian Neumann
6b013dbd40
Merge branch 'master' into scenario-management
2024-02-12 10:59:37 +01:00
Fabian Neumann
09ada56bf1
finishing touches and release notes
2024-02-12 10:49:45 +01:00
Fabian
94f72d7276
electricity demand: remove powerstastics flag, merge sources in retrieve_electricity_demand
2024-02-12 10:08:39 +01:00
Fabian
2bd8021058
retrieve electricity demand: make online version snakemake params
2024-02-12 09:48:42 +01:00
Fabian
5c7bbcd94d
retrieve electricity demand: use script in order to concat time-series
2024-02-12 09:48:28 +01:00
Fabian Neumann
a9dad3f34e
add new resources(), logs(), benchmarks(), config_providers()
2024-02-10 18:09:23 +01:00
Fabian Neumann
6de08bd523
Merge branch 'master' into scenario-management
2024-02-10 17:22:01 +01:00
pre-commit-ci[bot]
39890630ef
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2024-02-05 18:23:51 +00:00
pre-commit-ci[bot]
58ac6629fe
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2024-01-23 15:35:11 +00:00
Koen van Greevenbroek
48974edc8d
Make data retrieval rules use their own minimal conda env
...
This means that the main conda environment can be changed for
development purposes without necessitating the re-running of data
retrieval due to software environment change by default.
2024-01-23 16:27:12 +01:00
pre-commit-ci[bot]
68421e6aab
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2024-01-15 18:14:34 +00:00
pre-commit-ci[bot]
7b1600164f
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2024-01-04 19:19:02 +00:00
Fabian Neumann
9d0040912b
retrieve.smk: simplify gas infrastructure retrieve rules
2024-01-04 19:14:11 +01:00
Fabian Neumann
c3bb41c900
Merge branch 'master' into multiyear
2024-01-03 15:14:27 +01:00
pre-commit-ci[bot]
045eeba4cf
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2024-01-03 14:02:26 +00:00
Fabian Neumann
ecd85d23d3
add option to use LUISA land coverage data
2024-01-03 15:01:43 +01:00
Fabian Neumann
19b503d758
retrieve.smk: add scigrid storages to files of interest
2024-01-03 08:35:00 +01:00
Fabian Neumann
fac257ca97
Merge pull request #821 from PyPSA/md5-checksums
...
validate checksums for zenodo downloads
2024-01-02 17:23:06 +01:00
Fabian Neumann
19a50fb8c5
rules: add keep_local=True back in for storage providers
2023-12-31 14:07:45 +01:00
Fabian Neumann
48850431fb
Snakefile: switch to new storage providers
2023-12-30 14:37:45 +01:00
Fabian Neumann
71985d5e3a
validate checksums for zenodo downloads
2023-12-29 12:34:14 +01:00
Fabian Neumann
02b0996dac
remove write-protection on resources/natura.tiff
2023-12-29 11:29:19 +01:00
Fabian Neumann
4f815a9cd7
move wdpa and glc datasets to data directory and protect
2023-12-29 11:29:01 +01:00
Fabian Neumann
c5f48f957d
wdpa: give snakemake appearance of time-invariant wdpa files
2023-12-29 11:20:40 +01:00
Fabian Neumann
471c97d499
wdpa: correct _shp url
2023-12-29 11:20:14 +01:00
pre-commit-ci[bot]
fec641ce0a
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2023-12-22 13:41:25 +00:00
euronion
d817212896
Fix missing bYYYY
2023-12-22 14:40:30 +01:00
pre-commit-ci[bot]
3a474af71f
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2023-12-21 17:40:48 +00:00
euronion
25c5751565
Check WDPA url also a month forward
2023-12-21 18:39:06 +01:00
Fabian
4f67160993
fix snake format
2023-12-18 11:57:21 +01:00
Fabian Neumann
df40fdf904
Merge branch 'master' into ukraine_hackathon
2023-12-11 16:24:40 +01:00
Fabian Neumann
803b860923
move load_raw.csv to resources since it's config file dependent
2023-11-29 13:41:26 +01:00
Irieo
d2c1f9ec32
data: solve floating url problem for WDPA data retrieval
2023-11-03 14:14:01 +01:00
pre-commit-ci[bot]
d10779ca34
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2023-10-11 14:27:25 +00:00
Thomas Gilon
4c5ea9e207
Specify input files required to retrieve irena data
...
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2023-10-11 16:27:08 +02:00
Thomas Gilon
c8fcbaad6e
Update default value in retrieve.smk
...
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2023-10-11 16:24:52 +02:00
pre-commit-ci[bot]
230c1a327c
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2023-10-11 12:27:29 +00:00
Thomas Gilon
43e20a26e6
Add rule to update IRENA data for existing infrastructure
2023-10-11 14:26:04 +02:00
Fabian Neumann
091584bb0d
add download rules for WDPA land and marine
2023-10-09 16:22:27 +02:00
Fabian Neumann
ea99f3c43e
Merge branch 'master' into ukraine_hackathon
2023-10-09 09:53:42 +02:00
Fabian Neumann
b8fee80919
Merge branch 'master' into multiyear
2023-10-08 11:55:11 +02:00
pre-commit-ci[bot]
b8dee184a9
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2023-08-24 09:12:07 +00:00
Fabian Neumann
d0f2d52853
apply protected() to zenodo downloads; only retrieve monthly fuel/CO2 prices when necessary
2023-08-24 11:11:33 +02:00
Fabian
62c6143863
scenario-management: reenable shared resources, make shared resources wildcards dependent
2023-08-23 17:14:57 +02:00
Fabian
9d4ce430cc
electricity demand: remove powerstastics flag, merge sources in retrieve_electricity_demand
2023-08-17 10:17:12 +02:00
Fabian
32eb114b61
retrieve electricity demand: make online version snakemake params
2023-08-16 13:05:22 +02:00
Fabian
9cd449cf3c
retrieve electricity demand: use script in order to concat time-series
2023-08-16 11:41:48 +02:00
Fabian
e28ae59375
introduce scenario-management
2023-08-15 15:06:59 +02:00
Fabian Neumann
8d19043205
handle monthly fuel price download through script
2023-08-06 09:01:56 +02:00
Fabian
18297a6f13
retrieve: fix data path
2023-08-05 16:41:16 +02:00
Fabian
62cb6fbdd8
validation: download input data files instead
2023-08-04 17:38:21 +02:00
Fabian
bc6cc6c983
retrieve: update opsd electricity demand url to fixed date
2023-08-02 11:00:21 +02:00
pre-commit-ci[bot]
7530c42fbe
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2023-08-01 06:43:48 +00:00
Fabian Neumann
23eb3548c5
Merge branch 'master' into ukraine_hackathon
2023-08-01 08:42:47 +02:00
Fabian Neumann
ddd1840981
Merge remote-tracking branch 'origin/master' into multiyear
2023-07-26 10:23:32 +02:00
Fabian
6a50de143c
retrieve: fix load data version
2023-07-25 12:10:42 +02:00
pre-commit-ci[bot]
54e813ffef
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2023-07-19 09:38:07 +00:00
Fabian Hofmann
844a495180
Merge branch 'master' into validation
2023-07-19 11:37:47 +02:00
euronion
730b320596
Move logic to retrieve.smk
2023-07-14 08:53:39 +02:00
Fabian
9958425a44
complete structure for plotting electricity production
2023-07-05 11:07:36 +02:00
Fabian
6137a634d4
build_electricity_demand: make data source snapshot dependent
2023-07-03 12:51:26 +02:00
Fabian Hofmann
6004e04354
Merge branch 'master' into validation
2023-06-30 12:07:04 +02:00
Fabian
701231fdca
build_renewable_profile: fix execution for one thread
2023-06-30 11:34:41 +02:00
Fabian Hofmann
07224751e5
Merge branch 'master' into validation
2023-06-29 08:10:52 +02:00
virio-andreyana
8208ac033b
convert param back to config for "logging" and "run", made "enable" more specific
2023-05-17 18:43:30 +02:00
virio-andreyana
61893c3c9b
fixing build_electricity.smk and retrieve.smk
2023-05-15 11:26:36 +02:00
virio-andreyana
1fc48d8753
using python algorithm to edit rules and script
2023-05-15 10:33:17 +02:00
Fabian Neumann
6acd5da4d4
Merge branch 'multiyear-sec' into multiyear
2023-04-29 18:49:49 +02:00
Fabian Neumann
30beee0959
Merge branch 'master' into multiyear
2023-04-29 12:40:55 +02:00
Fabian
8994561569
Merge branch 'master' into validation
2023-04-25 16:58:48 +02:00
Fabian
8ecd0ef580
retrieve.smk: set electricity demand source to latest
2023-04-25 16:51:20 +02:00
Fabian
9b631189c5
retrieve.smk: rename retrieve_load_data -> retrieve_electricity_demand
2023-04-25 16:37:01 +02:00
Fabian Neumann
7405377fd7
snakefmt to Snakefile
2023-03-08 19:22:29 +01:00
Fabian Neumann
bf895de7cc
add logfiles to all rules
2023-03-08 19:22:00 +01:00
Fabian Neumann
abbf91ff9e
retrieval rules get retries directive for added robustness
2023-03-08 18:09:31 +01:00
Fabian Neumann
37a063554d
make cutouts protected
...
https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html\#protected-and-temporary-files
2023-03-08 17:56:50 +01:00
Fabian Neumann
4b3372b53b
apply snakefmt to rules/
2023-03-08 17:46:14 +01:00
Fabian Neumann
8eb37bc263
add conda directive to all rules for use with --use-conda
2023-03-08 17:44:25 +01:00
Fabian Neumann
6bae022d44
modularise Snakefile using rules/*.smk files
2023-03-08 17:29:01 +01:00
