Commit Graph

366 Commits

Author SHA1 Message Date
Fabian Neumann
87f5a11bbe Merge branch 'scenario-management' of github.com:PyPSA/pypsa-eur into scenario-management 2024-02-12 11:53:49 +01:00
Fabian Neumann
9400b6dd75 do not use config_provider outside rule definitions 2024-02-12 11:53:37 +01:00
pre-commit-ci[bot]
1a883debb5 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-02-12 10:00:05 +00:00
Fabian Neumann
6b013dbd40 Merge branch 'master' into scenario-management 2024-02-12 10:59:37 +01:00
Fabian Neumann
09ada56bf1 finishing touches and release notes 2024-02-12 10:49:45 +01:00
Fabian
94f72d7276 electricity demand: remove powerstastics flag, merge sources in retrieve_electricity_demand 2024-02-12 10:08:39 +01:00
Fabian
2bd8021058 retrieve electricity demand: make online version snakemake params 2024-02-12 09:48:42 +01:00
Fabian
5c7bbcd94d retrieve electricity demand: use script in order to concat time-series 2024-02-12 09:48:28 +01:00
pre-commit-ci[bot]
4ff06046fc [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-02-10 17:09:49 +00:00
Fabian Neumann
a9dad3f34e add new resources(), logs(), benchmarks(), config_providers() 2024-02-10 18:09:23 +01:00
pre-commit-ci[bot]
e838b63ded [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-02-10 16:22:38 +00:00
Fabian Neumann
6de08bd523 Merge branch 'master' into scenario-management 2024-02-10 17:22:01 +01:00
Fabian Neumann
3691f9f4c1 update version compatibility handling 2024-02-09 13:59:15 +01:00
Fabian Neumann
b9c71c3c15 add logs to plot_*_network rules 2024-02-07 18:52:01 +01:00
Fabian Neumann
602375af8c
Merge branch 'master' into fneum/year-specific-techs 2024-02-06 13:56:14 +01:00
pre-commit-ci[bot]
39890630ef [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-02-05 18:23:51 +00:00
Fabian Neumann
f0ec1d6716
Merge branch 'master' into perfect-foresight-fixes 2024-02-05 12:18:53 +01:00
Fabian Neumann
a834ff222a streamline code for year-dependent technologies (turbines/panels) 2024-02-05 12:10:35 +01:00
Fabian Neumann
bb4eb123e5 Merge branch 'year-specific-techs' of github.com:p-glaum/pypsa-eur into p-glaum-year-specific-techs 2024-02-05 09:06:32 +01:00
Fabian Neumann
e76749ab31 plot_hydrogen_network: add foresight params back in 2024-02-05 08:37:19 +01:00
Fabian Neumann
344989d76b
Apply suggestions from code review 2024-02-05 08:21:26 +01:00
Koen van Greevenbroek
25a24f5552 Remove unnecessary add_brownfield rule from perfect foresight 2024-02-02 08:49:29 +01:00
Koen van Greevenbroek
4c0491341c Move build_existing_heating_distribution rule 2024-02-02 08:49:29 +01:00
Fabian
0f80e2d089 plot: fix params and groupby deprecation warning 2024-01-31 17:07:01 +01:00
Fabian Neumann
b43088ea49
Merge branch 'master' into split-plotting-rules 2024-01-26 17:10:21 +01:00
Fabian Neumann
21a943d61e Merge branch 'split-plotting-rules' of github.com:PyPSA/pypsa-eur into split-plotting-rules 2024-01-26 17:07:54 +01:00
Fabian Neumann
8cb7396c11 make_summary: name all inputs 2024-01-26 17:07:19 +01:00
pre-commit-ci[bot]
d6288d3d4d [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-01-26 16:06:23 +00:00
Fabian Neumann
f84e73006f postprocess: keep network plotting triggers in make_summary 2024-01-26 17:05:38 +01:00
Fabian Neumann
7becfdea9c resolve merge conflict 2024-01-25 20:51:51 +01:00
pre-commit-ci[bot]
5fcfafe971 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-01-25 19:50:06 +00:00
Fabian Neumann
a2cd042472 plot clustered network topology before optimisation 2024-01-25 20:48:38 +01:00
pre-commit-ci[bot]
30ccde5b90 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-01-25 19:39:09 +00:00
Fabian Neumann
3f112f0e95 incorporate network plots in rule 2024-01-25 20:37:33 +01:00
Fabian Neumann
ffd4e1f1af plot_network: split into separate scripts for power, hydrogen, gas 2024-01-25 20:34:59 +01:00
Fabian Neumann
bb202ad2c4 plot_network: remove function plot_map_without() 2024-01-25 17:28:33 +01:00
Fabian Neumann
999ff85288 fix snakemake.inputs for add_existing_baseyear with perfect foresight 2024-01-25 17:19:55 +01:00
Fabian Neumann
c4b305bfbe
Merge branch 'master' into fneum/ariadne2 2024-01-25 16:52:46 +01:00
Koen van Greevenbroek
1fab74c463 avoid the use use of workflow.source_path
The above function returns a path to a cached source file, which
changes for every run, causing the resulting parameter to change and
the solve_network rule to re-run
2024-01-24 14:39:50 +01:00
Fabian Neumann
eeb3d0789b correctly read number of solver threads in rule definition 2024-01-23 18:03:14 +01:00
pre-commit-ci[bot]
58ac6629fe [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-01-23 15:35:11 +00:00
Koen van Greevenbroek
48974edc8d Make data retrieval rules use their own minimal conda env
This means that the main conda environment can be changed for
development purposes without necessitating the re-running of data
retrieval due to software environment change by default.
2024-01-23 16:27:12 +01:00
Fabian Neumann
6c593a551b build_hourly_heat_demand: only pass subset of snapshot config 2024-01-22 18:07:33 +01:00
Fabian Neumann
9865a97089 apply automated formatting 2024-01-22 09:29:32 +01:00
Fabian Neumann
d3cf329456 correctly read number of solver threads in rule definition 2024-01-22 09:28:51 +01:00
Fabian Neumann
2183e742b2 add release notes, minor code improvements 2024-01-22 09:18:26 +01:00
Tom Brown
d98ad95332 move building of distribution of existing heating to own script
This makes the distribution of existing heating to urban/rural,
residential/services and spatially more transparent.
2024-01-22 08:01:31 +01:00
Tom Brown
1a477d6b32 move calculation of district heating share to its own script
Now the script build_district_heat_share.py does what the old function
create_nodes_for_heating() in prepare_sector_networks.py did.

There is no need to build nodes lists for each heating sector, since
all nodes have district heating now.
2024-01-22 08:01:31 +01:00
Tom Brown
bd8a5ecf2b build_energy_totals: output district heat share to separate file 2024-01-22 08:01:31 +01:00
Tom Brown
6c20ce83d7 move building of daily heat profile to its own script
Previously this was handled inside prepare_sector_network.py.
2024-01-22 08:01:31 +01:00
pre-commit-ci[bot]
58b012e03d [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-01-19 16:00:21 +00:00
Fabian Neumann
9b3e68a6a8 correctly retrieve solver threads and add back memory logfile 2024-01-19 16:58:05 +01:00
pre-commit-ci[bot]
68421e6aab [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-01-15 18:14:34 +00:00
Philipp Glaum
e9873e7471 Merge remote-tracking branch 'upstream/master' into year-specific-techs 2024-01-11 17:25:50 +01:00
Philipp Glaum
a4802b7018 initial implementation tech specific renewable profiles 2024-01-11 17:23:25 +01:00
euronion
01f201743d Implement everywhere_powerplants 2024-01-08 09:30:21 +01:00
Fabian Neumann
f68e8d3d61
Merge branch 'master' into wildcard-opts-config 2024-01-05 12:02:11 +01:00
pre-commit-ci[bot]
7b1600164f [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-01-04 19:19:02 +00:00
Fabian Neumann
9d0040912b retrieve.smk: simplify gas infrastructure retrieve rules 2024-01-04 19:14:11 +01:00
Fabian Neumann
62bcded800 build_sector.smk: simplify gas infrastructure rules 2024-01-04 19:12:04 +01:00
Fabian Neumann
c3bcaee1a2 common.smk: find _helpers.py also if pypsa-eur is used as module 2024-01-04 13:37:35 +01:00
Fabian Neumann
2096131b03
Merge pull request #842 from PyPSA/luisa-2
add option to use LUISA land coverage data
2024-01-04 09:32:55 +01:00
pre-commit-ci[bot]
60493fc558 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-01-04 08:01:03 +00:00
Fabian Neumann
d145758fb7 gracefully handle absent extra_functionality file; add file to path 2024-01-04 09:00:31 +01:00
Fabian Neumann
b4fb395158 Merge branch 'master' into custom-extra-functionality 2024-01-04 08:32:29 +01:00
Fabian Neumann
c3bb41c900 Merge branch 'master' into multiyear 2024-01-03 15:14:27 +01:00
pre-commit-ci[bot]
045eeba4cf [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-01-03 14:02:26 +00:00
Fabian Neumann
ecd85d23d3 add option to use LUISA land coverage data 2024-01-03 15:01:43 +01:00
Fabian Neumann
83ecaac4de
Merge branch 'master' into wildcard-opts-config 2024-01-03 13:08:21 +01:00
Fabian Neumann
19b503d758 retrieve.smk: add scigrid storages to files of interest 2024-01-03 08:35:00 +01:00
Fabian Neumann
7c058f1ed3 add locations, capacities and costs of existing gas storage 2024-01-03 07:57:45 +01:00
Fabian Neumann
fa03c61187 gas_input: switch production data from scigrid to gem 2024-01-03 07:56:26 +01:00
Fabian Neumann
fac257ca97
Merge pull request #821 from PyPSA/md5-checksums
validate checksums for zenodo downloads
2024-01-02 17:23:06 +01:00
Fabian Neumann
f28e9b47d2 add custom_extra_functionality param to solve_electricity rule 2024-01-02 16:00:41 +01:00
Fabian Neumann
19a50fb8c5 rules: add keep_local=True back in for storage providers 2023-12-31 14:07:45 +01:00
Fabian Neumann
48850431fb Snakefile: switch to new storage providers 2023-12-30 14:37:45 +01:00
Fabian Neumann
a5ba2565a0 correct config location 2023-12-29 17:30:03 +01:00
Fabian Neumann
8a11bdb4b1 solve_network: option to inject custom extra functionalities from source file 2023-12-29 17:19:19 +01:00
pre-commit-ci[bot]
de3b6c9573 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-12-29 11:38:43 +00:00
Fabian Neumann
71985d5e3a validate checksums for zenodo downloads 2023-12-29 12:34:14 +01:00
Fabian Neumann
02b0996dac remove write-protection on resources/natura.tiff 2023-12-29 11:29:19 +01:00
Fabian Neumann
4f815a9cd7 move wdpa and glc datasets to data directory and protect 2023-12-29 11:29:01 +01:00
Fabian Neumann
accffe7bc2 wdpa time-invariant filenames in build_electricity.smk 2023-12-29 11:21:24 +01:00
Fabian Neumann
c5f48f957d wdpa: give snakemake appearance of time-invariant wdpa files 2023-12-29 11:20:40 +01:00
Fabian Neumann
471c97d499 wdpa: correct _shp url 2023-12-29 11:20:14 +01:00
pre-commit-ci[bot]
fec641ce0a [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-12-22 13:41:25 +00:00
euronion
d817212896 Fix missing bYYYY 2023-12-22 14:40:30 +01:00
pre-commit-ci[bot]
3a474af71f [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-12-21 17:40:48 +00:00
euronion
25c5751565 Check WDPA url also a month forward 2023-12-21 18:39:06 +01:00
Fabian
4f67160993 fix snake format 2023-12-18 11:57:21 +01:00
Fabian Neumann
ea7a66b873
Merge pull request #794 from PyPSA/mv-focus-weights
move focus_weights into clustering configuration segment
2023-12-11 20:10:46 +01:00
Fabian Neumann
938637013b
Merge pull request #801 from PyPSA/enspreso-zenodo
change JRC-ENSPRESO biomass data link to zenodo for enhanced reliability
2023-12-11 20:05:13 +01:00
Fabian Neumann
df40fdf904 Merge branch 'master' into ukraine_hackathon 2023-12-11 16:24:40 +01:00
Fabian Neumann
67438f3292 change JRC-ENSPRESO biomass data link to zenodo for enhanced reliability 2023-12-11 16:03:45 +01:00
pre-commit-ci[bot]
01c5e11f9f [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-12-06 09:10:07 +00:00
Fabian Neumann
e5bdb4796a move focus_weights into clustering configuration segment (backward compatible) 2023-12-06 10:09:17 +01:00
Fabian Neumann
4df6370198 update environment: matplotlib, xarray, pypsa 2023-12-04 14:39:23 +01:00
Fabian Neumann
803b860923 move load_raw.csv to resources since it's config file dependent 2023-11-29 13:41:26 +01:00
Fabian Neumann
7ca769bcf0 option to skip ship_raster download if not added as exclusion zone 2023-11-29 12:09:39 +01:00
Irieo
75f8a1b313 scripts: remove redundant rule reference 2023-11-16 17:47:48 +01:00
Irieo
d2c1f9ec32 data: solve floating url problem for WDPA data retrieval 2023-11-03 14:14:01 +01:00
Fabian Neumann
df5b9e9943
Merge pull request #759 from ekatef/fix_retrofit
Update calculations of retrofit costs
2023-10-18 10:26:32 +02:00
ekatef
f88ce0ea07 Adjust data structure 2023-10-13 14:51:25 +03:00
pre-commit-ci[bot]
d10779ca34 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-10-11 14:27:25 +00:00
Thomas Gilon
4c5ea9e207
Specify input files required to retrieve irena data
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2023-10-11 16:27:08 +02:00
Thomas Gilon
c8fcbaad6e
Update default value in retrieve.smk
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2023-10-11 16:24:52 +02:00
pre-commit-ci[bot]
230c1a327c [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-10-11 12:27:29 +00:00
Thomas Gilon
43e20a26e6 Add rule to update IRENA data for existing infrastructure 2023-10-11 14:26:04 +02:00
pre-commit-ci[bot]
326aeb682b [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-10-09 14:28:08 +00:00
Fabian Neumann
091584bb0d add download rules for WDPA land and marine 2023-10-09 16:22:27 +02:00
Fabian Neumann
42692c3ca5 UA-MD availability matrix: plot shape_availability 2023-10-09 16:21:04 +02:00
Fabian Neumann
27ebbaba24 UA-MD availability matrix: consider WDPA protection areas as substitute for Natura 2023-10-09 16:20:23 +02:00
Fabian Neumann
9d7b250230 UA-MD availability matrix: consider ship density 2023-10-09 16:19:31 +02:00
Fabian Neumann
ea99f3c43e Merge branch 'master' into ukraine_hackathon 2023-10-09 09:53:42 +02:00
pre-commit-ci[bot]
7e290d3e52 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-10-08 09:55:38 +00:00
Fabian Neumann
b8fee80919 Merge branch 'master' into multiyear 2023-10-08 11:55:11 +02:00
lisazeyen
21d1109853
Merge branch 'master' into multi-horizon 2023-09-29 08:40:05 +02:00
virio-andreyana
3f112534a5
Merge branch 'master' into wildcard-opts-config 2023-09-28 21:12:10 +02:00
pre-commit-ci[bot]
9424ec1bb8 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-09-26 13:13:25 +00:00
Fabian Neumann
5006bc4877 build_biomass_potentials: link to planning_horizons wildcard, interpolate missing years 2023-09-26 15:12:39 +02:00
virio-andreyana
5cbc420775
Merge branch 'master' into wildcard-opts-config 2023-09-23 19:00:14 +02:00
virio-andreyana
01cd1bbb74 add Fabians suggestion 2023-09-23 18:39:11 +02:00
Fabian
f96c2d0589 update solve smk files to properly consider scenanrios 2023-09-22 13:58:08 +02:00
chrstphtrs
8b5476e273 Move all rule to main Snakefile 2023-09-20 19:33:44 +02:00
pre-commit-ci[bot]
668ec9efad [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-09-15 09:56:18 +00:00
virio-andreyana
7be8cc0773 undo changes in sector networks and simplify config 2023-09-15 11:55:52 +02:00
pre-commit-ci[bot]
3eed341044 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-09-11 21:04:00 +00:00
virio-andreyana
823df52309 add backward compatible config in wildcards 2023-09-11 22:51:31 +02:00
lisazeyen
c0152c25a4 first fixes for CI 2023-08-30 15:57:47 +02:00
pre-commit-ci[bot]
bdaa646ed6 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-08-30 10:05:18 +00:00
lisazeyen
9ec7442c69 merge master 2023-08-30 12:03:49 +02:00
lisazeyen
be23b5c56a add network plot for perfect 2023-08-24 15:32:23 +02:00
lisazeyen
f2b01fbbe0 plot summary with usual script 2023-08-24 14:43:36 +02:00
Fabian
04a2bea176 scenario-management: use list for including wildcards with shared resource 2023-08-24 13:20:11 +02:00
pre-commit-ci[bot]
b8dee184a9 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-08-24 09:12:07 +00:00
Fabian Neumann
d0f2d52853 apply protected() to zenodo downloads; only retrieve monthly fuel/CO2 prices when necessary 2023-08-24 11:11:33 +02:00
Fabian Hofmann
3f3f752e8f
Merge branch 'master' into scenario-management 2023-08-24 10:19:14 +02:00
Fabian
62c6143863 scenario-management: reenable shared resources, make shared resources wildcards dependent 2023-08-23 17:14:57 +02:00
lisazeyen
a21b1f2d10 add params 2023-08-23 13:23:27 +02:00
lisazeyen
c629476721 adapt to new config path 2023-08-22 11:01:51 +02:00
lisazeyen
e5673731ab make plot_network conditional 2023-08-22 11:01:00 +02:00
lisazeyen
e92fc4ff4d merge master 2023-08-22 10:21:42 +02:00
Fabian
ebb5da655e scenarios: add collect function plot_elec_networks 2023-08-17 12:56:34 +02:00
Fabian
91eff472a7 scenario-management: fix set_scenario_config function; apply config_provider to some direct inputs 2023-08-17 12:31:07 +02:00
Fabian
9d4ce430cc electricity demand: remove powerstastics flag, merge sources in retrieve_electricity_demand 2023-08-17 10:17:12 +02:00
Fabian
32eb114b61 retrieve electricity demand: make online version snakemake params 2023-08-16 13:05:22 +02:00
Fabian
9cd449cf3c retrieve electricity demand: use script in order to concat time-series 2023-08-16 11:41:48 +02:00
Fabian
750b74db46 build_electricity: optionally download dynamic fuel prices 2023-08-16 10:51:03 +02:00
Fabian
b3a6e2c281 common: make cache config_getter, use partial functions 2023-08-16 10:50:09 +02:00
Fabian
3462080b89 ci: adjust scenario test run 2023-08-15 15:46:59 +02:00