Commit Graph

119 Commits

Author SHA1 Message Date
Ebbe Kyhl Gøtske ekg@mpe.au.dk
e34bd9b7fb Change costs year in snakefile and config file 2022-08-31 13:07:43 +02:00
lisazeyen
f683a93f08 adjust plot_summary to work with myopic, remove update config 2022-08-01 16:04:14 +02:00
lisazeyen
f10628930e move snakemake dependencies out for co2limit cb option to work 2022-08-01 15:21:11 +02:00
Fabian Neumann
b3dac6d3c4 subworkflow flags amend 2022-07-20 15:53:57 +02:00
Fabian Neumann
10243398e7 do not use subworkflow flags for pure data#
causes problems when config retrieve_databundle is disabled
2022-07-20 15:47:47 +02:00
Fabian Neumann
d4fd472762
Merge branch 'master' into transport 2022-04-12 14:48:57 +02:00
Fabian Neumann
6ed92475c9
Add basic CI for overnight and myopic (#234)
* Snakefile: add missing folders that are retrieved

* Snakefile: fix syntax error

* Add basic CI

* fix datafiles list to work with directories

* prepare: only add new gas pipes if augmentation not empty

* prepare: handle case where salt caverns dataframe is empty

* prepare: handle case where underwater faction not in link columns

* build_biomass: handle case where Balkan not in country list

* refer correct environment path

* fix Snakemake call

* make subworkflow dependency explicit again

* hashing pypsa-eur environment file does not seem to work

* change solver to glpk

* add data cache

* glpk without options

* try cbc

* alternative snakemake test workflow

* test: add exogenous pathways to config
2022-04-11 17:08:25 +02:00
Fabian Neumann
b112da0565 prepare: separate code for transport demand and nodal energy totals 2022-04-03 18:49:35 +02:00
Fabian Neumann
a2459881ef
allow H2 retrofitting from CH4 pipelines without endogenous CH4 grid (#204)
* allow H2 retrofitting from CH4 pipelines without endogenous CH4 grid

* add small capital cost to gas pipelines to incentivise decommissioning

* add release notes
2021-11-29 12:42:10 +01:00
Fabian Neumann
d04f6c02a7
Merge branch 'master' into retrofit-gas-pipelines 2021-11-29 09:13:51 +01:00
Fabian Neumann
263628fa22
add option to automatically download and extract sector databundle (#198) 2021-11-29 09:12:55 +01:00
Fabian Neumann
3ecb761b57
add regionalised hydrogen salt cavern storage potentials (#191)
* add regionalised hydrogen salt cavern storage potentials

* fix reading in salt-cavern potentials and typo in imports

* by default disable nearshore and offshore salt cavern potentials

Co-authored-by: lisazeyen <lisa.zeyen@web.de>
2021-11-29 09:12:07 +01:00
Fabian Neumann
5be6c24fe7 solve_network: take threads from solver setting 2021-11-24 10:33:46 +01:00
Koen van Greevenbroek
04e62fd52e Create config file from default if not found
Similarly to pypsa-eur, the config file is not found, we create it as
a copy of `config.default.yaml`. This also makes it easier to include
pypsa-eur-sec as a subworkflow in a snakemake workflow.
2021-11-22 15:00:42 +01:00
Fabian Neumann
b8a037026d gas_input: rewrite to not require pygeos via sjoin_nearest() 2021-11-14 16:51:34 +01:00
Fabian Neumann
4cd964b3bd gas_network: use IGGIELGN scigrid dataset 2021-11-13 16:48:08 +01:00
Fabian Neumann
b39a852d3b correct syntax errors 2021-11-12 12:50:46 +01:00
Fabian Neumann
12ef1302e6 add planned LNG terminals from gem.wiki and convert units to MW 2021-11-10 18:05:47 +01:00
Fabian Neumann
57c0dd29d1 disentangle gas input nodes by lng, prod, entry with mmc/d# 2021-11-09 14:52:08 +01:00
Fabian Neumann
985705403e revision gas infrastructure representation 2 2021-11-04 21:48:54 +01:00
Fabian Neumann
6a00d5bfca revision gas infrastructure representation 2021-11-03 20:34:43 +01:00
Fabian Neumann
37e2e53486
Merge branch 'master' into retrofit-gas-pipelines 2021-11-02 19:03:26 +01:00
Fabian Neumann
2e6e9c6802
Merge branch 'master' into agriculture-energy-co2 2021-10-02 10:51:28 +02:00
Fabian Neumann
90079248a3
Merge branch 'master' into dh-share 2021-09-30 14:48:26 +02:00
lisazeyen
8322350fee move calculation of district heating share to build_energy_totals 2021-09-29 14:36:56 +02:00
Fabian Neumann
8ecc54fbe6
Merge branch 'master' into dh-share 2021-09-28 17:29:29 +02:00
lisazeyen
dadfa8691b update mocksnakemake for testing 2021-09-28 16:36:40 +02:00
Fabian Neumann
8e3ef7f86b remove unused sector wildcard constraints 2021-08-16 16:29:05 +02:00
Fabian Neumann
ad23a648c4
Merge branch 'master' into enspreso-biomass 2021-08-10 10:43:52 +02:00
Fabian Neumann
bab51b0f54 fix read-in of pypsa-eur data 2021-08-10 10:28:22 +02:00
Fabian Neumann
7ca4f04611 biomass_transport: reduce output files 2021-08-09 17:53:14 +02:00
Fabian Neumann
6711d721b9 biomass_transport: fix cost calculation and get from remote 2021-08-09 16:30:38 +02:00
Fabian Neumann
d428c1b77d fix syntax and auto-retrieve biomass transport cost 2021-08-09 14:43:51 +02:00
Fabian Neumann
0da8f8f80e
Merge branch 'master' into biomass-transport 2021-08-06 15:58:23 +02:00
martavp
fab31e6524
Exogenous transition path for shipping, Steel, and Aluminum production (#136)
* Update .gitignore

* Add fictitious load to account for non-transformed shipping emissions

The share of shipping demand that is transformed is defined now for different years to be used with the myopic code.
The carbon emission from the remaining share is treated as a negative load on the atmospheric carbon dioxide bus, just like aviation and land transport emissions.

* Split colours for H2 in Industry and H2 in shipping when plotting balances.

When plotting the balance for H2, the rename dictionary merges all the demands containing H2.
This commit disables such merging and keeps different colours for H2 in shipping and H2 in industry. This is useful when one wants to look at the H2 balance and have an overview of where the H2 is consumed in the model.

* Make transformation of Steel and Aluminum production depends on year

Previously, the transformation of the Steel and Aluminum production was assumed to occur overnight.
This commit enables the definition of a transformation path via the config.yaml file.
This requires adding the {planning_horizon} to the input and output file name of the following rules:
build_industrial_production_per_country_tomorrow
build_industrial_production_per_node
build_industry_energy_demand_per_node
prepare_sector_network

* small follow-up to merge

* Add oil consumed in shipping as a load to EU oil bus

* Update scripts/prepare_sector_network.py

* add planning_horizons wildcard to benchmark paths

* fixup: double fraction_primary for steel

Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2021-08-04 18:19:02 +02:00
Fabian Neumann
ba9be74265 make build_biomass_transport_costs fully optional 2021-08-04 15:05:37 +02:00
Fabian Neumann
2563d1277a spatially-explicit biomass potentials from ENSPRESO (NUTS2) 2021-07-11 17:52:32 +02:00
lisazeyen
2e336e5e70 add missing wildcards for benchmarks, add new input data for district heating share 2021-07-08 14:41:13 +02:00
Fabian Neumann
732046dba0 merge master 2021-07-03 12:46:09 +02:00
Fabian Neumann
89eccd4f70 merge master 2021-07-02 15:00:19 +02:00
Fabian Neumann
fd1121af4a add rule for build_biomass_transport_costs 2021-07-02 11:07:35 +02:00
Fabian Neumann
e97c4dd3c0 merge master 2021-07-02 10:12:43 +02:00
Fabian Neumann
1fc1d2a17d
Revision complete (#139)
* ammonia_production: minor cleaning and move into __main__ (#106)

* biomass_potentials: code cleaning and automatic country index inferral (#107)

* Revision: build energy totals (#111)

* blacken

* energy_totals: preliminaries

* energy_totals: update build_swiss

* energy_totals: update build_eurostat

* energy_totals: update build_idees

* energy_totals: update build_energy_totals

* energy_totals: update build_eea_co2

* energy_totals: update build_eurostat_co2

* energy_totals: update build_co2_totals

* energy_totals: update build_transport_data

* energy_totals: add tqdm progressbar to idees

* energy_totals: adjust __main__ section

* energy_totals: handle inputs via Snakefile and config

* energy_totals: handle data and emissions year via config

* energy_totals: fix reading in eurostat for different years

* energy_totals: fix erroneous drop duplicates
This caused problems for waste management in HU and SI

* energy_totals: make scope selection of CO2 or GHG a config option

* Revision: build industrial production per country (#114)

* industry-ppc: format

* industry-ppc: rewrite for performance

* industry-ppc: move reference year to config

* industry-ppct: tidy up and format (#115)

* remove stale industry demand rules (#116)

* industry-epc: rewrite for performance (#117)

* Revision: industrial distribution key (#118)

* industry-distribution: first tidying

* industry-distribution: first tidying

* industry-distribution: fix syntax

* Revision: industrial energy demand per node today (#119)

* industry-epn: minor code cleaning

* industry-epn: remove accidental artifact

* industry-epn: remove accidental artifact II

* industry-ppn: code cleaning (#120)

* minor code cleaning (#121)

* Revision: industry sector ratios (#122)

* sector-ratios: basic reformatting

* sector-ratios: add new read_excel function that filters year already

* sector-ratios: rename jrc to idees

* sector-ratios: rename conv_factor to toe_to_MWh

* sector-ratios: modularise into functions

* Move overriding of component attributes to function and into data (#123)

* move overriding of component attributes to central function and store in separate folder

* fix return of helper.override_component_attrs

* prepare: fix accidental syntax error

* override_component_attrs: bugfix that aligns with pypsa components

* Revision: build population layout (#108)

* population_layouts: move inside __main__ and blacken

* population_layouts: misc code cleaning and multiprocessing

* population_layouts: fix fill_values assignment of urban fractions

* population_layouts: bugfig for UK-GB naming ambiguity

* population_layouts: sort countries alphabetically for better overview

* config: change path to atlite cutout

* Revision: build clustered population layouts (#112)

* population_layouts: move inside __main__ and blacken

* population_layouts: misc code cleaning and multiprocessing

* population_layouts: fix fill_values assignment of urban fractions

* population_layouts: bugfig for UK-GB naming ambiguity

* population_layouts: sort countries alphabetically for better overview

* cl_pop_layout: blacken

* cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning

* cl_pop_layout: add fraction column which is repeatedly calculated downstream

* Revision: build various heating-related time series (#113)

* population_layouts: move inside __main__ and blacken

* population_layouts: misc code cleaning and multiprocessing

* population_layouts: fix fill_values assignment of urban fractions

* population_layouts: bugfig for UK-GB naming ambiguity

* population_layouts: sort countries alphabetically for better overview

* cl_pop_layout: blacken

* cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning

* gitignore: add .vscode

* heating_profiles: update to new atlite and move into __main__

* heating_profiles: remove extra cutout

* heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries

* heating_profiles: load regions with .buffer(0) before squeeze()

* heating_profiles: account for transpose of dataarray

* heating_profiles: account for transpose of dataarray in add_exiting_baseyear

* Reduce verbosity of Snakefile (2) (#128)

* tidy Snakefile light

* Snakefile: fix indents

* Snakefile: add missing RDIR

* tidy config by removing quotes and expanding lists (#109)

* bugfix: reorder squeeze() and buffer()

* plot/summary: cosmetic changes including: (#131)

- matplotlibrc for default style and backend
- remove unused config options
- option to configure geomap colors
- option to configure geomap bounds

* solve: align with pypsa-eur using ilopf (#129)

* tidy myopic code scripts (#132)

* use mock_snakemake from pypsa-eur (#133)

* Snakefile: add benchmark files to each rule

* Snakefile: only run build_retro_cost if endogenously optimised

* Snakefile: remove old {network} wildcard constraints

* WIP: Revision: prepare_sector_network (#124)

* population_layouts: move inside __main__ and blacken

* population_layouts: misc code cleaning and multiprocessing

* population_layouts: fix fill_values assignment of urban fractions

* population_layouts: bugfig for UK-GB naming ambiguity

* population_layouts: sort countries alphabetically for better overview

* cl_pop_layout: blacken

* cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning

* move overriding of component attributes to central function and store in separate folder

* prepare: sort imports and remove six dependency

* prepare: remove add_emission_prices

* prepare: remove unused set_line_s_max_pu
This is a function from prepare_network

* prepare: remove unused set_line_volume_limit
This is a PyPSA-Eur function from prepare_network

* prepare: tidy add_co2limit

* remove six dependency

* prepare: tidy code first batch

* prepare: extend override_component_attrs to avoid hacky madd

* prepare: remove hacky madd() for individual components

* prepare: tidy shift function

* prepare: nodes and countries from n.buses not pop_layout

* prepare: tidy loading of pop_layout

* prepare: fix prepare_costs function

* prepare: optimise loading of traffic data

* prepare: move localizer into generate_periodic profiles

* prepare: some formatting of transport data

* prepare: eliminate some code duplication

* prepare: fix remove_h2_network
- only try to remove EU H2 store if it exists
- remove readding nodal Stores because they are never removed

* prepare: move cost adjustment to own function

* prepare: fix a syntax error

* prepare: add investment_year to get() assuming global variable

* prepare: move co2_totals out of prepare_data()

* Snakefile: remove unused prepare_sector_network inputs

* prepare: move limit p/s_nom of lines/links into function

* prepare: tidy add_co2limit file handling

* Snakefile: fix tabs

* override_component_attrs: add n/a defaults

* README: Add network picture to make scope clear

* README: Fix date of preprint (was too optimistic...)

* prepare: move some more config options to config.yaml

* prepare: runtime bugfixes

* fix benchmark path

* adjust plot ylims

* add unit attribute to bus, correct cement capture efficiency

* bugfix: land usage constrained missed inplace operation

Co-authored-by: Tom Brown <tom@nworbmot.org>

* add release notes

* remove old fix_branches() function

* deps: make geopy optional, remove unused imports

* increase default BarConvTol

* get ready for upcoming PyPSA release

* re-remove ** bug

* amend release notes

Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 20:09:04 +02:00
lisazeyen
17a1153648 comment functions, remove not used input 2021-06-21 12:34:08 +02:00
martavp
0a92d23698 Make transformation of Steel and Aluminum production depends on year
Previously, the transformation of the Steel and Aluminum production was assumed to occur overnight.
This commit enables the definition of a transformation path via the config.yaml file.
This requires adding the {planning_horizon} to the input and output file name of the following rules:
build_industrial_production_per_country_tomorrow
build_industrial_production_per_node
build_industry_energy_demand_per_node
prepare_sector_network
2021-06-17 16:13:05 +02:00
lisazeyen
feefaa3d41 Merge branch 'master' into retrofit-gas-pipelines 2021-06-17 06:38:47 +02:00
Fabian Neumann
555002a35b fixup: remove remaining instances of {network} wildcard from recent PR 2021-05-19 13:57:44 +02:00
lisazeyen
33f7f5026e Merge branch 'master' into retrofit-gas-pipelines 2021-05-12 15:20:49 +02:00
Tom Brown
8bfb1490c1
Merge branch 'master' into retro-updated 2021-04-30 18:12:20 +02:00
Fabian Neumann
a156d44f28
remove unused {network} wildcard (#104) 2021-04-29 16:52:35 +02:00