pypsa-eur/Snakefile

532 lines
24 KiB
Plaintext
Raw Normal View History

configfile: "config.yaml"
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
wildcard_constraints:
lv="[a-z0-9\.]+",
simpl="[a-zA-Z0-9]*",
clusters="[0-9]+m?",
sectors="[+a-zA-Z0-9]+",
opts="[-+a-zA-Z0-9]*",
sector_opts="[-+a-zA-Z0-9\.\s]*"
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
SDIR = config['summary_dir'] + '/' + config['run']
RDIR = config['results_dir'] + config['run']
CDIR = config['costs_dir']
subworkflow pypsaeur:
workdir: "../pypsa-eur"
snakefile: "../pypsa-eur/Snakefile"
configfile: "../pypsa-eur/config.yaml"
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
rule all:
input: SDIR + '/graphs/costs.pdf'
rule solve_all_networks:
input:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
expand(RDIR + "/postnetworks/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}.nc",
**config['scenario'])
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
rule prepare_sector_networks:
input:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
expand(RDIR + "/prenetworks/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}.nc",
**config['scenario'])
rule build_population_layouts:
input:
nuts3_shapes=pypsaeur('resources/nuts3_shapes.geojson'),
urban_percent="data/urban_percent.csv"
output:
pop_layout_total="resources/pop_layout_total.nc",
pop_layout_urban="resources/pop_layout_urban.nc",
pop_layout_rural="resources/pop_layout_rural.nc"
resources: mem_mb=20000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_population_layouts"
threads: 8
script: "scripts/build_population_layouts.py"
rule build_clustered_population_layouts:
input:
pop_layout_total="resources/pop_layout_total.nc",
pop_layout_urban="resources/pop_layout_urban.nc",
pop_layout_rural="resources/pop_layout_rural.nc",
regions_onshore=pypsaeur('resources/regions_onshore_elec_s{simpl}_{clusters}.geojson')
output:
clustered_pop_layout="resources/pop_layout_elec_s{simpl}_{clusters}.csv"
resources: mem_mb=10000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_clustered_population_layouts/s{simpl}_{clusters}"
script: "scripts/build_clustered_population_layouts.py"
rule build_simplified_population_layouts:
input:
pop_layout_total="resources/pop_layout_total.nc",
pop_layout_urban="resources/pop_layout_urban.nc",
pop_layout_rural="resources/pop_layout_rural.nc",
regions_onshore=pypsaeur('resources/regions_onshore_elec_s{simpl}.geojson')
output:
clustered_pop_layout="resources/pop_layout_elec_s{simpl}.csv"
resources: mem_mb=10000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_clustered_population_layouts/s{simpl}"
script: "scripts/build_clustered_population_layouts.py"
rule build_gas_network:
input:
gas_network="data/gas_network/gas_network_dataset.csv",
country_shapes=pypsaeur("resources/country_shapes.geojson"),
regions_onshore=pypsaeur("resources/regions_onshore_elec_s{simpl}_{clusters}.geojson"),
regions_offshore=pypsaeur("resources/regions_offshore_elec_s{simpl}_{clusters}.geojson")
output:
clustered_gas_network="resources/gas_network_elec_s{simpl}_{clusters}.csv"
resources: mem_mb=10000
script: "scripts/build_gas_network.py"
rule build_heat_demands:
input:
pop_layout_total="resources/pop_layout_total.nc",
pop_layout_urban="resources/pop_layout_urban.nc",
pop_layout_rural="resources/pop_layout_rural.nc",
regions_onshore=pypsaeur("resources/regions_onshore_elec_s{simpl}_{clusters}.geojson")
output:
heat_demand_urban="resources/heat_demand_urban_elec_s{simpl}_{clusters}.nc",
heat_demand_rural="resources/heat_demand_rural_elec_s{simpl}_{clusters}.nc",
heat_demand_total="resources/heat_demand_total_elec_s{simpl}_{clusters}.nc"
resources: mem_mb=20000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_heat_demands/s{simpl}_{clusters}"
script: "scripts/build_heat_demand.py"
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
rule build_temperature_profiles:
input:
pop_layout_total="resources/pop_layout_total.nc",
pop_layout_urban="resources/pop_layout_urban.nc",
pop_layout_rural="resources/pop_layout_rural.nc",
regions_onshore=pypsaeur("resources/regions_onshore_elec_s{simpl}_{clusters}.geojson")
output:
temp_soil_total="resources/temp_soil_total_elec_s{simpl}_{clusters}.nc",
temp_soil_rural="resources/temp_soil_rural_elec_s{simpl}_{clusters}.nc",
temp_soil_urban="resources/temp_soil_urban_elec_s{simpl}_{clusters}.nc",
temp_air_total="resources/temp_air_total_elec_s{simpl}_{clusters}.nc",
temp_air_rural="resources/temp_air_rural_elec_s{simpl}_{clusters}.nc",
temp_air_urban="resources/temp_air_urban_elec_s{simpl}_{clusters}.nc"
resources: mem_mb=20000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_temperature_profiles/s{simpl}_{clusters}"
script: "scripts/build_temperature_profiles.py"
rule build_cop_profiles:
input:
temp_soil_total="resources/temp_soil_total_elec_s{simpl}_{clusters}.nc",
temp_soil_rural="resources/temp_soil_rural_elec_s{simpl}_{clusters}.nc",
temp_soil_urban="resources/temp_soil_urban_elec_s{simpl}_{clusters}.nc",
temp_air_total="resources/temp_air_total_elec_s{simpl}_{clusters}.nc",
temp_air_rural="resources/temp_air_rural_elec_s{simpl}_{clusters}.nc",
temp_air_urban="resources/temp_air_urban_elec_s{simpl}_{clusters}.nc"
output:
cop_soil_total="resources/cop_soil_total_elec_s{simpl}_{clusters}.nc",
cop_soil_rural="resources/cop_soil_rural_elec_s{simpl}_{clusters}.nc",
cop_soil_urban="resources/cop_soil_urban_elec_s{simpl}_{clusters}.nc",
cop_air_total="resources/cop_air_total_elec_s{simpl}_{clusters}.nc",
cop_air_rural="resources/cop_air_rural_elec_s{simpl}_{clusters}.nc",
cop_air_urban="resources/cop_air_urban_elec_s{simpl}_{clusters}.nc"
resources: mem_mb=20000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_cop_profiles/s{simpl}_{clusters}"
script: "scripts/build_cop_profiles.py"
rule build_solar_thermal_profiles:
input:
pop_layout_total="resources/pop_layout_total.nc",
pop_layout_urban="resources/pop_layout_urban.nc",
pop_layout_rural="resources/pop_layout_rural.nc",
regions_onshore=pypsaeur("resources/regions_onshore_elec_s{simpl}_{clusters}.geojson")
output:
solar_thermal_total="resources/solar_thermal_total_elec_s{simpl}_{clusters}.nc",
solar_thermal_urban="resources/solar_thermal_urban_elec_s{simpl}_{clusters}.nc",
solar_thermal_rural="resources/solar_thermal_rural_elec_s{simpl}_{clusters}.nc"
resources: mem_mb=20000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_solar_thermal_profiles/s{simpl}_{clusters}"
script: "scripts/build_solar_thermal_profiles.py"
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
def input_eurostat(w):
# 2016 includes BA, 2017 does not
report_year = config["energy"]["eurostat_report_year"]
return f"data/eurostat-energy_balances-june_{report_year}_edition"
rule build_energy_totals:
input:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
nuts3_shapes=pypsaeur('resources/nuts3_shapes.geojson'),
co2="data/eea/UNFCCC_v23.csv",
swiss="data/switzerland-sfoe/switzerland-new_format.csv",
idees="data/jrc-idees-2015",
eurostat=input_eurostat
output:
energy_name='resources/energy_totals.csv',
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
co2_name='resources/co2_totals.csv',
transport_name='resources/transport_data.csv'
threads: 16
resources: mem_mb=10000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_energy_totals"
script: 'scripts/build_energy_totals.py'
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
rule build_biomass_potentials:
input:
jrc_potentials="data/biomass/JRC Biomass Potentials.xlsx"
output:
biomass_potentials_all='resources/biomass_potentials_all.csv',
biomass_potentials='resources/biomass_potentials.csv'
threads: 1
resources: mem_mb=1000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_biomass_potentials"
script: 'scripts/build_biomass_potentials.py'
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
rule build_ammonia_production:
input:
usgs="data/myb1-2017-nitro.xls"
output:
ammonia_production="resources/ammonia_production.csv"
threads: 1
resources: mem_mb=1000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_ammonia_production"
script: 'scripts/build_ammonia_production.py'
rule build_industry_sector_ratios:
input:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
ammonia_production="resources/ammonia_production.csv",
idees="data/jrc-idees-2015"
output:
industry_sector_ratios="resources/industry_sector_ratios.csv"
threads: 1
resources: mem_mb=1000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_industry_sector_ratios"
script: 'scripts/build_industry_sector_ratios.py'
rule build_industrial_production_per_country:
input:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
ammonia_production="resources/ammonia_production.csv",
jrc="data/jrc-idees-2015",
eurostat="data/eurostat-energy_balances-may_2018_edition",
output:
industrial_production_per_country="resources/industrial_production_per_country.csv"
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
threads: 8
resources: mem_mb=1000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_industrial_production_per_country"
script: 'scripts/build_industrial_production_per_country.py'
rule build_industrial_production_per_country_tomorrow:
input:
industrial_production_per_country="resources/industrial_production_per_country.csv"
output:
industrial_production_per_country_tomorrow="resources/industrial_production_per_country_tomorrow.csv"
threads: 1
resources: mem_mb=1000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_industrial_production_per_country_tomorrow"
script: 'scripts/build_industrial_production_per_country_tomorrow.py'
rule build_industrial_distribution_key:
input:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
regions_onshore=pypsaeur('resources/regions_onshore_elec_s{simpl}_{clusters}.geojson'),
clustered_pop_layout="resources/pop_layout_elec_s{simpl}_{clusters}.csv",
hotmaps_industrial_database="data/Industrial_Database.csv",
output:
industrial_distribution_key="resources/industrial_distribution_key_elec_s{simpl}_{clusters}.csv"
threads: 1
resources: mem_mb=1000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_industrial_distribution_key/s{simpl}_{clusters}"
script: 'scripts/build_industrial_distribution_key.py'
rule build_industrial_production_per_node:
input:
industrial_distribution_key="resources/industrial_distribution_key_elec_s{simpl}_{clusters}.csv",
industrial_production_per_country_tomorrow="resources/industrial_production_per_country_tomorrow.csv"
output:
industrial_production_per_node="resources/industrial_production_elec_s{simpl}_{clusters}.csv"
threads: 1
resources: mem_mb=1000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_industrial_production_per_node/s{simpl}_{clusters}"
script: 'scripts/build_industrial_production_per_node.py'
rule build_industrial_energy_demand_per_node:
input:
industry_sector_ratios="resources/industry_sector_ratios.csv",
industrial_production_per_node="resources/industrial_production_elec_s{simpl}_{clusters}.csv",
industrial_energy_demand_per_node_today="resources/industrial_energy_demand_today_elec_s{simpl}_{clusters}.csv"
output:
industrial_energy_demand_per_node="resources/industrial_energy_demand_elec_s{simpl}_{clusters}.csv"
threads: 1
resources: mem_mb=1000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_industrial_energy_demand_per_node/s{simpl}_{clusters}"
script: 'scripts/build_industrial_energy_demand_per_node.py'
rule build_industrial_energy_demand_per_country_today:
input:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
jrc="data/jrc-idees-2015",
ammonia_production="resources/ammonia_production.csv",
industrial_production_per_country="resources/industrial_production_per_country.csv"
output:
industrial_energy_demand_per_country_today="resources/industrial_energy_demand_per_country_today.csv"
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
threads: 8
resources: mem_mb=1000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_industrial_energy_demand_per_country_today"
script: 'scripts/build_industrial_energy_demand_per_country_today.py'
rule build_industrial_energy_demand_per_node_today:
input:
industrial_distribution_key="resources/industrial_distribution_key_elec_s{simpl}_{clusters}.csv",
industrial_energy_demand_per_country_today="resources/industrial_energy_demand_per_country_today.csv"
output:
industrial_energy_demand_per_node_today="resources/industrial_energy_demand_today_elec_s{simpl}_{clusters}.csv"
threads: 1
resources: mem_mb=1000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: "benchmarks/build_industrial_energy_demand_per_node_today/s{simpl}_{clusters}"
script: 'scripts/build_industrial_energy_demand_per_node_today.py'
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
if config["sector"]["retrofitting"]["retro_endogen"]:
rule build_retro_cost:
input:
building_stock="data/retro/data_building_stock.csv",
data_tabula="data/retro/tabula-calculator-calcsetbuilding.csv",
air_temperature = "resources/temp_air_total_elec_s{simpl}_{clusters}.nc",
u_values_PL="data/retro/u_values_poland.csv",
tax_w="data/retro/electricity_taxes_eu.csv",
construction_index="data/retro/comparative_level_investment.csv",
floor_area_missing="data/retro/floor_area_missing.csv",
clustered_pop_layout="resources/pop_layout_elec_s{simpl}_{clusters}.csv",
cost_germany="data/retro/retro_cost_germany.csv",
window_assumptions="data/retro/window_assumptions.csv",
output:
retro_cost="resources/retro_cost_elec_s{simpl}_{clusters}.csv",
floor_area="resources/floor_area_elec_s{simpl}_{clusters}.csv"
resources: mem_mb=1000
benchmark: "benchmarks/build_retro_cost/s{simpl}_{clusters}"
script: "scripts/build_retro_cost.py"
build_retro_cost_output = rules.build_retro_cost.output
else:
build_retro_cost_output = {}
rule prepare_sector_network:
input:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
overrides="data/override_component_attrs",
network=pypsaeur('networks/elec_s{simpl}_{clusters}_ec_lv{lv}_{opts}.nc'),
energy_totals_name='resources/energy_totals.csv',
co2_totals_name='resources/co2_totals.csv',
transport_name='resources/transport_data.csv',
clustered_gas_network="resources/gas_network_elec_s{simpl}_{clusters}.csv",
2021-07-02 13:00:19 +00:00
traffic_data_KFZ="data/emobility/KFZ__count",
traffic_data_Pkw="data/emobility/Pkw__count",
biomass_potentials='resources/biomass_potentials.csv',
heat_profile="data/heat_load_profile_BDEW.csv",
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
costs=CDIR + "costs_{planning_horizons}.csv",
profile_offwind_ac=pypsaeur("resources/profile_offwind-ac.nc"),
profile_offwind_dc=pypsaeur("resources/profile_offwind-dc.nc"),
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
h2_cavern="data/hydrogen_salt_cavern_potentials.csv",
busmap_s=pypsaeur("resources/busmap_elec_s{simpl}.csv"),
busmap=pypsaeur("resources/busmap_elec_s{simpl}_{clusters}.csv"),
clustered_pop_layout="resources/pop_layout_elec_s{simpl}_{clusters}.csv",
simplified_pop_layout="resources/pop_layout_elec_s{simpl}.csv",
industrial_demand="resources/industrial_energy_demand_elec_s{simpl}_{clusters}.csv",
heat_demand_urban="resources/heat_demand_urban_elec_s{simpl}_{clusters}.nc",
heat_demand_rural="resources/heat_demand_rural_elec_s{simpl}_{clusters}.nc",
heat_demand_total="resources/heat_demand_total_elec_s{simpl}_{clusters}.nc",
temp_soil_total="resources/temp_soil_total_elec_s{simpl}_{clusters}.nc",
temp_soil_rural="resources/temp_soil_rural_elec_s{simpl}_{clusters}.nc",
temp_soil_urban="resources/temp_soil_urban_elec_s{simpl}_{clusters}.nc",
temp_air_total="resources/temp_air_total_elec_s{simpl}_{clusters}.nc",
temp_air_rural="resources/temp_air_rural_elec_s{simpl}_{clusters}.nc",
temp_air_urban="resources/temp_air_urban_elec_s{simpl}_{clusters}.nc",
cop_soil_total="resources/cop_soil_total_elec_s{simpl}_{clusters}.nc",
cop_soil_rural="resources/cop_soil_rural_elec_s{simpl}_{clusters}.nc",
cop_soil_urban="resources/cop_soil_urban_elec_s{simpl}_{clusters}.nc",
cop_air_total="resources/cop_air_total_elec_s{simpl}_{clusters}.nc",
cop_air_rural="resources/cop_air_rural_elec_s{simpl}_{clusters}.nc",
cop_air_urban="resources/cop_air_urban_elec_s{simpl}_{clusters}.nc",
solar_thermal_total="resources/solar_thermal_total_elec_s{simpl}_{clusters}.nc",
solar_thermal_urban="resources/solar_thermal_urban_elec_s{simpl}_{clusters}.nc",
solar_thermal_rural="resources/solar_thermal_rural_elec_s{simpl}_{clusters}.nc",
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
**build_retro_cost_output
output: RDIR + '/prenetworks/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}.nc'
threads: 1
resources: mem_mb=2000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: RDIR + "/benchmarks/prepare_network/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}"
script: "scripts/prepare_sector_network.py"
rule plot_network:
input:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
overrides="data/override_component_attrs",
network=RDIR + "/postnetworks/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}.nc"
output:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
map=RDIR + "/maps/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}-costs-all_{planning_horizons}.pdf",
today=RDIR + "/maps/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}-today.pdf"
threads: 2
resources: mem_mb=10000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: RDIR + "/benchmarks/plot_network/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}"
script: "scripts/plot_network.py"
rule copy_config:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
output: SDIR + '/configs/config.yaml'
threads: 1
resources: mem_mb=1000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: SDIR + "/benchmarks/copy_config"
script: "scripts/copy_config.py"
rule make_summary:
input:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
overrides="data/override_component_attrs",
networks=expand(
RDIR + "/postnetworks/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}.nc",
**config['scenario']
),
costs=CDIR + "costs_{}.csv".format(config['scenario']['planning_horizons'][0]),
plots=expand(
RDIR + "/maps/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}-costs-all_{planning_horizons}.pdf",
**config['scenario']
)
output:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
nodal_costs=SDIR + '/csvs/nodal_costs.csv',
nodal_capacities=SDIR + '/csvs/nodal_capacities.csv',
nodal_cfs=SDIR + '/csvs/nodal_cfs.csv',
cfs=SDIR + '/csvs/cfs.csv',
costs=SDIR + '/csvs/costs.csv',
capacities=SDIR + '/csvs/capacities.csv',
curtailment=SDIR + '/csvs/curtailment.csv',
energy=SDIR + '/csvs/energy.csv',
supply=SDIR + '/csvs/supply.csv',
supply_energy=SDIR + '/csvs/supply_energy.csv',
prices=SDIR + '/csvs/prices.csv',
weighted_prices=SDIR + '/csvs/weighted_prices.csv',
market_values=SDIR + '/csvs/market_values.csv',
price_statistics=SDIR + '/csvs/price_statistics.csv',
metrics=SDIR + '/csvs/metrics.csv'
threads: 2
resources: mem_mb=10000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: SDIR + "/benchmarks/make_summary"
script: "scripts/make_summary.py"
rule plot_summary:
input:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
costs=SDIR + '/csvs/costs.csv',
energy=SDIR + '/csvs/energy.csv',
balances=SDIR + '/csvs/supply_energy.csv'
output:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
costs=SDIR + '/graphs/costs.pdf',
energy=SDIR + '/graphs/energy.pdf',
balances=SDIR + '/graphs/balances-energy.pdf'
threads: 2
resources: mem_mb=10000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: SDIR + "/benchmarks/plot_summary"
script: "scripts/plot_summary.py"
if config["foresight"] == "overnight":
rule solve_network:
input:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
overrides="data/override_component_attrs",
network=RDIR + "/prenetworks/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}.nc",
costs=CDIR + "costs_{planning_horizons}.csv",
config=SDIR + '/configs/config.yaml'
output: RDIR + "/postnetworks/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}.nc"
shadow: "shallow"
log:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
solver=RDIR + "/logs/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}_solver.log",
python=RDIR + "/logs/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}_python.log",
memory=RDIR + "/logs/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}_memory.log"
threads: 4
resources: mem_mb=config['solving']['mem']
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: RDIR + "/benchmarks/solve_network/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}"
script: "scripts/solve_network.py"
if config["foresight"] == "myopic":
rule add_existing_baseyear:
input:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
overrides="data/override_component_attrs",
network=RDIR + '/prenetworks/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}.nc',
powerplants=pypsaeur('resources/powerplants.csv'),
busmap_s=pypsaeur("resources/busmap_elec_s{simpl}.csv"),
busmap=pypsaeur("resources/busmap_elec_s{simpl}_{clusters}.csv"),
clustered_pop_layout="resources/pop_layout_elec_s{simpl}_{clusters}.csv",
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
costs=CDIR + "costs_{}.csv".format(config['scenario']['planning_horizons'][0]),
cop_soil_total="resources/cop_soil_total_elec_s{simpl}_{clusters}.nc",
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
cop_air_total="resources/cop_air_total_elec_s{simpl}_{clusters}.nc",
existing_heating='data/existing_infrastructure/existing_heating_raw.csv',
country_codes='data/Country_codes.csv',
existing_solar='data/existing_infrastructure/solar_capacity_IRENA.csv',
existing_onwind='data/existing_infrastructure/onwind_capacity_IRENA.csv',
existing_offwind='data/existing_infrastructure/offwind_capacity_IRENA.csv',
output: RDIR + '/prenetworks-brownfield/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}.nc'
wildcard_constraints:
planning_horizons=config['scenario']['planning_horizons'][0] #only applies to baseyear
threads: 1
resources: mem_mb=2000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: RDIR + '/benchmarks/add_existing_baseyear/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}'
script: "scripts/add_existing_baseyear.py"
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
def solved_previous_horizon(wildcards):
planning_horizons = config["scenario"]["planning_horizons"]
i = planning_horizons.index(int(wildcards.planning_horizons))
planning_horizon_p = str(planning_horizons[i-1])
return RDIR + "/postnetworks/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_" + planning_horizon_p + ".nc"
rule add_brownfield:
input:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
overrides="data/override_component_attrs",
network=RDIR + '/prenetworks/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}.nc',
network_p=solved_previous_horizon, #solved network at previous time step
costs=CDIR + "costs_{planning_horizons}.csv",
cop_soil_total="resources/cop_soil_total_elec_s{simpl}_{clusters}.nc",
cop_air_total="resources/cop_air_total_elec_s{simpl}_{clusters}.nc"
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
output: RDIR + "/prenetworks-brownfield/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}.nc"
threads: 4
resources: mem_mb=10000
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: RDIR + '/benchmarks/add_brownfield/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}'
script: "scripts/add_brownfield.py"
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
ruleorder: add_existing_baseyear > add_brownfield
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
rule solve_network_myopic:
input:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
overrides="data/override_component_attrs",
network=RDIR + "/prenetworks-brownfield/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}.nc",
costs=CDIR + "costs_{planning_horizons}.csv",
config=SDIR + '/configs/config.yaml'
output: RDIR + "/postnetworks/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}.nc"
shadow: "shallow"
log:
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
solver=RDIR + "/logs/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}_solver.log",
python=RDIR + "/logs/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}_python.log",
memory=RDIR + "/logs/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}_memory.log"
threads: 4
resources: mem_mb=config['solving']['mem']
Revision complete (#139) * ammonia_production: minor cleaning and move into __main__ (#106) * biomass_potentials: code cleaning and automatic country index inferral (#107) * Revision: build energy totals (#111) * blacken * energy_totals: preliminaries * energy_totals: update build_swiss * energy_totals: update build_eurostat * energy_totals: update build_idees * energy_totals: update build_energy_totals * energy_totals: update build_eea_co2 * energy_totals: update build_eurostat_co2 * energy_totals: update build_co2_totals * energy_totals: update build_transport_data * energy_totals: add tqdm progressbar to idees * energy_totals: adjust __main__ section * energy_totals: handle inputs via Snakefile and config * energy_totals: handle data and emissions year via config * energy_totals: fix reading in eurostat for different years * energy_totals: fix erroneous drop duplicates This caused problems for waste management in HU and SI * energy_totals: make scope selection of CO2 or GHG a config option * Revision: build industrial production per country (#114) * industry-ppc: format * industry-ppc: rewrite for performance * industry-ppc: move reference year to config * industry-ppct: tidy up and format (#115) * remove stale industry demand rules (#116) * industry-epc: rewrite for performance (#117) * Revision: industrial distribution key (#118) * industry-distribution: first tidying * industry-distribution: first tidying * industry-distribution: fix syntax * Revision: industrial energy demand per node today (#119) * industry-epn: minor code cleaning * industry-epn: remove accidental artifact * industry-epn: remove accidental artifact II * industry-ppn: code cleaning (#120) * minor code cleaning (#121) * Revision: industry sector ratios (#122) * sector-ratios: basic reformatting * sector-ratios: add new read_excel function that filters year already * sector-ratios: rename jrc to idees * sector-ratios: rename conv_factor to toe_to_MWh * sector-ratios: modularise into functions * Move overriding of component attributes to function and into data (#123) * move overriding of component attributes to central function and store in separate folder * fix return of helper.override_component_attrs * prepare: fix accidental syntax error * override_component_attrs: bugfix that aligns with pypsa components * Revision: build population layout (#108) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * config: change path to atlite cutout * Revision: build clustered population layouts (#112) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * cl_pop_layout: add fraction column which is repeatedly calculated downstream * Revision: build various heating-related time series (#113) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * gitignore: add .vscode * heating_profiles: update to new atlite and move into __main__ * heating_profiles: remove extra cutout * heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries * heating_profiles: load regions with .buffer(0) before squeeze() * heating_profiles: account for transpose of dataarray * heating_profiles: account for transpose of dataarray in add_exiting_baseyear * Reduce verbosity of Snakefile (2) (#128) * tidy Snakefile light * Snakefile: fix indents * Snakefile: add missing RDIR * tidy config by removing quotes and expanding lists (#109) * bugfix: reorder squeeze() and buffer() * plot/summary: cosmetic changes including: (#131) - matplotlibrc for default style and backend - remove unused config options - option to configure geomap colors - option to configure geomap bounds * solve: align with pypsa-eur using ilopf (#129) * tidy myopic code scripts (#132) * use mock_snakemake from pypsa-eur (#133) * Snakefile: add benchmark files to each rule * Snakefile: only run build_retro_cost if endogenously optimised * Snakefile: remove old {network} wildcard constraints * WIP: Revision: prepare_sector_network (#124) * population_layouts: move inside __main__ and blacken * population_layouts: misc code cleaning and multiprocessing * population_layouts: fix fill_values assignment of urban fractions * population_layouts: bugfig for UK-GB naming ambiguity * population_layouts: sort countries alphabetically for better overview * cl_pop_layout: blacken * cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning * move overriding of component attributes to central function and store in separate folder * prepare: sort imports and remove six dependency * prepare: remove add_emission_prices * prepare: remove unused set_line_s_max_pu This is a function from prepare_network * prepare: remove unused set_line_volume_limit This is a PyPSA-Eur function from prepare_network * prepare: tidy add_co2limit * remove six dependency * prepare: tidy code first batch * prepare: extend override_component_attrs to avoid hacky madd * prepare: remove hacky madd() for individual components * prepare: tidy shift function * prepare: nodes and countries from n.buses not pop_layout * prepare: tidy loading of pop_layout * prepare: fix prepare_costs function * prepare: optimise loading of traffic data * prepare: move localizer into generate_periodic profiles * prepare: some formatting of transport data * prepare: eliminate some code duplication * prepare: fix remove_h2_network - only try to remove EU H2 store if it exists - remove readding nodal Stores because they are never removed * prepare: move cost adjustment to own function * prepare: fix a syntax error * prepare: add investment_year to get() assuming global variable * prepare: move co2_totals out of prepare_data() * Snakefile: remove unused prepare_sector_network inputs * prepare: move limit p/s_nom of lines/links into function * prepare: tidy add_co2limit file handling * Snakefile: fix tabs * override_component_attrs: add n/a defaults * README: Add network picture to make scope clear * README: Fix date of preprint (was too optimistic...) * prepare: move some more config options to config.yaml * prepare: runtime bugfixes * fix benchmark path * adjust plot ylims * add unit attribute to bus, correct cement capture efficiency * bugfix: land usage constrained missed inplace operation Co-authored-by: Tom Brown <tom@nworbmot.org> * add release notes * remove old fix_branches() function * deps: make geopy optional, remove unused imports * increase default BarConvTol * get ready for upcoming PyPSA release * re-remove ** bug * amend release notes Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 18:09:04 +00:00
benchmark: RDIR + "/benchmarks/solve_network/elec_s{simpl}_{clusters}_lv{lv}_{opts}_{sector_opts}_{planning_horizons}"
script: "scripts/solve_network.py"