* allow H2 retrofitting from CH4 pipelines without endogenous CH4 grid
* add small capital cost to gas pipelines to incentivise decommissioning
* add release notes
* add regionalised hydrogen salt cavern storage potentials
* fix reading in salt-cavern potentials and typo in imports
* by default disable nearshore and offshore salt cavern potentials
Co-authored-by: lisazeyen <lisa.zeyen@web.de>
* for now use n.snapshot_weightings.generators
* require pypsa master; use .objective for Nyears
* implement suggestions from code review
* add release note
* Update .gitignore
* Add fictitious load to account for non-transformed shipping emissions
The share of shipping demand that is transformed is defined now for different years to be used with the myopic code.
The carbon emission from the remaining share is treated as a negative load on the atmospheric carbon dioxide bus, just like aviation and land transport emissions.
* Split colours for H2 in Industry and H2 in shipping when plotting balances.
When plotting the balance for H2, the rename dictionary merges all the demands containing H2.
This commit disables such merging and keeps different colours for H2 in shipping and H2 in industry. This is useful when one wants to look at the H2 balance and have an overview of where the H2 is consumed in the model.
* Make transformation of Steel and Aluminum production depends on year
Previously, the transformation of the Steel and Aluminum production was assumed to occur overnight.
This commit enables the definition of a transformation path via the config.yaml file.
This requires adding the {planning_horizon} to the input and output file name of the following rules:
build_industrial_production_per_country_tomorrow
build_industrial_production_per_node
build_industry_energy_demand_per_node
prepare_sector_network
* small follow-up to merge
* Add oil consumed in shipping as a load to EU oil bus
* Update scripts/prepare_sector_network.py
* add planning_horizons wildcard to benchmark paths
* fixup: double fraction_primary for steel
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
* Update config.default.yaml
Some people find it counterintuitive that battery and H2 storage are not kept from PyPSA-Eur. An explicit mention can help here.
* Update installation.rst
We only use one config.yaml file
* Add an arrow from biomass to carbon dioxide.
This signals that we also have BECCS in the model.
Co-authored-by: martavp <martavictoriaperez@gmail.com>
* ammonia_production: minor cleaning and move into __main__ (#106)
* biomass_potentials: code cleaning and automatic country index inferral (#107)
* Revision: build energy totals (#111)
* blacken
* energy_totals: preliminaries
* energy_totals: update build_swiss
* energy_totals: update build_eurostat
* energy_totals: update build_idees
* energy_totals: update build_energy_totals
* energy_totals: update build_eea_co2
* energy_totals: update build_eurostat_co2
* energy_totals: update build_co2_totals
* energy_totals: update build_transport_data
* energy_totals: add tqdm progressbar to idees
* energy_totals: adjust __main__ section
* energy_totals: handle inputs via Snakefile and config
* energy_totals: handle data and emissions year via config
* energy_totals: fix reading in eurostat for different years
* energy_totals: fix erroneous drop duplicates
This caused problems for waste management in HU and SI
* energy_totals: make scope selection of CO2 or GHG a config option
* Revision: build industrial production per country (#114)
* industry-ppc: format
* industry-ppc: rewrite for performance
* industry-ppc: move reference year to config
* industry-ppct: tidy up and format (#115)
* remove stale industry demand rules (#116)
* industry-epc: rewrite for performance (#117)
* Revision: industrial distribution key (#118)
* industry-distribution: first tidying
* industry-distribution: first tidying
* industry-distribution: fix syntax
* Revision: industrial energy demand per node today (#119)
* industry-epn: minor code cleaning
* industry-epn: remove accidental artifact
* industry-epn: remove accidental artifact II
* industry-ppn: code cleaning (#120)
* minor code cleaning (#121)
* Revision: industry sector ratios (#122)
* sector-ratios: basic reformatting
* sector-ratios: add new read_excel function that filters year already
* sector-ratios: rename jrc to idees
* sector-ratios: rename conv_factor to toe_to_MWh
* sector-ratios: modularise into functions
* Move overriding of component attributes to function and into data (#123)
* move overriding of component attributes to central function and store in separate folder
* fix return of helper.override_component_attrs
* prepare: fix accidental syntax error
* override_component_attrs: bugfix that aligns with pypsa components
* Revision: build population layout (#108)
* population_layouts: move inside __main__ and blacken
* population_layouts: misc code cleaning and multiprocessing
* population_layouts: fix fill_values assignment of urban fractions
* population_layouts: bugfig for UK-GB naming ambiguity
* population_layouts: sort countries alphabetically for better overview
* config: change path to atlite cutout
* Revision: build clustered population layouts (#112)
* population_layouts: move inside __main__ and blacken
* population_layouts: misc code cleaning and multiprocessing
* population_layouts: fix fill_values assignment of urban fractions
* population_layouts: bugfig for UK-GB naming ambiguity
* population_layouts: sort countries alphabetically for better overview
* cl_pop_layout: blacken
* cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning
* cl_pop_layout: add fraction column which is repeatedly calculated downstream
* Revision: build various heating-related time series (#113)
* population_layouts: move inside __main__ and blacken
* population_layouts: misc code cleaning and multiprocessing
* population_layouts: fix fill_values assignment of urban fractions
* population_layouts: bugfig for UK-GB naming ambiguity
* population_layouts: sort countries alphabetically for better overview
* cl_pop_layout: blacken
* cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning
* gitignore: add .vscode
* heating_profiles: update to new atlite and move into __main__
* heating_profiles: remove extra cutout
* heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries
* heating_profiles: load regions with .buffer(0) before squeeze()
* heating_profiles: account for transpose of dataarray
* heating_profiles: account for transpose of dataarray in add_exiting_baseyear
* Reduce verbosity of Snakefile (2) (#128)
* tidy Snakefile light
* Snakefile: fix indents
* Snakefile: add missing RDIR
* tidy config by removing quotes and expanding lists (#109)
* bugfix: reorder squeeze() and buffer()
* plot/summary: cosmetic changes including: (#131)
- matplotlibrc for default style and backend
- remove unused config options
- option to configure geomap colors
- option to configure geomap bounds
* solve: align with pypsa-eur using ilopf (#129)
* tidy myopic code scripts (#132)
* use mock_snakemake from pypsa-eur (#133)
* Snakefile: add benchmark files to each rule
* Snakefile: only run build_retro_cost if endogenously optimised
* Snakefile: remove old {network} wildcard constraints
* WIP: Revision: prepare_sector_network (#124)
* population_layouts: move inside __main__ and blacken
* population_layouts: misc code cleaning and multiprocessing
* population_layouts: fix fill_values assignment of urban fractions
* population_layouts: bugfig for UK-GB naming ambiguity
* population_layouts: sort countries alphabetically for better overview
* cl_pop_layout: blacken
* cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning
* move overriding of component attributes to central function and store in separate folder
* prepare: sort imports and remove six dependency
* prepare: remove add_emission_prices
* prepare: remove unused set_line_s_max_pu
This is a function from prepare_network
* prepare: remove unused set_line_volume_limit
This is a PyPSA-Eur function from prepare_network
* prepare: tidy add_co2limit
* remove six dependency
* prepare: tidy code first batch
* prepare: extend override_component_attrs to avoid hacky madd
* prepare: remove hacky madd() for individual components
* prepare: tidy shift function
* prepare: nodes and countries from n.buses not pop_layout
* prepare: tidy loading of pop_layout
* prepare: fix prepare_costs function
* prepare: optimise loading of traffic data
* prepare: move localizer into generate_periodic profiles
* prepare: some formatting of transport data
* prepare: eliminate some code duplication
* prepare: fix remove_h2_network
- only try to remove EU H2 store if it exists
- remove readding nodal Stores because they are never removed
* prepare: move cost adjustment to own function
* prepare: fix a syntax error
* prepare: add investment_year to get() assuming global variable
* prepare: move co2_totals out of prepare_data()
* Snakefile: remove unused prepare_sector_network inputs
* prepare: move limit p/s_nom of lines/links into function
* prepare: tidy add_co2limit file handling
* Snakefile: fix tabs
* override_component_attrs: add n/a defaults
* README: Add network picture to make scope clear
* README: Fix date of preprint (was too optimistic...)
* prepare: move some more config options to config.yaml
* prepare: runtime bugfixes
* fix benchmark path
* adjust plot ylims
* add unit attribute to bus, correct cement capture efficiency
* bugfix: land usage constrained missed inplace operation
Co-authored-by: Tom Brown <tom@nworbmot.org>
* add release notes
* remove old fix_branches() function
* deps: make geopy optional, remove unused imports
* increase default BarConvTol
* get ready for upcoming PyPSA release
* re-remove ** bug
* amend release notes
Co-authored-by: Tom Brown <tom@nworbmot.org>
* add github action ci
* only one environment update call
* line break in echo did not work
* fix syntax
* fix version syntax
* switch to glpk
* reduce time from month to week
* list environment
* use new ipopt version
https://github.com/conda-forge/ipopt-feedstock/issues/55
* remove accidental additions
* request ipopt lower than 3.13.3
https://github.com/conda-forge/ipopt-feedstock/issues/64
* add badges and release notes
* add badge to readme and make ci.yaml cc-0
* Add focus_weights to pre-clustering
Hey guys,
another quick fix since I noticed it wasn't implemented yet: When pre-clustering the network, the focus_weights have not yet been considered. This may distort clustering results when pre-clustering to a low resolution.
* Update release_notes.rst
* adjust buil_cutout.py and Snakefile
* try adjusting build_renewable_profiles, currently crashing due to weird pyproj error
* build_renewable_profiles: -remove printing gid
* build_renewable_profiles: use dask for paralellization, use dense functions
* build_renewable_profiles:
- revise imports
- add logging for long calculation
- revise explaining comment
- revise distance calculation
* build profiles: adjust to cutout.grid
* * fix area to square km
* rename potmatrix -> capacity_potential
* rename available to availibility
* config.default update cutout params
build_renewable_potentials: major refactoring and simplification
hydro_profiles: update code
* build profiles: fix weight output dimensions
* build profiles: fix typo, fix selection of buses
* build profiles: reinsert paths variable
* follow up
* build profiles: move to dask calculation only
* CI: set build cutout to true (add CDSAPI)
* build profiles: use pyproj, test with gleas and geokit upstream
* environment.yaml fix atlite version
* build profiles: use dask 'processes' for more than 25 regions
* build profiles: specify dask scheduler according to number of regions
* backpedal a bit, only allow scheduler='processes'
* follow up, code style and fixup
* build profiles: add logger info for underwater fraction calc
* config adjust cutout parameters
Snakefile fixup
* config.default.yaml: adjust resolution
* config: use one cutout in total
build_cutout: automatic detetection of geographical boundaries
* env: add python>=3.8 requirement
build_cutout: fixup for region bound
* config: allow base cutout
* folllow up, fix up
* follow up II
* clean up
* clean up II
* build profiles: move back to multiprocessing due to performance issues
* small code style corrections
* move in pool context
* swqitch to ratsterio
* switch to rasterio for availibility calculation
* tiny fixup
* * build continental raster for offshore distance calculation
* adjust Snakefile to new script build_raster
* rename continental raster to onshore raster
add projected_mask function (not yet tested)
add docstrings, modularize
* Snakefile: remove build_onhore_raster rule, build mask directly from geometry instead
build_natura_raster: adjust code, add function for exporting
build_profiles:
* add buffer to shore distance to init_globals function
* update docstrings
* improve handling of nodata grid codes
* add geometry mask if natura raster not activated
(the 255 value is an 'eligible' value for the corine data base,
do this for excluding data outside the shape)
* build_profiles: adjust docstrings
* update environment
* build profiles: fixup reproject woth padding
* follow up, small fixups
* fix resampling method
checkpoint: reproduces solar profile in tut data
* reintegrate plot map
code style
* config: rename cutout into "base"
* build profiles: adjust to new atlite code
* natura raster: small fixup
* build natura raster: compress tiff file
* config: adjust cutout names
* build profiles: cover case if no or partial overlap between natura raster and cutout
* config-tutorial: adjust cutout params
* buid-profifiles: fixup in gebco filter
* follow up
* update config files
* build profiles: select layoutmatrix != 0
* build profiles: speed up average_distance and underwaterfraction
* build profiles: fix typo
* update release notes
build_cutout: only build needed features
* update envs
* config: add temperature to sarah features
* temporary fix for atlite v0.2.1 and new xarray version release
* env: remove xarray specification
* * remove rule build_country_flh
* build profiles: remove sneaked in line
* doc: update configuration.rst (section atlite) and corresponding csv table
* release notes: fix quotes
* build profiles: use 3035 for area calculation
* Update envs/environment.docs.yaml
* Update scripts/build_cutout.py
* Update doc/release_notes.rst
Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>
* Update doc/configuration.rst
Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>
* Update scripts/build_cutout.py
Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>
* update release notes
* release notes: add deprecation of 'keep_all_available_areas'
build profiles: remove warning for 'keep_all_available_areas'
* build cutout: rearrage code, set buffer correctly
* Rename tutorial cutout to remove name clash with real cutout.
* Update release_notes.rst: Rename tutorial cutout.
* retrieve: update cutouts and downloads (alternative) (#237)
* retrieve: update cutouts and downloads
* retrieve: remove unnecessary import
* use snakemake remote file functionality
* Snakefile: update zenodo link
* update natura remote link (closes#234)
* env: update atlite version to 0.2.2
* env: fix dask version due to memory issues
* test: retrieve cutout instead of build
* test: use tutorial cutout for CI
Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
This substitutes the previous way of doing it. Now, to multiply the reference p_nom_max by 3, one should include in the config file 'solar+p3' (instead of the previous solarx3)
* release_notes: order for release
* doc: fix smaller typos and tidy up
* config: bump version
* doc: fix line references
* doc: bump confpy version
* envs: update fixed versions yaml
* Snakefile: simplify all_elec to all
* fix clustering of offwind-ac and offwind-dc in sites option
* add release nodes
* attach renewable assets by location (lat and lon) from OPSD register to network
* adapt default config to changes
* undo changes from a differen PR in cluster_network.py
* undo changes from a different PR, add release notes for this PR
* correct release notes
* add comments for relevant settings in add_electricity.py
* adjust configtable for electricity to OPSD renewable option and add estimate_renewable_capacities_from_capacitiy_stats
* reset cluster_network to HEAD
* add_electricity: Capacity is float
* config: add GB to OPSD_VRE_countries
* review and modify implementation
* update release notes
* Update envs/environment.yaml
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
Co-authored-by: martha.frysztacki <eb5194@iai-esm003.iai.kit.edu>
Co-authored-by: eb5194 <martha.frysztacki@kit.edu>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
* add option to use custom clustermaps from data folder
* adapt default config to custom busmap
* input file from Snakefile
* adapt input description
* add option to use custom clustermaps from data folder
* adapt default config to custom busmap
* input file from Snakefile
* adapt input description
* Snakefile: custom_busmap in cluster_network input is now csv
* cluster_network: custom_busmap is now read as csv file, adaptions of description
* simplify_network: adapt descriptions
* configfiles: add cutom_clustermaps switch
* unify clustarmap and busmap names
* unify clustermap and busmap names
* test/config: unify clustermap and busmap names
* cluster_network: make clustering_for_n_clusters compatible with simplify_network
* simplify_network: make compatible with changes in cluster_network.py
* Update scripts/cluster_network.py
* Update scripts/simplify_network.py
* Update scripts/simplify_network.py
* Update scripts/cluster_network.py
* Update scripts/cluster_network.py
* cluster_network: move custom_busmap flag to enable; simplify names
* cluster_network: move custom_busmap flag to enable; simplify names
* custom_busmap: add documentation
* cluster_network: add default for custom_busmap for compatibility with old configs
* cluster_network: add default for custom_busmap for compatibility with old configs
Co-authored-by: martha.frysztacki <eb5194@iai-esm003.iai.kit.edu>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
It is identical to config.default.yaml except for two parameters
(foresight and planning_horizons) so I decided to consolidate the
example configs. Instructions for how to use the myopic foresight can
be found in the documentation (now updated).
* fix emission prices
I'm not sure if the previous setup was intentional, but regarding that different generators might have different efficiencies and the emissions are carrier specific only, it makes more sense set net emission price.
* small fix
* update release_notes and config
Rather than taking a mean of the clustered connection costs.
Apply cost update also for overnight scenarios based on planning year.
Add land costs for onshore wind.
* Separate config switches for retrieving and building cutout, natura raster.
* Include new switches in tutorial and test configs.
* Update release_notes.rst
* Update toplevel.csv
* Update tutorial.rst
* Update configuration.rst
* Add logging to logfiles to all snakemake workflow scripts.
* Fix missing quotation marks in Snakefile.
* Apply suggestions from code review
Co-Authored-By: Fabian Neumann <fabian.neumann@outlook.de>
* Apply suggestions from code review
Co-Authored-By: Fabian Neumann <fabian.neumann@outlook.de>
* doc: fix _ec_ filenames in docs
* Allow logging message format to be specified in config.yaml.
* Add logging for Snakemake rule 'retrieve_databundle '.
* Add limited logging to STDERR only for retrieve_*.py scripts.
* Import progressbar module only on demand.
* Fix logging to file and enable concurrent printing to STDERR for most scripts.
* Add new 'logging_format' option to Travis CI test config.yaml.
* Add missing parenthesis (bug fix) and cross-os compatible paths.
* Fix typos in messages.
* Use correct log files for logging (bug fix).
* doc: fix line references
* config: logging_format in all configs
* doc: add doc for logging_format
* environment: update to powerplantmatching 0.4.3
* doc: update line references for tutorial.rst
* Change logging configuration scheme for config.yaml.
* Add helper function for doing basic logging configuration.
* Add logpath for prepare_links_p_nom rule.
* Outsource basic logging configuration for all scripts to _helper submodule.
* Update documentation for changed config.yaml structure.
Instead of 'logging_level' and 'logging_format', now 'logging' with subcategories is used.
* _helpers: Change configure_logging signature.
