When plotting the balance for H2, the rename dictionary merges all the demands containing H2.
This commit disables such merging and keeps different colours for H2 in shipping and H2 in industry. This is useful when one wants to look at the H2 balance and have an overview of where the H2 is consumed in the model.
The share of shipping demand that is transformed is defined now for different years to be used with the myopic code.
The carbon emission from the remaining share is treated as a negative load on the atmospheric carbon dioxide bus, just like aviation and land transport emissions.
* adjust buil_cutout.py and Snakefile
* try adjusting build_renewable_profiles, currently crashing due to weird pyproj error
* build_renewable_profiles: -remove printing gid
* build_renewable_profiles: use dask for paralellization, use dense functions
* build_renewable_profiles:
- revise imports
- add logging for long calculation
- revise explaining comment
- revise distance calculation
* build profiles: adjust to cutout.grid
* * fix area to square km
* rename potmatrix -> capacity_potential
* rename available to availibility
* config.default update cutout params
build_renewable_potentials: major refactoring and simplification
hydro_profiles: update code
* build profiles: fix weight output dimensions
* build profiles: fix typo, fix selection of buses
* build profiles: reinsert paths variable
* follow up
* build profiles: move to dask calculation only
* CI: set build cutout to true (add CDSAPI)
* build profiles: use pyproj, test with gleas and geokit upstream
* environment.yaml fix atlite version
* build profiles: use dask 'processes' for more than 25 regions
* build profiles: specify dask scheduler according to number of regions
* backpedal a bit, only allow scheduler='processes'
* follow up, code style and fixup
* build profiles: add logger info for underwater fraction calc
* config adjust cutout parameters
Snakefile fixup
* config.default.yaml: adjust resolution
* config: use one cutout in total
build_cutout: automatic detetection of geographical boundaries
* env: add python>=3.8 requirement
build_cutout: fixup for region bound
* config: allow base cutout
* folllow up, fix up
* follow up II
* clean up
* clean up II
* build profiles: move back to multiprocessing due to performance issues
* small code style corrections
* move in pool context
* swqitch to ratsterio
* switch to rasterio for availibility calculation
* tiny fixup
* * build continental raster for offshore distance calculation
* adjust Snakefile to new script build_raster
* rename continental raster to onshore raster
add projected_mask function (not yet tested)
add docstrings, modularize
* Snakefile: remove build_onhore_raster rule, build mask directly from geometry instead
build_natura_raster: adjust code, add function for exporting
build_profiles:
* add buffer to shore distance to init_globals function
* update docstrings
* improve handling of nodata grid codes
* add geometry mask if natura raster not activated
(the 255 value is an 'eligible' value for the corine data base,
do this for excluding data outside the shape)
* build_profiles: adjust docstrings
* update environment
* build profiles: fixup reproject woth padding
* follow up, small fixups
* fix resampling method
checkpoint: reproduces solar profile in tut data
* reintegrate plot map
code style
* config: rename cutout into "base"
* build profiles: adjust to new atlite code
* natura raster: small fixup
* build natura raster: compress tiff file
* config: adjust cutout names
* build profiles: cover case if no or partial overlap between natura raster and cutout
* config-tutorial: adjust cutout params
* buid-profifiles: fixup in gebco filter
* follow up
* update config files
* build profiles: select layoutmatrix != 0
* build profiles: speed up average_distance and underwaterfraction
* build profiles: fix typo
* update release notes
build_cutout: only build needed features
* update envs
* config: add temperature to sarah features
* temporary fix for atlite v0.2.1 and new xarray version release
* env: remove xarray specification
* * remove rule build_country_flh
* build profiles: remove sneaked in line
* doc: update configuration.rst (section atlite) and corresponding csv table
* release notes: fix quotes
* build profiles: use 3035 for area calculation
* Update envs/environment.docs.yaml
* Update scripts/build_cutout.py
* Update doc/release_notes.rst
Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>
* Update doc/configuration.rst
Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>
* Update scripts/build_cutout.py
Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>
* update release notes
* release notes: add deprecation of 'keep_all_available_areas'
build profiles: remove warning for 'keep_all_available_areas'
* build cutout: rearrage code, set buffer correctly
* Rename tutorial cutout to remove name clash with real cutout.
* Update release_notes.rst: Rename tutorial cutout.
* retrieve: update cutouts and downloads (alternative) (#237)
* retrieve: update cutouts and downloads
* retrieve: remove unnecessary import
* use snakemake remote file functionality
* Snakefile: update zenodo link
* update natura remote link (closes#234)
* env: update atlite version to 0.2.2
* env: fix dask version due to memory issues
* test: retrieve cutout instead of build
* test: use tutorial cutout for CI
Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
Recently ICE vehicles were added, but the emissions were not accounted
for. Now, like aviation emissions, they are added as a negative load
to the "co2 atmosphere" bus.
Reasoning: we can also have fossil and biomass liquid hydrocarbons, as
well as production from the Fischer-Tropsch process, particularly for
simulations before 2050.
This substitutes the previous way of doing it. Now, to multiply the reference p_nom_max by 3, one should include in the config file 'solar+p3' (instead of the previous solarx3)
For the myopic method, based on the carbon budget indicated in the config file (sector_opts), a CO2 limit is calculated for every planning_horizon following an exponential or beta decay. A file with CO2 limit in every planning_horizon and a plot showing historical and planned CO2 emissions
are saved in the results
Use DEA assumptions for post-combustion carbon capture.
Also rename CCS as CC whenever only carbon capture is involved, since
sequestration (or CCU) is a separate step.
Only change was to remove the Store-Link-Bus combinations for
batteries and H2 storage from PyPSA-Eur, since they are implemented
with different names, costs and voltage level in PyPSA-Eur-Sec.
Removals are now done in a more transparent way in the config.yaml.
* release_notes: order for release
* doc: fix smaller typos and tidy up
* config: bump version
* doc: fix line references
* doc: bump confpy version
* envs: update fixed versions yaml
* Snakefile: simplify all_elec to all
* fix clustering of offwind-ac and offwind-dc in sites option
* add release nodes
* attach renewable assets by location (lat and lon) from OPSD register to network
* adapt default config to changes
* undo changes from a differen PR in cluster_network.py
* undo changes from a different PR, add release notes for this PR
* correct release notes
* add comments for relevant settings in add_electricity.py
* adjust configtable for electricity to OPSD renewable option and add estimate_renewable_capacities_from_capacitiy_stats
* reset cluster_network to HEAD
* add_electricity: Capacity is float
* config: add GB to OPSD_VRE_countries
* review and modify implementation
* update release notes
* Update envs/environment.yaml
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
Co-authored-by: martha.frysztacki <eb5194@iai-esm003.iai.kit.edu>
Co-authored-by: eb5194 <martha.frysztacki@kit.edu>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
* add option to use custom clustermaps from data folder
* adapt default config to custom busmap
* input file from Snakefile
* adapt input description
* add option to use custom clustermaps from data folder
* adapt default config to custom busmap
* input file from Snakefile
* adapt input description
* Snakefile: custom_busmap in cluster_network input is now csv
* cluster_network: custom_busmap is now read as csv file, adaptions of description
* simplify_network: adapt descriptions
* configfiles: add cutom_clustermaps switch
* unify clustarmap and busmap names
* unify clustermap and busmap names
* test/config: unify clustermap and busmap names
* cluster_network: make clustering_for_n_clusters compatible with simplify_network
* simplify_network: make compatible with changes in cluster_network.py
* Update scripts/cluster_network.py
* Update scripts/simplify_network.py
* Update scripts/simplify_network.py
* Update scripts/cluster_network.py
* Update scripts/cluster_network.py
* cluster_network: move custom_busmap flag to enable; simplify names
* cluster_network: move custom_busmap flag to enable; simplify names
* custom_busmap: add documentation
* cluster_network: add default for custom_busmap for compatibility with old configs
* cluster_network: add default for custom_busmap for compatibility with old configs
Co-authored-by: martha.frysztacki <eb5194@iai-esm003.iai.kit.edu>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
Specify as dictionary, use get_parameter to get correct value.
Also remove old parameter "space_heating_fraction" since this is
superceded by the new exogenous retro code.
It is identical to config.default.yaml except for two parameters
(foresight and planning_horizons) so I decided to consolidate the
example configs. Instructions for how to use the myopic foresight can
be found in the documentation (now updated).
Strategy is too keep as much of configuration in config.yaml as
possible.
We also aim to allow exogenous investment-year-dependent
configurations to be done in a similar manner (e.g. share of district
heating or FCEV transport).
List classes in config.yaml, rather than integer selection in
build_biomass_potentials.py.
Also output potentials for all years and scenarios for analysis.
This allows us to control the substitution of natural gas for hydrogen
in NH3 production.
Remaining basic chemicals are olefins, BTX and chlorine.
For 2015 NH3 production, we use the USGS data source.
This was handled before in industry_sector_ratios.csv which was
confusing.
Now industry_sector_ratios.csv represents the genuine energy
consumption per tonne of material for each industrial route
(MWh/tMaterial).
An new file is created with ktMaterial/a in
industrial_production_per_country_tomorrow.csv which contains changes
to the fraction of primary/secondary routes compared to today's
production in industrial_production_per_country.csv.
This is less confusing I think.
* fix emission prices
I'm not sure if the previous setup was intentional, but regarding that different generators might have different efficiencies and the emissions are carrier specific only, it makes more sense set net emission price.
* small fix
* update release_notes and config
- add_brownfield.py: Have to make sure that for each CHP there is both
a heat and electric link, but they have different p_nom for each
CHP, so have to make sure we don't remove one without the other.
- solve_network.py: Make sure extra_functionality constraints for CHP
power-heat feasibility graph also work for non-extendable CHPs.
In order to calculate connection costs, average values for underground_fraction and average_distance are calculated for all the buses in the initial network mapped to the clustered network.
Previously they were distributed only by country to the first node in
the country.
Now conventional power plants are assigned to the correct node using
the bus map from PyPSA-Eur.
Wind and solar are distributed in each country by capacity factor.
The code has been refactored and a bug was fixed whereby total
capacities of wind and solar in each country were not correct.
Now the years in the config.yaml for myopic are integers not strings.
config.default.yaml should be copied to config.yaml by new users.
That way config.yaml doesn't change every time the developers use new
settings.
A basic "Getting Started" section is added to the README.
Changes to be committed:
modified: .gitignore
modified: README.md
renamed: config.yaml -> config.default.yaml
* Separate config switches for retrieving and building cutout, natura raster.
* Include new switches in tutorial and test configs.
* Update release_notes.rst
* Update toplevel.csv
* Update tutorial.rst
* Update configuration.rst
* rewrite mocksnakemake for parsing real Snakefile
* continue add function to scripts
* going through all scripts, setting new mocksnakemake
* fix plotting scripts
* fix build_country_flh
* fix build_country_flh II
* adjust config files
* fix make_summary for tutorial network
* create dir also for output
* incorporate suggestions
* consistent import of mocksnakemake
* consistent import of mocksnakemake II
* Update scripts/_helpers.py
Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com>
* Update scripts/_helpers.py
Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com>
* Update scripts/_helpers.py
Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com>
* Update scripts/_helpers.py
Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com>
* Update scripts/plot_network.py
Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com>
* Update scripts/plot_network.py
Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com>
* Update scripts/retrieve_databundle.py
Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com>
* use pathlib for mocksnakemake
* rename mocksnakemake into mock_snakemake
* revert change in data
* Update scripts/_helpers.py
Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com>
* remove setting logfile in mock_snakemake, use Path in configure_logging
* fix fallback path and base_dir
fix return type of make_io_accessable
* reformulate mock_snakemake
* incorporate suggestion, fix typos
* mock_snakemake: apply absolute paths again, add assertion error
*.py: make hard coded io path accessable for mock_snakemake
* retrieve_natura_raster: use snakemake.output for fn_out
* include suggestion
* Apply suggestions from code review
Co-Authored-By: Jonas Hörsch <jonas.hoersch@posteo.de>
* linting, add return ad end of file
* Update scripts/plot_p_nom_max.py
Co-Authored-By: Jonas Hörsch <jonas.hoersch@posteo.de>
* Update scripts/plot_p_nom_max.py
fixes#112
Co-Authored-By: Jonas Hörsch <jonas.hoersch@posteo.de>
* plot_p_nom_max: small correction
* config.tutorial.yaml fix snapshots end
* use techs instead of technology
* revert try out from previous commit, complete replacing
* change clusters -> clusts in plot_p_nom_max due to wildcard constraints of clusters
* change clusters -> clusts in plot_p_nom_max due to wildcard constraints of clusters II
* Add logging to logfiles to all snakemake workflow scripts.
* Fix missing quotation marks in Snakefile.
* Apply suggestions from code review
Co-Authored-By: Fabian Neumann <fabian.neumann@outlook.de>
* Apply suggestions from code review
Co-Authored-By: Fabian Neumann <fabian.neumann@outlook.de>
* doc: fix _ec_ filenames in docs
* Allow logging message format to be specified in config.yaml.
* Add logging for Snakemake rule 'retrieve_databundle '.
* Add limited logging to STDERR only for retrieve_*.py scripts.
* Import progressbar module only on demand.
* Fix logging to file and enable concurrent printing to STDERR for most scripts.
* Add new 'logging_format' option to Travis CI test config.yaml.
* Add missing parenthesis (bug fix) and cross-os compatible paths.
* Fix typos in messages.
* Use correct log files for logging (bug fix).
* doc: fix line references
* config: logging_format in all configs
* doc: add doc for logging_format
* environment: update to powerplantmatching 0.4.3
* doc: update line references for tutorial.rst
* Change logging configuration scheme for config.yaml.
* Add helper function for doing basic logging configuration.
* Add logpath for prepare_links_p_nom rule.
* Outsource basic logging configuration for all scripts to _helper submodule.
* Update documentation for changed config.yaml structure.
Instead of 'logging_level' and 'logging_format', now 'logging' with subcategories is used.
* _helpers: Change configure_logging signature.
