Commit Graph

65 Commits

Author SHA1 Message Date
Fabian Neumann
b04a55a248 merge former countries hydro profiles 2022-07-20 15:03:48 +02:00
Fabian Neumann
27e5dec4e6 documentation fixes 2022-06-30 09:01:49 +02:00
Fabian Neumann
74f8ca1c7f
Merge branch 'master' into eu-energy-security 2022-06-28 08:09:39 +02:00
martacki
f1dedd9429 hierarchical clustering: release notes & revert to old default clustering algorithm (kmeans) 2022-06-27 21:11:49 +02:00
Martha Frysztacki
50a518a155
Merge branch 'master' into introduce_hac_clustering 2022-06-27 20:45:40 +02:00
Fabian Neumann
51de606aab
Revert "remove build/retrieve natura raster, directly use shapefile" 2022-06-27 19:00:41 +02:00
Martha Frysztacki
5587327be3
Merge branch 'master' into harmonize_clustering_strategies 2022-06-27 14:02:45 +02:00
martacki
2dfa2753ba clustering strats to configurables: config spacing 2022-06-27 13:59:04 +02:00
Fabian Neumann
5ca231824e
Merge branch 'master' into eu-energy-security 2022-06-27 08:58:37 +02:00
martacki
c9c738e96b clustering strats to configurables: set defaults for yaml-incompatible strats + include in stubaggregation 2022-06-24 20:57:53 +02:00
Fabian Neumann
4c5ec8e5f1
Merge branch 'master' into introduce_hac_clustering 2022-06-24 20:22:47 +02:00
Fabian Hofmann
f6b7317d04
Update config.tutorial.yaml
Co-authored-by: Martha Frysztacki <martha.frysztacki@kit.edu>
2022-06-23 15:14:38 +02:00
martacki
b4d8dd8ecb add changes from PR #379 2022-06-20 19:21:08 +02:00
martacki
bef4967e84 clustering strategies moved to configurables 2022-06-20 18:58:23 +02:00
Fabian Neumann
b4c819c757 config: add fixed_year parameter for load 2022-06-16 08:53:07 +02:00
Fabian Neumann
9a37fe6572 merge master 2022-06-15 09:22:51 +02:00
Fabian Neumann
732d114416 add results_dir to all configs 2022-04-08 19:59:27 +02:00
Fabian Neumann
7fee2944d8
Merge branch 'master' into rm-raster-natura 2022-03-28 12:07:33 +02:00
Fabian Neumann
d94719a33d fix tutorial and add release notes 2022-03-24 15:53:02 +01:00
Fabian Neumann
35f8b54a46 switch to Belgium for CI and tutorial 2022-03-24 15:51:36 +01:00
Fabian Neumann
31f9864daf omit NATURA from testing and tutorial due to file size# 2022-03-20 21:01:07 +01:00
Fabian Neumann
5a7c0d7967 remove build/retrieve natura raster, directly use shapefile 2022-03-20 10:03:17 +01:00
Fabian Neumann
ce03353ac3 change year to weather_year 2022-02-18 10:27:10 +01:00
Fabian Neumann
52b0d29fae merge master 2022-02-18 10:24:26 +01:00
Martha Frysztacki
25b3245923
Merge branch 'master' into introduce_hac_clustering 2022-02-04 14:35:14 +01:00
Fabian Neumann
a747c88d08 disable solar PV CF correction factor for default satellite data (closes #285) 2022-01-13 15:42:48 +01:00
martacki
299e71e2b3 introduce hierarchical agglomeratice clustering (hac) 2021-11-17 13:46:33 +01:00
Fabian Neumann
c0d1656091 build_load_data: separate retrieving and processing load data 2021-11-10 10:37:09 +01:00
Fabian Neumann
b88322587f
Release 0.4 (#274)
* update release notes [skip ci]

* update fixed versions environment.yaml

* format release notes
2021-09-14 16:15:25 +02:00
martacki
9825fa32b2 change layout of configuration settings 2021-06-01 12:00:55 +02:00
martacki
eef8e3fe41 different line in configuration settings 2021-06-01 11:00:31 +02:00
martacki
f3f587e3f8 simplify to substations - initial draft 2021-05-21 13:54:38 +02:00
FabianHofmann
b8a7685931
Atlite availability (#224)
* adjust buil_cutout.py and Snakefile

* try adjusting build_renewable_profiles, currently crashing due to weird pyproj error

* build_renewable_profiles: -remove printing gid

* build_renewable_profiles: use dask for paralellization, use dense functions

* build_renewable_profiles:
	- revise imports
	- add logging for long calculation
	- revise explaining comment
	- revise distance calculation

* build profiles: adjust to cutout.grid

* * fix area to square km
* rename potmatrix -> capacity_potential
* rename available to availibility

* config.default update cutout params
build_renewable_potentials: major refactoring and simplification
hydro_profiles: update code

* build profiles: fix weight output dimensions

* build profiles: fix typo, fix selection of buses

* build profiles: reinsert paths variable

* follow up

* build profiles: move to dask calculation only

* CI: set build cutout to true (add CDSAPI)

* build profiles: use pyproj, test with gleas and geokit upstream

* environment.yaml fix atlite version

* build profiles: use dask 'processes' for more than 25 regions

* build profiles: specify dask scheduler according to number of regions

* backpedal a bit, only allow scheduler='processes'

* follow up, code style and fixup

* build profiles: add logger info for underwater fraction calc

* config adjust cutout parameters
Snakefile fixup

* config.default.yaml: adjust resolution

* config: use one cutout in total
build_cutout: automatic detetection of geographical boundaries

* env: add python>=3.8 requirement
build_cutout: fixup for region bound

* config: allow base cutout

* folllow up, fix up

* follow up II

* clean up

* clean up II

* build profiles: move back to multiprocessing due to performance issues

* small code style corrections

* move in pool context

* swqitch to ratsterio

* switch to rasterio for availibility calculation

* tiny fixup

* * build continental raster for offshore distance calculation
* adjust Snakefile to new script build_raster

* rename continental raster to onshore raster
add projected_mask function (not yet tested)
add docstrings, modularize

* Snakefile: remove build_onhore_raster rule, build mask directly from geometry instead
build_natura_raster: adjust code, add function for exporting
build_profiles:
	* add buffer to shore distance to init_globals function
	* update docstrings
	* improve handling of nodata grid codes
	* add geometry mask if natura raster not activated
	(the 255 value is an 'eligible' value for the corine data base,
	do this for excluding data outside the shape)

* build_profiles: adjust docstrings

* update environment

* build profiles: fixup reproject woth padding

* follow up, small fixups

* fix resampling method

checkpoint: reproduces solar profile in tut data

* reintegrate plot map
code style

* config: rename cutout into "base"

* build profiles: adjust to new atlite code

* natura raster: small fixup

* build natura raster: compress tiff file

* config: adjust cutout names

* build profiles: cover case if no or partial overlap between natura raster and cutout

* config-tutorial: adjust cutout params

* buid-profifiles: fixup in gebco filter

* follow up

* update config files

* build profiles: select layoutmatrix != 0

* build profiles: speed up average_distance and underwaterfraction

* build profiles: fix typo

* update release notes
build_cutout: only build needed features

* update envs

* config: add temperature to sarah features

* temporary fix for atlite v0.2.1 and new xarray version release

* env: remove xarray specification

* * remove rule build_country_flh
* build profiles: remove sneaked in line

* doc: update configuration.rst (section atlite) and corresponding csv table

* release notes: fix quotes

* build profiles: use 3035 for area calculation

* Update envs/environment.docs.yaml

* Update scripts/build_cutout.py

* Update doc/release_notes.rst

Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>

* Update doc/configuration.rst

Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>

* Update scripts/build_cutout.py

Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>

* update release notes

* release notes: add deprecation of 'keep_all_available_areas'
build profiles: remove warning for 'keep_all_available_areas'

* build cutout: rearrage code, set buffer correctly

* Rename tutorial cutout to remove name clash with real cutout.

* Update release_notes.rst: Rename tutorial cutout.

* retrieve: update cutouts and downloads (alternative) (#237)

* retrieve: update cutouts and downloads

* retrieve: remove unnecessary import

* use snakemake remote file functionality

* Snakefile: update zenodo link

* update natura remote link (closes #234)

* env: update atlite version to 0.2.2

* env: fix dask version due to memory issues

* test: retrieve cutout instead of build

* test: use tutorial cutout for CI

Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2021-04-27 17:58:31 +02:00
Fabian Neumann
70078d03b2 add unresolved merge conflict to be addressed 2021-04-27 16:14:52 +02:00
Fabian Neumann
08674a6cda
Release v0.3 (#213)
* release_notes: order for release

* doc: fix smaller typos and tidy up

* config: bump version

* doc: fix line references

* doc: bump confpy version

* envs: update fixed versions yaml

* Snakefile: simplify all_elec to all
2020-12-05 17:54:50 +01:00
Fabian Neumann
2fc1ea0255
some refactoring and code cleaning (#190)
* some refactoring and code cleaning

* execute pre-commit

* pre-commit: limit large files

* add license to .pre-commit

* add pre-commit to env

* solve: tidy memory logger

* travis: add conda list for easier debugging

* undo config test/tutorial without plotting, rm matplotlibrc, .licenses

* remove {networks} wildcard

* unadd pre-commit config

* add release notes

* restore REUSE compliance

* fix docs environment python version

* env: remove gurobi from dependencies

* fix unclean merge block

* fix elif to if

* lighter rtd style

* lighter rtd style II
2020-12-03 19:50:53 +01:00
Martha Maria
cfb979a0d0
Custom busmap for clustering (#193)
* add option to use custom clustermaps from data folder

* adapt default config to custom busmap

* input file from Snakefile

* adapt input description

* add option to use custom clustermaps from data folder

* adapt default config to custom busmap

* input file from Snakefile

* adapt input description

* Snakefile: custom_busmap in cluster_network input is now csv

* cluster_network: custom_busmap is now read as csv file, adaptions of description

* simplify_network: adapt descriptions

* configfiles: add cutom_clustermaps switch

* unify clustarmap and busmap names

* unify clustermap and busmap names

* test/config: unify clustermap and busmap names

* cluster_network: make clustering_for_n_clusters compatible with simplify_network

* simplify_network: make compatible with changes in cluster_network.py

* Update scripts/cluster_network.py

* Update scripts/simplify_network.py

* Update scripts/simplify_network.py

* Update scripts/cluster_network.py

* Update scripts/cluster_network.py

* cluster_network: move custom_busmap flag to enable; simplify names

* cluster_network: move custom_busmap flag to enable; simplify names

* custom_busmap: add documentation

* cluster_network: add default for custom_busmap for compatibility with old configs

* cluster_network: add default for custom_busmap for compatibility with old configs

Co-authored-by: martha.frysztacki <eb5194@iai-esm003.iai.kit.edu>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2020-12-03 15:17:16 +01:00
FabianHofmann
9792069ab3
Update load processing (#211)
* build_load_data

* Add documentation

* updating load data import

* Update Config files

* Update load.csv

* Update add_electricity.py

* change log file name

* Update scripts/add_electricity.py

Co-authored-by: FabianHofmann <hofmann@fias.uni-frankfurt.de>

* Update scripts/build_load_data.py

Co-authored-by: FabianHofmann <hofmann@fias.uni-frankfurt.de>

* Update scripts/build_load_data.py

Co-authored-by: FabianHofmann <hofmann@fias.uni-frankfurt.de>

* Update scripts/build_load_data.py

Co-authored-by: FabianHofmann <hofmann@fias.uni-frankfurt.de>

* Update build_load_data.py

* Update build_load_data.py

* Update scripts/build_load_data.py

Co-authored-by: FabianHofmann <hofmann@fias.uni-frankfurt.de>

* update gap handling in build_load_data

* Update build_load_data.py

* Update config.test1.yaml

* update test.config

* Update config.tutorial.yaml

* update load csv function for load data

* Update build_load_data.py

* Update config.test1.yaml

* Update add_electricity.py

* Update build_load_data.py

* Added error messages if load data contains gaps after modifications

* general adjustments:
	- reduce data source to only entsoe statistics
	- sanitize code
	- adjust logging messages
	- adjust daocstrings

* update Snakefile config and docs

* update release notes
rename build_load -> build_load_data in config

* small follow up

* - reintroduce choice between powerstatistics and transparency
- remove load_ timeseries from databundle
- always build load_data
- reinsert scaling factor in config
- fix url to 2019 version

* update doc: configtable, release notes
update config.yaml

* follow up

Co-authored-by: Jan Frederick <jan.frederick.unnewehr@inatech.uni-freiburg.de>
Co-authored-by: JanFrederickUnnewehr <50404069+JanFrederickUnnewehr@users.noreply.github.com>
2020-12-03 12:49:04 +01:00
Fabian Neumann
de0478968c
config: make storage modelling with Store and Link default (#205) 2020-11-25 13:18:30 +01:00
Fabian Neumann
dacfc8a9a4 multiyear: fallbacks for hydro and load profiles 2020-10-20 13:53:43 +02:00
Fabian Neumann
cb2b78ad97 first step towards sweeping through multiple weather years 2020-10-18 14:25:12 +02:00
Fabian Neumann
f3b1d3d513 undo config test/tutorial without plotting, rm matplotlibrc, .licenses 2020-10-03 11:38:51 +02:00
Fabian Neumann
6deac6ced3 merge master 2020-10-03 11:15:39 +02:00
Fabian Neumann
ab6dc633bf
Options for regional equity and autarky (#166)
* solve: add option for equity constraints

* solve: scale more

* prepare: add option to limit line/link capacity

* solve: add inflow to EQ constraints

* solve: reindex inflow to match load

* update config files and references

* add autarky option

* move release notes

* add spillage to equity requirements

* prepare: fix accidental code removal

* prepare: add country autarky option

* consider snapshot_weightings for inflow

* trigger CI
2020-09-26 13:10:50 +02:00
Fabian Neumann
f6dd2e12f3 some refactoring and code cleaning 2020-09-11 12:40:53 +02:00
FabianHofmann
04f19f214d
fix emission prices (#171)
* fix emission prices

I'm not sure if the previous setup was intentional, but regarding that different generators might have different efficiencies and the emissions are carrier specific only, it makes more sense set net emission price.

* small fix

* update release_notes and config
2020-08-25 12:12:00 +02:00
Fabian Neumann
2627c96a9e
Prepare Release v0.2.0 (#165)
* prepare release v0.2.0

* prepare release v0.2.0 v2 [skip travis]
2020-06-08 20:43:35 +02:00
Fabian Neumann
5ea29551b9
add REUSE.software license information (#160)
* add REUSE.software license information

* add release note on licensing

* Update doc/index.rst

Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>

* Update doc/index.rst

Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>

* Update doc/release_notes.rst [skip travis]

Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>
2020-05-29 09:50:55 +02:00
euronion
2f85b0ede5
Separate retrieval and building of cutouts, natura raster into distinct configurations each. (#136)
* Separate config switches for retrieving and building cutout, natura raster.

* Include new switches in tutorial and test configs.

* Update release_notes.rst

* Update toplevel.csv

* Update tutorial.rst

* Update configuration.rst
2020-03-16 15:51:47 +01:00
Fabian Neumann
8bed137ca5 unify skip_iterations & skip_iterating to skip_iterations 2020-02-19 17:03:19 +01:00