Commit Graph

163 Commits

Author SHA1 Message Date
Fabian Hofmann
10b564fc57
Merge branch 'master' into rasterio-1.3 2022-09-19 12:40:58 +02:00
Fabian Neumann
b5b88c6554 lift rasterio restriction to allow atlite 0.2.9 2022-09-16 16:14:30 +02:00
pre-commit-ci[bot]
5d1ef8a640 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2022-09-16 13:04:06 +00:00
Fabian Neumann
61f4f397e4 remove non-essential packages 2022-09-10 15:41:21 +02:00
Fabian Neumann
c13d3a520c settle envs/environment.yaml 2022-09-10 15:38:30 +02:00
Fabian Neumann
ed6d93f6f7 add missing SPDX license identifier 2022-09-10 15:36:01 +02:00
Fabian Neumann
5f08dca2fd prepare release 0.6.0 2022-09-10 15:31:43 +02:00
Fabian Neumann
eb72d8aaad add release 0.5 envs/environment.fixed.yaml 2022-07-27 13:08:12 +02:00
Fabian Neumann
8d132464bd
Merge branch 'release-v0.5' into introduce_modularity_clustering 2022-07-27 09:04:39 +02:00
Fabian Neumann
8fc0c57ed6 update copyright years 2022-07-26 15:05:59 +02:00
Fabian Neumann
e523cb4cb5 doc: minor updates 2022-07-26 15:01:21 +02:00
Fabian Neumann
239dcfdd1e
Merge branch 'master' into introduce_modularity_clustering 2022-07-26 14:04:56 +02:00
Fabian Neumann
e3555523d8 env: use new pypsa 0.20 2022-07-26 14:04:06 +02:00
Fabian Neumann
84c8a9bc0e env: add pyxlsb to read .xslb files with pandas 2022-07-23 09:24:03 +02:00
Fabian Neumann
6047c8d750 add mergedeep to dependencies 2022-07-20 11:47:07 +02:00
martacki
387271b03b new spatial clustering method: modularity 2022-07-13 14:51:48 +02:00
Fabian Neumann
74f8ca1c7f
Merge branch 'master' into eu-energy-security 2022-06-28 08:09:39 +02:00
Fabian Neumann
971e970250
Merge branch 'master' into landlock-fix-alterative 2022-06-28 07:47:13 +02:00
Fabian Neumann
8e57d68b73
Merge branch 'master' into landlock-fix-alterative 2022-06-27 09:44:42 +02:00
Fabian Neumann
5ca231824e
Merge branch 'master' into eu-energy-security 2022-06-27 08:58:37 +02:00
Fabian Neumann
4c5ec8e5f1
Merge branch 'master' into introduce_hac_clustering 2022-06-24 20:22:47 +02:00
Fabian Neumann
d321b2b762 resolve merge conflicts 2022-06-24 20:15:32 +02:00
Fabian
c42d2bd97d refactor save to geojson file functionality to allow import/export of empty geodfs 2022-06-24 15:11:21 +02:00
Fabian
95e8a9534a env: remove duplicated ppm dependency 2022-06-23 16:07:33 +02:00
Fabian Hofmann
619b471f72
Merge branch 'master' into eu-energy-security 2022-06-07 10:05:11 +02:00
Max Parzen
21f627c74e update version 2022-06-03 17:05:32 +01:00
Max Parzen
798c015bf6 restrict rasterio version 2022-06-03 17:03:10 +01:00
Ebbe Kyhl
3bb8a7967a
Powerplantmatching version 0.5.3 as lower bound 2022-05-30 08:15:54 +02:00
Ebbe Kyhl
4706422f4b
Update version of powerplantmatching
Hi,

I recently became aware that I was using an older version (0.4.8) of the powerplantmatching. I tested my setup with the newer version (0.5.3), and it runs without any issues.

The following is a comment/question on the powerplantmatching dataset, which maybe is relevant to mention:
With regard to pumped-hydro storage (PHS), the newest version of powerplantmatching entails an energy storage capacity of 4.3 TWh (Europe-aggregate, assuming 6-hours duration for plants that do not have duration specified). In the earlier version 0.4.8, this was vastly overestimated at 10 TWh. As comparison, Geth et al. (2015) showed 1.3 TWh (including Norway and Switzerland) using 2012-numbers. PHS power capacity has increased from roughly 50 to 55 GW from 2014 to 2020 (iha, 2015, 2021), so energy storage capacity most likely is increased as well. But is it fair to say that energy storage capacity has been quadrupled since 2012 (from 1.3 TWh to 4.3 TWh)? Or how can we interpret this difference? 

Sources:
Geth et al., 2015, https://doi.org/10.1016/j.rser.2015.07.145 
iha, 2015, https://www.aler-renovaveis.org/contents/lerpublication/iha_2015_sept_hydropower-status-report.pdf
iha, 2021, https://assets-global.website-files.com/5f749e4b9399c80b5e421384/60c37321987070812596e26a_IHA20212405-status-report-02_LR.pdf
2022-05-27 16:14:01 +02:00
Fabian
e5cb2d34fb env: update to powerplantmatching >= 0.5.3 2022-04-08 11:30:29 +02:00
Fabian
b1143dc39b env: update to powerplantmatching >= v0.5.2 2022-04-07 21:48:00 +02:00
Fabian Neumann
9812e64e82 resolve cherry merge conflict 2 2022-04-05 16:34:02 +02:00
Fabian
7bc9b8012c powerplants: update to ppm >= v0.5.1 2022-04-04 18:07:48 +02:00
Fabian Neumann
a217415c39 env: bump minimum version number of atlite due to new cutouts with azimuth wind 2022-04-03 09:42:35 +02:00
Fabian Neumann
621520a7b2
Merge branch 'master' into rm-raster-natura 2022-03-29 09:18:43 +02:00
davide-f
9eac6d9bbe Fix environment 2022-03-28 21:45:03 +02:00
Fabian Neumann
d3589be761 test explicit fiona for Windows 2022-03-28 13:17:21 +02:00
Martha Frysztacki
3e07fdfd54
Merge branch 'master' into introduce_hac_clustering 2022-03-07 11:44:29 +01:00
martacki
f02d5fe821 fix clustering setup for hac according to fneum suggestions 2022-03-07 11:43:04 +01:00
euronion
9f71af89ad Update environment.yaml 2022-03-04 21:15:42 +01:00
Fabian Neumann
ca111297d7
Merge pull request #311 from PyPSA/test-pd-1.4
test pandas 1.4 compatibility for updated environment.yaml
2022-02-18 10:09:58 +01:00
Martha Frysztacki
25b3245923
Merge branch 'master' into introduce_hac_clustering 2022-02-04 14:35:14 +01:00
Fabian Neumann
72e277a007
update environment and address deprecations (#291)
* update environment and address deprecations

* check pandas<1.3

* limit snakemake due to ascii encoding error, address review comments

* remove version restriction on snakemake
2022-01-14 07:43:21 +00:00
euronion
b660277e37
Merge pull request #303 from PyPSA/misc/improve-ci-speed
[DNMY] Improve CI performance.
2022-01-13 18:25:12 +01:00
martacki
299e71e2b3 introduce hierarchical agglomeratice clustering (hac) 2021-11-17 13:46:33 +01:00
Fabian Neumann
54e35a3c86 update environment.fixed.yaml 2021-09-22 16:49:30 +02:00
Fabian Neumann
785caecdb5 no proj 8 2021-09-20 11:18:47 +02:00
Fabian Neumann
da883b1283 limit pyproj to 3.1.0 2021-09-20 11:12:26 +02:00
Fabian Neumann
ede9146735 limit pyproj 2021-09-20 11:02:57 +02:00
Fabian Neumann
2e02af8b4b change license 2021-09-14 16:37:41 +02:00
Fabian Neumann
b88322587f
Release 0.4 (#274)
* update release notes [skip ci]

* update fixed versions environment.yaml

* format release notes
2021-09-14 16:15:25 +02:00
Fabian Neumann
1d68a300e5
update environment.yaml for pypsa 0.18.0 (#268)
* update environment.yaml

* test pandas<1.3

Co-authored-by: Fabian Hofmann <hofmann@fias.uni-frankfurt.de>
2021-09-14 14:14:31 +02:00
Fabian Neumann
b82c55543b
Readthedocs documentation with pip (#267)
* Use pip with requirements.txt file for setting up doc environment

* Try w/o cartopy

* Try w/o cartopy take 2

* Try w/o cartopy take 3

* try pip

* try pip ii

* python 3.8

* fix links

* remove conda docs and fix line references

* remove conda on .readthedocs.yml

* correct ambiguous line reference

Co-authored-by: Jonas Hoersch <coroa@posteo.de>
2021-08-13 10:11:35 +02:00
Fabian Neumann
7a8ef1a2d4
Add tabula-py to dependencies (for PyPSA-Eur-Sec) 2021-08-09 14:37:51 +02:00
Fabian Neumann
4cc5e49ca8
Adapt to new snapshot weightings (#259)
* for now use n.snapshot_weightings.generators

* require pypsa master; use .objective for Nyears

* implement suggestions from code review

* add release note
2021-08-06 13:43:12 +00:00
Fabian Neumann
e2c3118f08 don't allow pandas 1.3.0 yet 2021-07-12 08:50:37 +02:00
Fabian Neumann
d094119d47
List first-order dependencies and add pypsa-eur-sec specialties (#257)
* env: list first-order dependencies and add pypsa-eur-sec specialties

* limit numpy version and require atlite 0.2.5

* fix accidental 0 major numpy version

* add future dependency country_converter
2021-06-29 08:45:09 +02:00
FabianHofmann
b8a7685931
Atlite availability (#224)
* adjust buil_cutout.py and Snakefile

* try adjusting build_renewable_profiles, currently crashing due to weird pyproj error

* build_renewable_profiles: -remove printing gid

* build_renewable_profiles: use dask for paralellization, use dense functions

* build_renewable_profiles:
	- revise imports
	- add logging for long calculation
	- revise explaining comment
	- revise distance calculation

* build profiles: adjust to cutout.grid

* * fix area to square km
* rename potmatrix -> capacity_potential
* rename available to availibility

* config.default update cutout params
build_renewable_potentials: major refactoring and simplification
hydro_profiles: update code

* build profiles: fix weight output dimensions

* build profiles: fix typo, fix selection of buses

* build profiles: reinsert paths variable

* follow up

* build profiles: move to dask calculation only

* CI: set build cutout to true (add CDSAPI)

* build profiles: use pyproj, test with gleas and geokit upstream

* environment.yaml fix atlite version

* build profiles: use dask 'processes' for more than 25 regions

* build profiles: specify dask scheduler according to number of regions

* backpedal a bit, only allow scheduler='processes'

* follow up, code style and fixup

* build profiles: add logger info for underwater fraction calc

* config adjust cutout parameters
Snakefile fixup

* config.default.yaml: adjust resolution

* config: use one cutout in total
build_cutout: automatic detetection of geographical boundaries

* env: add python>=3.8 requirement
build_cutout: fixup for region bound

* config: allow base cutout

* folllow up, fix up

* follow up II

* clean up

* clean up II

* build profiles: move back to multiprocessing due to performance issues

* small code style corrections

* move in pool context

* swqitch to ratsterio

* switch to rasterio for availibility calculation

* tiny fixup

* * build continental raster for offshore distance calculation
* adjust Snakefile to new script build_raster

* rename continental raster to onshore raster
add projected_mask function (not yet tested)
add docstrings, modularize

* Snakefile: remove build_onhore_raster rule, build mask directly from geometry instead
build_natura_raster: adjust code, add function for exporting
build_profiles:
	* add buffer to shore distance to init_globals function
	* update docstrings
	* improve handling of nodata grid codes
	* add geometry mask if natura raster not activated
	(the 255 value is an 'eligible' value for the corine data base,
	do this for excluding data outside the shape)

* build_profiles: adjust docstrings

* update environment

* build profiles: fixup reproject woth padding

* follow up, small fixups

* fix resampling method

checkpoint: reproduces solar profile in tut data

* reintegrate plot map
code style

* config: rename cutout into "base"

* build profiles: adjust to new atlite code

* natura raster: small fixup

* build natura raster: compress tiff file

* config: adjust cutout names

* build profiles: cover case if no or partial overlap between natura raster and cutout

* config-tutorial: adjust cutout params

* buid-profifiles: fixup in gebco filter

* follow up

* update config files

* build profiles: select layoutmatrix != 0

* build profiles: speed up average_distance and underwaterfraction

* build profiles: fix typo

* update release notes
build_cutout: only build needed features

* update envs

* config: add temperature to sarah features

* temporary fix for atlite v0.2.1 and new xarray version release

* env: remove xarray specification

* * remove rule build_country_flh
* build profiles: remove sneaked in line

* doc: update configuration.rst (section atlite) and corresponding csv table

* release notes: fix quotes

* build profiles: use 3035 for area calculation

* Update envs/environment.docs.yaml

* Update scripts/build_cutout.py

* Update doc/release_notes.rst

Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>

* Update doc/configuration.rst

Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>

* Update scripts/build_cutout.py

Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>

* update release notes

* release notes: add deprecation of 'keep_all_available_areas'
build profiles: remove warning for 'keep_all_available_areas'

* build cutout: rearrage code, set buffer correctly

* Rename tutorial cutout to remove name clash with real cutout.

* Update release_notes.rst: Rename tutorial cutout.

* retrieve: update cutouts and downloads (alternative) (#237)

* retrieve: update cutouts and downloads

* retrieve: remove unnecessary import

* use snakemake remote file functionality

* Snakefile: update zenodo link

* update natura remote link (closes #234)

* env: update atlite version to 0.2.2

* env: fix dask version due to memory issues

* test: retrieve cutout instead of build

* test: use tutorial cutout for CI

Co-authored-by: euronion <42553970+euronion@users.noreply.github.com>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2021-04-27 17:58:31 +02:00
lukasnacken
4e44822514
Add openpyxl to environment.yaml to fix #215 (#216) 2021-01-13 17:24:42 +01:00
Fabian Neumann
9966c751bd envs: restore REUSE compliance [skip travis] 2020-12-07 09:45:14 +01:00
Fabian Neumann
08674a6cda
Release v0.3 (#213)
* release_notes: order for release

* doc: fix smaller typos and tidy up

* config: bump version

* doc: fix line references

* doc: bump confpy version

* envs: update fixed versions yaml

* Snakefile: simplify all_elec to all
2020-12-05 17:54:50 +01:00
FabianHofmann
3fa5bbad1c
Opsd renewable ppls (#212)
* fix clustering of offwind-ac and offwind-dc in sites option

* add release nodes

* attach renewable assets by location (lat and lon) from OPSD register to network

* adapt default config to changes

* undo changes from a differen PR in cluster_network.py

* undo changes from a different PR, add release notes for this PR

* correct release notes

* add comments for relevant settings in add_electricity.py

* adjust configtable for electricity to OPSD renewable option and add estimate_renewable_capacities_from_capacitiy_stats

* reset cluster_network to HEAD

* add_electricity: Capacity is float

* config: add GB to OPSD_VRE_countries

* review and modify implementation

* update release notes

* Update envs/environment.yaml

Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>

Co-authored-by: martha.frysztacki <eb5194@iai-esm003.iai.kit.edu>
Co-authored-by: eb5194 <martha.frysztacki@kit.edu>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2020-12-03 23:13:41 +01:00
Fabian Neumann
2fc1ea0255
some refactoring and code cleaning (#190)
* some refactoring and code cleaning

* execute pre-commit

* pre-commit: limit large files

* add license to .pre-commit

* add pre-commit to env

* solve: tidy memory logger

* travis: add conda list for easier debugging

* undo config test/tutorial without plotting, rm matplotlibrc, .licenses

* remove {networks} wildcard

* unadd pre-commit config

* add release notes

* restore REUSE compliance

* fix docs environment python version

* env: remove gurobi from dependencies

* fix unclean merge block

* fix elif to if

* lighter rtd style

* lighter rtd style II
2020-12-03 19:50:53 +01:00