Commit Graph

163 Commits

Author SHA1 Message Date
github-actions[bot]
f183736b09
[github-actions.ci] Update fixed environment (#1335)
* [create-pull-request] automated change

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: lkstrp <62255395+lkstrp@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-09-27 14:02:25 +02:00
Lukas Trippe
0bf72cac6f
build: pin rasterio<1.4 (#1334)
* build: pin `rasterio<1.4`

* fix fixed updater

* run tests daily
2024-09-27 13:58:41 +02:00
Toni Seibold
51f8c2935a
Adding config for post discretization bugfix (#1309)
* Adding config for post discretization bugfix
2024-09-24 22:25:15 +02:00
Lukas Trippe
e65ce91a73
fix: failing macos pypsa installation (#1325) 2024-09-24 10:26:03 +02:00
Uzair Aftab
0d1c36ba6e
fix: move pypsa dependency to pip (#1322) 2024-09-24 08:42:01 +02:00
Lukas Trippe
6387c46c2e
test: use makefile (#1315)
* test: use makefile

* more make commands
2024-09-23 10:13:20 +02:00
github-actions[bot]
f9bddc914e
[github-actions.ci] Update fixed environment (#1297)
* [create-pull-request] automated change

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: lkstrp <62255395+lkstrp@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-09-16 14:07:44 +02:00
Lukas Trippe
5fdbe93f67
test: fix and update test workflows (#1296)
* test: fix and update test workflows

* resolve conflict

* bring back test.sh
2024-09-16 13:14:41 +02:00
cpschau
65054ae903
Simplify integration of unsustainable solid biomass (#1275)
* remove extra bus and link; remove redundant addition of oil carrier_bus

* remove redundant addition of oil carrier buses

* changed PyPSA requirement in envs/environment.yaml
2024-09-11 17:20:00 +02:00
Bobby Xiong
ac606c1f67
Build biomass transport costs from pre-extracted .csv instead of pdf (#1272)
* Removed tabula dependency for building biomass transport costs. Instead use pre-extracted .csvs. Updated rule and script accordingly.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updated script to optionally allow running the script with tabula-py dependency to recreate csv, if needed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-09-10 20:51:28 +02:00
Fabian Neumann
ca9556597e limit powerplantmatching version to below v0.6 due to upcoming breaking changes 2024-09-10 20:40:31 +02:00
Philipp Glaum
30bcddb66a prepare_sector_networks: account correctely for msw co2
environment.yaml: add license comment
2024-09-03 14:10:20 +02:00
Fabian Neumann
3a1ee934da prepare release v0.12.0 2024-08-30 18:06:26 +02:00
Bobby Xiong
0c36de9bf8
Introducing OpenStreetMap high-voltage grid to PyPSA-Eur (#1079)
* Implemented  which uses the overpass API to download power features for individual countries.

* Extended  rule by input.

* Bug fixes and improvements to clean_osm_data.py. Added  in retrieve_osm_data.py.

* Updated clean_osm_data and retrieve_osm_data to create clean substations.

* Finished clean_osm_data function.

* Added check whether line is a circle. If so, drop it.

* Extended build_electricity.smk by build_osm_network.py

* Added build_osm_network

* Working osm-network-fast

* Bug fixes.

* Finalised and cleaned  including docstrings.

* Added try catch to retrieve_osm_data. Allows for parallelisation of downloads.

* Updated cleaning process.

* Set maximum number of threads for retrieving to 4, wrt. fair usage policy and potential request errors.

* Intermediate update on clean_osm_data.py. Added docstrings.

* Bug fix.

* Bug fix.

* Bug fixes in data types out of clean_osm_data

* Significant improvements to retrieve_osm_data, clean_osm_data. Cleaned code. Speed improvements

* Cleaned config.

* Fixes.

* Bug fixes.

* Updated default config

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Removed overpass from required packages. Not needed anymore.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added links_relations (route = power, frequency = 0) to retrieval. This will change how HVDC links are extracted in the near future.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Work-in-progress clean_osm_data

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added clean links output to clean_osm_data. Script uses OSM relations to retrieve clean HVDC links.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* New code for integrating HVDC links. Using relations. Base network implementation functioning.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* removed manual line dropping.

* Updated clean script

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* reverted Snakefile to default: sync settings

* added prebuilt functionality.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updated build_electricity.smk to work with scenario management.

* removed commented-out code.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* removed commented-out code.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fixed bug in pdf export by substituting pdf export with svg.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Bug-fix Snakefile

* dropped not needed columns from build_osm_network.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updated build_shapes, config.default and clean_osm_data.

* pre-commit changes.

* test

* Added initial prepare_osm_network_release.py script

* Finalised prepare_osm_network_release script to build clean and stable OSM base_network input files.

* Added new rules/development.smk

* Updated clean_osm_data to add substation_centroid to linestrings

* Updated clean_osm_data to add substation_centroid to linestrings

* Updated clean_osm_data to add substation_centroid to linestrings

* Updated clean_osm_data to add substation_centroid to linestrings

* Added osm-prebuilt functionality and zenodo sandbox repository.

* Updated clean_osm_data to geopandas v.1.01

* Made base_network and build_osm_network function more robust for empty links.

* Made base_network and build_osm_network function more robust for empty links.

* Bug fix in base_network. Voltage level null is now kept (relevant e.g. for Corsica)

* Merge with hcanges in upstream PR 1146. Fixing UA and MD.

* Updated Zenodo and fixed prepare_osm_network_release

* Updated osm network release.

* Updated prepare osm network release.

* Updated MD, UA scripts.

* Cleaned determine_availability_matrix_MD_UA.py, removed redundant code

* Bug fixes.

* Bug fixes for UA MD scripts.

* Rename of build script.

* Bug fix: only distribute load to buses with substation.

* Updated zenodo sandbox repository.

* Updated config.default

* Cleaned config.default.yaml: Related settings grouped together and redundant voltage settings aggregated.

* Cleaned config.default.yaml: Related settings grouped together and redundant voltage settings aggregated. Added release notes.

* Updated Zenodo repositories for OSM-prebuilt to offcial publication.

* Updated configtables

* Updated links.csv: Under_construction lines to in commission.

* Updated link 8394 and parameter_corrections: Continuation of North-Sea-Link.

* Major update: fix simplify_network, fix Corsica, updated build_osm_network to include lines overpassing nodes.

* remove config backup

* Bug fix: Carrier type of all supernodes corrected to 'AC'

* Bug fix: Carrier type of all supernodes corrected to 'AC'

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updated rules and base_network for compatibility with TYNDP projects.

* Updated Zenodo repository and prebuilt network to include 150 kV HVDC connections.

* Removed outdated config backup.

* Implemented all comments from PR #1079. Cleaned up OSM implementation.

* Bug fix: Added all voltages, 200 kV-750 kV, to default config.

* Cleaning and bugfixes.

* Updated Zenodo repository to https://zenodo.org/records/13358976. Added converter voltages, 'underground' property for DC lines/cables, and included Konti-Skan HVDC (DK-SE). Added compatibility with https://github.com/PyPSA/pypsa-eur/pull/1079 and https://github.com/PyPSA/pypsa-eur/pull/1085

* Apply suggestions from code review

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* simplify_network: handle complicated transformer topologies

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* syntax fix

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2024-08-22 15:01:20 +02:00
cpschau
0221372b49
modified environments to include tabula-py update and fixed PDF econding (#1219) 2024-08-15 14:52:33 +02:00
Philipp Glaum
a9f67b313f
handle new and upgraded TYNDP&NEP lines/links in base network (OSM compatible) (#1085)
* add general implementation to add tyndp and nep. TODO: Remove duplicate?

* pre_osm_merge

* clean and update base_network.py
update default config settings

* clean and update base_network.py
update default config settings

* base_network.py:remove adding of transmission projects
add_transmission_project.py: add new script for creating lines and link csv from transmission projects
add_electricity.py: add new projects from created csv files

* cluster_network.py: do not allow to group lines with different build years together-> requires pypsa update
simplify_network.py: fix bug of simplify links

* remove legacies of removing transmission projects from base_network

* restructure folders:new folder for tranmission project csv
add_transmission_projects: improve logging, cleanup

* update lines csvs and add default line type for upgraded and new lines

* remove duplicate lines which were already in gridkit

* allow to connect ac lines only to ac buses

* add manual links csv (adjusted links_tyndp.csv) and update default config

* add realease note

* remove links_tyndp.csv file, references in build_elec.smk and function in base_network.py to add them

* add configuration options for transmission projects to documentation and add template folder for transmission projects

* update pypsa version in environments

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* integrate Fabian's review comments 2

* add electricity:add import pathlib and duplicate printing out of adding line projects

* update NEP line csv

* address Fabian's comments

* build_transmission_projects: address Fabian's final comments
simplify_network: use modus to get line type which appears most often

* build_transmission_project: change from .geometry to ["geometry"]

* build_transmission_projects: remove redundanty line

* build_transmission_projects: remove buffer for europe shape because of higher resolution
default config: fix wrong key for skip argument in transmission_projects

* update configtables and default config

* update manual links csv and delete undetected duplicate links in tyndp2020

* final adjustments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2024-08-15 11:42:21 +02:00
Fabian Neumann
fb41016c60
EEZ: Update EEZ to v12, auto-download and remove from databundle (#1188)
* eez: update to version 12, autodownload, remove pycountry

* eez: do not simplify as it distorts topology

* remove missed merge conflicts
2024-08-07 15:28:55 +02:00
Fabian Neumann
c3151902f6
Compatibility with geopandas version 1 (#1136)
* address geopandas and pandas deprecations/warnings

* compatibility with geopandas v1

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-07-08 08:29:16 +02:00
Martha Frysztacki
358561a619
Update envs/environment.yaml
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2024-07-03 08:33:03 +02:00
Martha Frysztacki
061fd00bd1
Merge branch 'master' into fix_env 2024-07-01 15:10:18 +02:00
martacki
bba4857025 restrict geopandas until conflicts are resolved 2024-07-01 15:07:20 +02:00
Fabian Neumann
14b8de8348 compatibility with snakemake 8.14 closes #1108 2024-06-20 14:38:04 +02:00
Fabian Neumann
4b1ecca690 fix reuse of envs/*.yaml 2024-05-26 07:25:40 +02:00
Fabian Neumann
1da76dd1ae
prepare release v0.11.0 (#1081)
* prepare release notes v0.11.0

* streamline release notes

* update envs/environment.fixed.yaml

* doc: update workflow DAGs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* pre-commit formatting [no ci]

* update envs/environment.fixed.yaml

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-05-25 22:47:04 +02:00
Fabian Neumann
9b50c846a0 solve: add option for post-discretization of lines and links 2024-05-15 14:56:50 +02:00
Fabian Neumann
2b0303e913 bump powerplantmatching to 0.5.15 2024-05-12 18:06:50 +02:00
Fabian Hofmann
2db729f872
Update environment.fixed.yaml 2024-05-03 10:27:05 +02:00
Fabian
91e990c0b0 update env fixed 2024-05-03 09:36:40 +02:00
Fabian Neumann
03fd9bfb52
Merge pull request #1018 from koen-vg/existing-caps-from-pm
Use powerplantmatching IRENASTAT for renewable capacities in `add_existing_baseyear`
2024-04-30 15:15:37 +02:00
Fabian Neumann
edc1d4991a reenable windows ci with snakemake 8.11 2024-04-26 09:35:54 +02:00
Koen van Greevenbroek
884d308d8b Bump powerplantmatching version
Needed in order to get renewable capacities from pm.IRENA newer than
2020; updated to 2023 in version 0.5.13
2024-04-25 17:45:51 +02:00
Fabian Neumann
e19e38b149 update version requirements for retrieve.yaml 2024-03-11 08:59:26 +01:00
Fabian Neumann
ed71978b92 use humanfriendly walltime specifications 2024-03-09 14:47:01 +01:00
Fabian Neumann
c8eb46a39c add generic cluster snakemake plugin 2024-03-09 12:07:15 +01:00
Fabian Neumann
d983d79ed2 Merge branch 'master' into snakemake-8 2024-03-08 17:50:08 +01:00
Fabian Neumann
b6542d275f add snakemake slurm plugin to env 2024-03-08 17:45:54 +01:00
pre-commit-ci[bot]
ac2322cd16 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-02-20 10:03:34 +01:00
Fabian Neumann
f39fd1441a update fixed versions environment.yaml 2024-02-19 17:23:41 +01:00
Fabian Neumann
16e2286188 update copyright years to 2024 2024-02-19 16:21:48 +01:00
Fabian Neumann
cc77456195 env: add pre-commit 2024-02-02 10:11:59 +01:00
Fabian
bd55e368f7 test: revert setting highs as default solver, as not available for macos 2024-01-31 13:04:46 +01:00
Fabian
b7750d21ea CI: use scip and highs as solvers for clustering and solving, instead of ipopt and glpk 2024-01-31 09:44:13 +01:00
Fabian
56b22a3b4e env: correct pyscipopt dependency
cluster_network: address deprecation warning
2024-01-30 10:29:08 +01:00
Fabian
81e7c4eb67 remove pyomo dependency in cluster network, use scip as OS solver 2024-01-29 12:08:56 +01:00
Fabian Neumann
9f0555863b powerplantmatching: block version 0.5.9 with breaking bug 2024-01-23 18:19:46 +01:00
Koen van Greevenbroek
48974edc8d Make data retrieval rules use their own minimal conda env
This means that the main conda environment can be changed for
development purposes without necessitating the re-running of data
retrieval due to software environment change by default.
2024-01-23 16:27:12 +01:00
euronion
91843ace93 Restore REUSE compatability for environment.fixed.yaml 2024-01-08 09:48:06 +01:00
pre-commit-ci[bot]
6e630a85e1 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-01-08 08:33:52 +00:00
Fabian Neumann
b2744bc0ca update envs/environment.fixed.yaml 2024-01-05 17:36:35 +01:00
pre-commit-ci[bot]
27ee2666be [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-12-31 13:13:10 +00:00