Commit Graph

136 Commits

Author SHA1 Message Date
Lukas Trippe
0bf72cac6f
build: pin rasterio<1.4 (#1334)
* build: pin `rasterio<1.4`

* fix fixed updater

* run tests daily
2024-09-27 13:58:41 +02:00
Toni Seibold
51f8c2935a
Adding config for post discretization bugfix (#1309)
* Adding config for post discretization bugfix
2024-09-24 22:25:15 +02:00
Lukas Trippe
e65ce91a73
fix: failing macos pypsa installation (#1325) 2024-09-24 10:26:03 +02:00
Uzair Aftab
0d1c36ba6e
fix: move pypsa dependency to pip (#1322) 2024-09-24 08:42:01 +02:00
Lukas Trippe
5fdbe93f67
test: fix and update test workflows (#1296)
* test: fix and update test workflows

* resolve conflict

* bring back test.sh
2024-09-16 13:14:41 +02:00
cpschau
65054ae903
Simplify integration of unsustainable solid biomass (#1275)
* remove extra bus and link; remove redundant addition of oil carrier_bus

* remove redundant addition of oil carrier buses

* changed PyPSA requirement in envs/environment.yaml
2024-09-11 17:20:00 +02:00
Bobby Xiong
ac606c1f67
Build biomass transport costs from pre-extracted .csv instead of pdf (#1272)
* Removed tabula dependency for building biomass transport costs. Instead use pre-extracted .csvs. Updated rule and script accordingly.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updated script to optionally allow running the script with tabula-py dependency to recreate csv, if needed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-09-10 20:51:28 +02:00
Fabian Neumann
ca9556597e limit powerplantmatching version to below v0.6 due to upcoming breaking changes 2024-09-10 20:40:31 +02:00
Bobby Xiong
0c36de9bf8
Introducing OpenStreetMap high-voltage grid to PyPSA-Eur (#1079)
* Implemented  which uses the overpass API to download power features for individual countries.

* Extended  rule by input.

* Bug fixes and improvements to clean_osm_data.py. Added  in retrieve_osm_data.py.

* Updated clean_osm_data and retrieve_osm_data to create clean substations.

* Finished clean_osm_data function.

* Added check whether line is a circle. If so, drop it.

* Extended build_electricity.smk by build_osm_network.py

* Added build_osm_network

* Working osm-network-fast

* Bug fixes.

* Finalised and cleaned  including docstrings.

* Added try catch to retrieve_osm_data. Allows for parallelisation of downloads.

* Updated cleaning process.

* Set maximum number of threads for retrieving to 4, wrt. fair usage policy and potential request errors.

* Intermediate update on clean_osm_data.py. Added docstrings.

* Bug fix.

* Bug fix.

* Bug fixes in data types out of clean_osm_data

* Significant improvements to retrieve_osm_data, clean_osm_data. Cleaned code. Speed improvements

* Cleaned config.

* Fixes.

* Bug fixes.

* Updated default config

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Removed overpass from required packages. Not needed anymore.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added links_relations (route = power, frequency = 0) to retrieval. This will change how HVDC links are extracted in the near future.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Work-in-progress clean_osm_data

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added clean links output to clean_osm_data. Script uses OSM relations to retrieve clean HVDC links.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* New code for integrating HVDC links. Using relations. Base network implementation functioning.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* removed manual line dropping.

* Updated clean script

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* reverted Snakefile to default: sync settings

* added prebuilt functionality.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updated build_electricity.smk to work with scenario management.

* removed commented-out code.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* removed commented-out code.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fixed bug in pdf export by substituting pdf export with svg.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Bug-fix Snakefile

* dropped not needed columns from build_osm_network.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updated build_shapes, config.default and clean_osm_data.

* pre-commit changes.

* test

* Added initial prepare_osm_network_release.py script

* Finalised prepare_osm_network_release script to build clean and stable OSM base_network input files.

* Added new rules/development.smk

* Updated clean_osm_data to add substation_centroid to linestrings

* Updated clean_osm_data to add substation_centroid to linestrings

* Updated clean_osm_data to add substation_centroid to linestrings

* Updated clean_osm_data to add substation_centroid to linestrings

* Added osm-prebuilt functionality and zenodo sandbox repository.

* Updated clean_osm_data to geopandas v.1.01

* Made base_network and build_osm_network function more robust for empty links.

* Made base_network and build_osm_network function more robust for empty links.

* Bug fix in base_network. Voltage level null is now kept (relevant e.g. for Corsica)

* Merge with hcanges in upstream PR 1146. Fixing UA and MD.

* Updated Zenodo and fixed prepare_osm_network_release

* Updated osm network release.

* Updated prepare osm network release.

* Updated MD, UA scripts.

* Cleaned determine_availability_matrix_MD_UA.py, removed redundant code

* Bug fixes.

* Bug fixes for UA MD scripts.

* Rename of build script.

* Bug fix: only distribute load to buses with substation.

* Updated zenodo sandbox repository.

* Updated config.default

* Cleaned config.default.yaml: Related settings grouped together and redundant voltage settings aggregated.

* Cleaned config.default.yaml: Related settings grouped together and redundant voltage settings aggregated. Added release notes.

* Updated Zenodo repositories for OSM-prebuilt to offcial publication.

* Updated configtables

* Updated links.csv: Under_construction lines to in commission.

* Updated link 8394 and parameter_corrections: Continuation of North-Sea-Link.

* Major update: fix simplify_network, fix Corsica, updated build_osm_network to include lines overpassing nodes.

* remove config backup

* Bug fix: Carrier type of all supernodes corrected to 'AC'

* Bug fix: Carrier type of all supernodes corrected to 'AC'

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updated rules and base_network for compatibility with TYNDP projects.

* Updated Zenodo repository and prebuilt network to include 150 kV HVDC connections.

* Removed outdated config backup.

* Implemented all comments from PR #1079. Cleaned up OSM implementation.

* Bug fix: Added all voltages, 200 kV-750 kV, to default config.

* Cleaning and bugfixes.

* Updated Zenodo repository to https://zenodo.org/records/13358976. Added converter voltages, 'underground' property for DC lines/cables, and included Konti-Skan HVDC (DK-SE). Added compatibility with https://github.com/PyPSA/pypsa-eur/pull/1079 and https://github.com/PyPSA/pypsa-eur/pull/1085

* Apply suggestions from code review

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* simplify_network: handle complicated transformer topologies

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* syntax fix

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2024-08-22 15:01:20 +02:00
cpschau
0221372b49
modified environments to include tabula-py update and fixed PDF econding (#1219) 2024-08-15 14:52:33 +02:00
Philipp Glaum
a9f67b313f
handle new and upgraded TYNDP&NEP lines/links in base network (OSM compatible) (#1085)
* add general implementation to add tyndp and nep. TODO: Remove duplicate?

* pre_osm_merge

* clean and update base_network.py
update default config settings

* clean and update base_network.py
update default config settings

* base_network.py:remove adding of transmission projects
add_transmission_project.py: add new script for creating lines and link csv from transmission projects
add_electricity.py: add new projects from created csv files

* cluster_network.py: do not allow to group lines with different build years together-> requires pypsa update
simplify_network.py: fix bug of simplify links

* remove legacies of removing transmission projects from base_network

* restructure folders:new folder for tranmission project csv
add_transmission_projects: improve logging, cleanup

* update lines csvs and add default line type for upgraded and new lines

* remove duplicate lines which were already in gridkit

* allow to connect ac lines only to ac buses

* add manual links csv (adjusted links_tyndp.csv) and update default config

* add realease note

* remove links_tyndp.csv file, references in build_elec.smk and function in base_network.py to add them

* add configuration options for transmission projects to documentation and add template folder for transmission projects

* update pypsa version in environments

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* integrate Fabian's review comments 2

* add electricity:add import pathlib and duplicate printing out of adding line projects

* update NEP line csv

* address Fabian's comments

* build_transmission_projects: address Fabian's final comments
simplify_network: use modus to get line type which appears most often

* build_transmission_project: change from .geometry to ["geometry"]

* build_transmission_projects: remove redundanty line

* build_transmission_projects: remove buffer for europe shape because of higher resolution
default config: fix wrong key for skip argument in transmission_projects

* update configtables and default config

* update manual links csv and delete undetected duplicate links in tyndp2020

* final adjustments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2024-08-15 11:42:21 +02:00
Fabian Neumann
fb41016c60
EEZ: Update EEZ to v12, auto-download and remove from databundle (#1188)
* eez: update to version 12, autodownload, remove pycountry

* eez: do not simplify as it distorts topology

* remove missed merge conflicts
2024-08-07 15:28:55 +02:00
Fabian Neumann
c3151902f6
Compatibility with geopandas version 1 (#1136)
* address geopandas and pandas deprecations/warnings

* compatibility with geopandas v1

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-07-08 08:29:16 +02:00
Martha Frysztacki
358561a619
Update envs/environment.yaml
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2024-07-03 08:33:03 +02:00
Martha Frysztacki
061fd00bd1
Merge branch 'master' into fix_env 2024-07-01 15:10:18 +02:00
martacki
bba4857025 restrict geopandas until conflicts are resolved 2024-07-01 15:07:20 +02:00
Fabian Neumann
14b8de8348 compatibility with snakemake 8.14 closes #1108 2024-06-20 14:38:04 +02:00
Fabian Neumann
4b1ecca690 fix reuse of envs/*.yaml 2024-05-26 07:25:40 +02:00
Fabian Neumann
9b50c846a0 solve: add option for post-discretization of lines and links 2024-05-15 14:56:50 +02:00
Fabian Neumann
2b0303e913 bump powerplantmatching to 0.5.15 2024-05-12 18:06:50 +02:00
Fabian Neumann
03fd9bfb52
Merge pull request #1018 from koen-vg/existing-caps-from-pm
Use powerplantmatching IRENASTAT for renewable capacities in `add_existing_baseyear`
2024-04-30 15:15:37 +02:00
Fabian Neumann
edc1d4991a reenable windows ci with snakemake 8.11 2024-04-26 09:35:54 +02:00
Koen van Greevenbroek
884d308d8b Bump powerplantmatching version
Needed in order to get renewable capacities from pm.IRENA newer than
2020; updated to 2023 in version 0.5.13
2024-04-25 17:45:51 +02:00
Fabian Neumann
ed71978b92 use humanfriendly walltime specifications 2024-03-09 14:47:01 +01:00
Fabian Neumann
c8eb46a39c add generic cluster snakemake plugin 2024-03-09 12:07:15 +01:00
Fabian Neumann
d983d79ed2 Merge branch 'master' into snakemake-8 2024-03-08 17:50:08 +01:00
Fabian Neumann
b6542d275f add snakemake slurm plugin to env 2024-03-08 17:45:54 +01:00
Fabian Neumann
16e2286188 update copyright years to 2024 2024-02-19 16:21:48 +01:00
Fabian Neumann
cc77456195 env: add pre-commit 2024-02-02 10:11:59 +01:00
Fabian
bd55e368f7 test: revert setting highs as default solver, as not available for macos 2024-01-31 13:04:46 +01:00
Fabian
b7750d21ea CI: use scip and highs as solvers for clustering and solving, instead of ipopt and glpk 2024-01-31 09:44:13 +01:00
Fabian
56b22a3b4e env: correct pyscipopt dependency
cluster_network: address deprecation warning
2024-01-30 10:29:08 +01:00
Fabian
81e7c4eb67 remove pyomo dependency in cluster network, use scip as OS solver 2024-01-29 12:08:56 +01:00
Fabian Neumann
9f0555863b powerplantmatching: block version 0.5.9 with breaking bug 2024-01-23 18:19:46 +01:00
pre-commit-ci[bot]
27ee2666be [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-12-31 13:13:10 +00:00
Fabian Neumann
ea95f2b38c bumb minimum snakemake version to v8 2023-12-31 13:01:17 +01:00
Fabian Neumann
48850431fb Snakefile: switch to new storage providers 2023-12-30 14:37:45 +01:00
Fabian Neumann
4349903a72 env: remove snakemake upper version constraint 2023-12-30 14:36:18 +01:00
Fabian Neumann
9761988859 increase pypsa min version to 0.26.1 2023-12-29 11:32:53 +01:00
euronion
31f77bc888 Update environment.yaml 2023-12-21 17:52:34 +01:00
euronion
17a4f53b9d Restrict snakemake version to < 8.0.0 2023-12-21 11:57:24 +01:00
Fabian Hofmann
68aef7b628
constrain xarray more recent version 2023-12-18 11:23:32 +01:00
martacki
84d38228ae fix environment 2023-12-18 09:35:17 +01:00
Fabian Neumann
b96bcf11a4 relax pypsa version constraint to allos 0.26.0 2023-12-12 18:31:24 +01:00
Fabian Neumann
1b88dc7660 limit pypsa version until #793 resolved 2023-12-06 10:14:31 +01:00
Fabian Neumann
4df6370198 update environment: matplotlib, xarray, pypsa 2023-12-04 14:39:23 +01:00
Fabian Neumann
86bfdcf20c temporarily limit linopy version 2023-11-30 11:50:20 +01:00
Fabian Neumann
7884392326
Merge pull request #748 from PyPSA/follow-up-637
follow-up to #637: remove duplicate code and update tsam version
2023-10-18 10:27:29 +02:00
martacki
29fad3d638 change environment name back to pypsa-eur 2023-10-12 17:11:41 +02:00
martacki
9cde392227 hotfix to overcome xarray chunking error by reverting to an older version 2023-10-12 17:09:03 +02:00