Commit Graph

5235 Commits

Author SHA1 Message Date
Bobby Xiong
0c36de9bf8
Introducing OpenStreetMap high-voltage grid to PyPSA-Eur (#1079)
* Implemented  which uses the overpass API to download power features for individual countries.

* Extended  rule by input.

* Bug fixes and improvements to clean_osm_data.py. Added  in retrieve_osm_data.py.

* Updated clean_osm_data and retrieve_osm_data to create clean substations.

* Finished clean_osm_data function.

* Added check whether line is a circle. If so, drop it.

* Extended build_electricity.smk by build_osm_network.py

* Added build_osm_network

* Working osm-network-fast

* Bug fixes.

* Finalised and cleaned  including docstrings.

* Added try catch to retrieve_osm_data. Allows for parallelisation of downloads.

* Updated cleaning process.

* Set maximum number of threads for retrieving to 4, wrt. fair usage policy and potential request errors.

* Intermediate update on clean_osm_data.py. Added docstrings.

* Bug fix.

* Bug fix.

* Bug fixes in data types out of clean_osm_data

* Significant improvements to retrieve_osm_data, clean_osm_data. Cleaned code. Speed improvements

* Cleaned config.

* Fixes.

* Bug fixes.

* Updated default config

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Removed overpass from required packages. Not needed anymore.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added links_relations (route = power, frequency = 0) to retrieval. This will change how HVDC links are extracted in the near future.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Work-in-progress clean_osm_data

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added clean links output to clean_osm_data. Script uses OSM relations to retrieve clean HVDC links.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* New code for integrating HVDC links. Using relations. Base network implementation functioning.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* removed manual line dropping.

* Updated clean script

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* reverted Snakefile to default: sync settings

* added prebuilt functionality.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updated build_electricity.smk to work with scenario management.

* removed commented-out code.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* removed commented-out code.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fixed bug in pdf export by substituting pdf export with svg.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Bug-fix Snakefile

* dropped not needed columns from build_osm_network.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updated build_shapes, config.default and clean_osm_data.

* pre-commit changes.

* test

* Added initial prepare_osm_network_release.py script

* Finalised prepare_osm_network_release script to build clean and stable OSM base_network input files.

* Added new rules/development.smk

* Updated clean_osm_data to add substation_centroid to linestrings

* Updated clean_osm_data to add substation_centroid to linestrings

* Updated clean_osm_data to add substation_centroid to linestrings

* Updated clean_osm_data to add substation_centroid to linestrings

* Added osm-prebuilt functionality and zenodo sandbox repository.

* Updated clean_osm_data to geopandas v.1.01

* Made base_network and build_osm_network function more robust for empty links.

* Made base_network and build_osm_network function more robust for empty links.

* Bug fix in base_network. Voltage level null is now kept (relevant e.g. for Corsica)

* Merge with hcanges in upstream PR 1146. Fixing UA and MD.

* Updated Zenodo and fixed prepare_osm_network_release

* Updated osm network release.

* Updated prepare osm network release.

* Updated MD, UA scripts.

* Cleaned determine_availability_matrix_MD_UA.py, removed redundant code

* Bug fixes.

* Bug fixes for UA MD scripts.

* Rename of build script.

* Bug fix: only distribute load to buses with substation.

* Updated zenodo sandbox repository.

* Updated config.default

* Cleaned config.default.yaml: Related settings grouped together and redundant voltage settings aggregated.

* Cleaned config.default.yaml: Related settings grouped together and redundant voltage settings aggregated. Added release notes.

* Updated Zenodo repositories for OSM-prebuilt to offcial publication.

* Updated configtables

* Updated links.csv: Under_construction lines to in commission.

* Updated link 8394 and parameter_corrections: Continuation of North-Sea-Link.

* Major update: fix simplify_network, fix Corsica, updated build_osm_network to include lines overpassing nodes.

* remove config backup

* Bug fix: Carrier type of all supernodes corrected to 'AC'

* Bug fix: Carrier type of all supernodes corrected to 'AC'

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updated rules and base_network for compatibility with TYNDP projects.

* Updated Zenodo repository and prebuilt network to include 150 kV HVDC connections.

* Removed outdated config backup.

* Implemented all comments from PR #1079. Cleaned up OSM implementation.

* Bug fix: Added all voltages, 200 kV-750 kV, to default config.

* Cleaning and bugfixes.

* Updated Zenodo repository to https://zenodo.org/records/13358976. Added converter voltages, 'underground' property for DC lines/cables, and included Konti-Skan HVDC (DK-SE). Added compatibility with https://github.com/PyPSA/pypsa-eur/pull/1079 and https://github.com/PyPSA/pypsa-eur/pull/1085

* Apply suggestions from code review

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* simplify_network: handle complicated transformer topologies

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* syntax fix

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2024-08-22 15:01:20 +02:00
lisazeyen
e273f4ec9d
Merge pull request #1228 from PyPSA/co2-seq-per-period
Co2 sequestration potential depending on investment period
2024-08-20 15:33:18 +02:00
lisazeyen
7490647084 update release notes and doc 2024-08-20 15:06:02 +02:00
lisazeyen
5fb89068fd right intend for co2 seq potential 2024-08-20 15:06:02 +02:00
pre-commit-ci[bot]
2682c313ba [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-08-20 12:04:47 +00:00
lisazeyen
dcc84dfb9f update config 2024-08-20 14:02:12 +02:00
lisazeyen
3ad83c2097 Merge remote-tracking branch 'origin' 2024-08-20 13:51:44 +02:00
lisazeyen
f4819d0b4c co2 sequestration depending on period 2024-08-20 13:51:33 +02:00
lisazeyen
05dba2d36b
Merge pull request #1227 from PyPSA/industry-ratios-infinity-bug
avoid infinity in sector rations
2024-08-20 10:25:11 +02:00
pre-commit-ci[bot]
8b0474e41c [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-08-20 08:16:00 +00:00
lisazeyen
50c2fee6ce
avoid infinity in sector rations 2024-08-20 10:14:59 +02:00
lisazeyen
49053f3fb7
Merge pull request #1225 from PyPSA/fix-transport-data-year-selection
Make year selection more secure in transport data creation
2024-08-20 09:47:27 +02:00
pre-commit-ci[bot]
54a2d24517 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-08-19 11:39:36 +00:00
Fabian
ca47becc87 build_transport_data: select swiss years according to data selection
use index.unique instead of index.levels where needed
2024-08-19 13:33:28 +02:00
lisazeyen
885a881e78
bug fix for reference year<2018 for ammonia 2024-08-16 14:39:04 +02:00
lisazeyen
ad100c494b
increase CH transport data until 2022 2024-08-16 13:57:02 +02:00
lisazeyen
31c8907370
Merge pull request #1180 from PyPSA/country-specific-dh-forward-temperatures
Add option of country-specific district heating supply temperatures
2024-08-16 13:43:52 +02:00
pre-commit-ci[bot]
18759053f0 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-08-16 11:42:20 +00:00
lisazeyen
02fec080ee
Merge branch 'master' into country-specific-dh-forward-temperatures 2024-08-16 13:40:59 +02:00
lisazeyen
eb16d74a47
increase min cluster to 38 because of corsica link 2024-08-16 13:30:01 +02:00
Markus Millinger
ebbb75491b
Solid biomass to hydrogen process (#1213)
* Added solid biomass to hydrogen, bioH2, process

* Add doc for new config

* add release notes

* enable bioH2 also without biomass spatially resolved

* Update doc/release_notes.rst

Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>

* Added overnight cost

---------

Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2024-08-16 13:18:10 +02:00
Philipp Glaum
8480d62d33
fix bug in simplyfy network (#1222) 2024-08-16 13:16:52 +02:00
lisazeyen
b3f85d750f
Merge branch 'master' into country-specific-dh-forward-temperatures 2024-08-16 11:28:46 +02:00
Bobby Xiong
55cca6a0f9
Bug fix: Carrier type of added supernodes in simplify_network need to be set to "AC" (#1221)
* Fixed simplify_network.py to handle more complex topologies, i.e. if two links are connected to nearby AC buses separated by an AC line.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added fix for Corsica node: If region containing Corsica is modelled, include substation on Corsica as supernode (end point).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Bug fix: Carrier type of all supernodes corrected to 'AC'

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Bug fix: Carrier type of all supernodes corrected to 'AC'

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Bug fix: Carrier type of all supernodes corrected to 'AC'

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-15 19:45:49 +02:00
cpschau
0221372b49
modified environments to include tabula-py update and fixed PDF econding (#1219) 2024-08-15 14:52:33 +02:00
Fabian Hofmann
55370a57fb
retrieve: remove cutout protection (#1220) 2024-08-15 14:51:51 +02:00
Philipp Glaum
a9f67b313f
handle new and upgraded TYNDP&NEP lines/links in base network (OSM compatible) (#1085)
* add general implementation to add tyndp and nep. TODO: Remove duplicate?

* pre_osm_merge

* clean and update base_network.py
update default config settings

* clean and update base_network.py
update default config settings

* base_network.py:remove adding of transmission projects
add_transmission_project.py: add new script for creating lines and link csv from transmission projects
add_electricity.py: add new projects from created csv files

* cluster_network.py: do not allow to group lines with different build years together-> requires pypsa update
simplify_network.py: fix bug of simplify links

* remove legacies of removing transmission projects from base_network

* restructure folders:new folder for tranmission project csv
add_transmission_projects: improve logging, cleanup

* update lines csvs and add default line type for upgraded and new lines

* remove duplicate lines which were already in gridkit

* allow to connect ac lines only to ac buses

* add manual links csv (adjusted links_tyndp.csv) and update default config

* add realease note

* remove links_tyndp.csv file, references in build_elec.smk and function in base_network.py to add them

* add configuration options for transmission projects to documentation and add template folder for transmission projects

* update pypsa version in environments

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* integrate Fabian's review comments 2

* add electricity:add import pathlib and duplicate printing out of adding line projects

* update NEP line csv

* address Fabian's comments

* build_transmission_projects: address Fabian's final comments
simplify_network: use modus to get line type which appears most often

* build_transmission_project: change from .geometry to ["geometry"]

* build_transmission_projects: remove redundanty line

* build_transmission_projects: remove buffer for europe shape because of higher resolution
default config: fix wrong key for skip argument in transmission_projects

* update configtables and default config

* update manual links csv and delete undetected duplicate links in tyndp2020

* final adjustments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2024-08-15 11:42:21 +02:00
Fabian Neumann
f72afe7d10 bugfix: make sure to include Ukraine offshore shapes with new EEZ files 2024-08-14 16:27:06 +02:00
Bobby Xiong
1cf85b0be8
Fix for Corsica in simplify_network: Include local substation (#1215)
* Fixed simplify_network.py to handle more complex topologies, i.e. if two links are connected to nearby AC buses separated by an AC line.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added fix for Corsica node: If region containing Corsica is modelled, include substation on Corsica as supernode (end point).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-14 14:22:03 +02:00
Bobby Xiong
3ef6d519d7
Fix simplify_network.py to handle more complex topologies (#1211)
* Fixed simplify_network.py to handle more complex topologies, i.e. if two links are connected to nearby AC buses separated by an AC line.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-14 10:24:38 +02:00
Fabian Neumann
f263862455
use lower resolution EEZ shapes to reduce excessive RAM use (#1210) 2024-08-13 21:03:07 +02:00
Fabian Neumann
6ca8e80095
add option to use atlite to smooth wind turbine power curves (#1209) 2024-08-13 18:13:32 +02:00
pre-commit-ci[bot]
9243567872 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-08-13 15:08:49 +00:00
AmosSchledorn
a017802976 Merge remote-tracking branch 'origin/master' into country-specific-dh-forward-temperatures 2024-08-13 17:07:29 +02:00
AmosSchledorn
7542d57820 style: move get_country_from_node_name function from _helpers tu run 2024-08-13 17:07:20 +02:00
pre-commit-ci[bot]
d58a848f97
[pre-commit.ci] pre-commit autoupdate (#1208)
updates:
- [github.com/psf/black-pre-commit-mirror: 24.4.2 → 24.8.0](https://github.com/psf/black-pre-commit-mirror/compare/24.4.2...24.8.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-13 16:46:34 +02:00
Bobby Xiong
c79a9faa2c
Updated under_construction status of links that are commissioned by now (#1205)
* Updated links 14802, 14801, 12931, 14803, 14848, 14804: under_construction=False

* Updated links 14802, 14801, 14803, 14848, 14804: under_construction=False

* Updated link 8394 and parameter_corrections: Continuation of North-Sea-Link.
2024-08-12 16:56:10 +02:00
lisazeyen
07fe3ca9e2
Merge pull request #1167 from PyPSA/jrc-idees-2020
Jrc idees 2021
2024-08-12 15:39:52 +02:00
lisazeyen
6686e6e196
Update doc/release_notes.rst
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2024-08-12 15:27:24 +02:00
Philipp Glaum
a76cb29964 fix spelling mistake 2024-08-12 11:26:18 +02:00
Philipp Glaum
aa91c7b246 add documentation for configuration 2024-08-12 11:11:05 +02:00
Philipp Glaum
560929a2a8 add release note 2024-08-12 10:55:32 +02:00
Philipp Glaum
d14ced2549 cleanup 2024-08-12 10:43:13 +02:00
Philipp Glaum
5c577b6290 remove methanol transport because it has no impact 2024-08-12 10:38:01 +02:00
Philipp Glaum
77385721b6 Merge remote-tracking branch 'upstream/master' into add_methanol_techs 2024-08-12 09:52:25 +02:00
pre-commit-ci[bot]
156eccefb7 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-08-09 12:24:39 +00:00
lisazeyen
9b8139abbe add docstring 2024-08-09 14:24:12 +02:00
lisazeyen
c5256c1f37 add coke oven transformation output 2024-08-09 14:24:12 +02:00
lisazeyen
fe9185ef3f
Merge branch 'master' into jrc-idees-2020 2024-08-09 14:17:25 +02:00
lisazeyen
70db93fd32 correct sheet name chemical feedstocks 2024-08-09 13:14:47 +02:00