Commit Graph

2818 Commits

Author SHA1 Message Date
Tom Brown
0bcb215597 use JRC-IDEES thermal energy service instead of FE for buildings
This change consists of:

- reading the final energy (FE) to thermal energy service (TES)
  efficiency for each each country, seperately for gas and oil (this
  efficiency represents e.g. the losses in older non-condensing
  boilers)

- using TES instead of FE for the n.loads, so that it is pure energy
  service instead of including losses in legacy equipment

- using the stored efficiencies for baseyear equipment in
  add_existing_baseyear

In the baseyear (e.g. 2020) this should have no effect on FE, since
the reduction to TES is conpensated by the lower efficiencies of
existing equipment.

In the future (e.g. 2050) this leads to a reduction in heating demand,
since new equipment is more efficient than existing.
2024-09-02 19:32:13 +02:00
Fabian Neumann
e815c36b5c revert ffill of ammonia production 2024-08-30 12:13:15 +02:00
Philipp Glaum
bc09f1df77
prepare_sector_network.py: add oil bus whenever industry is added (#1247)
* prepare_sector_network.py: add oil bus whenever industry is added

* prepare_sector_network: Move added oil and methnaol buses to top of "add_industry", and add oil buses where used in "add_biomass"
2024-08-30 12:06:11 +02:00
Fabian Neumann
2cf6951858 build_ammonia_production: forward fill missing values 2024-08-30 12:02:39 +02:00
Fabian Neumann
b207b4ebe8
remove unused geth2015 hydro capacities (#1246) 2024-08-30 10:26:48 +02:00
Fabian Neumann
1b2ec575bc
retrieve urban population fraction from World Bank API (#1248)
* retrieve urban population fraction from World Bank API

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-30 10:26:34 +02:00
pre-commit-ci[bot]
bca4f95a07 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-08-29 13:38:41 +00:00
Philipp Glaum
7665de61d3 prepare_sector_network: convert e_max_pu timeseries for msw stores to pd.DataFrame. 2024-08-29 15:34:44 +02:00
Fabian Neumann
550bda4b44 eurostat_co2: fix update to 2023 energy balances 2024-08-29 11:49:37 +02:00
Fabian Neumann
3f94f89a84
build_powerplants: can assign to all buses not just substations (#1239) 2024-08-26 15:38:30 +00:00
Fabian Neumann
bbff4297f9
temporarily disable n.shapes until memory issues resolved (#1238) 2024-08-26 15:16:06 +00:00
Fabian Neumann
cab85cbb61
update GEM Europe gas tracker to May 2024 version (#1235) 2024-08-26 13:04:15 +00:00
Fabian Neumann
b44f544d77 simplify_network: do not use mode for linetype inferral 2024-08-26 13:56:10 +02:00
Fabian Neumann
a357ba11e9
add capital cost for liquid carbonaceous fuel stores (closes #489) (#1234)
* add capital cost for liquid carbonaceous fuel stores (closes #489)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-26 11:21:15 +02:00
Bobby Xiong
ef0bbd5f37
Custom busmap (#1231)
* Added functionality to set custom_busmaps for different base_networks.

* Updated configtables and location for busmaps.

* Added release nots.

* removed run parameter in mock_snakemake (only used for debugging).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-23 17:35:46 +02:00
Fabian Neumann
73d1353608 cluster_network: fix if no clustering required 2024-08-23 09:17:21 +02:00
Bobby Xiong
0c36de9bf8
Introducing OpenStreetMap high-voltage grid to PyPSA-Eur (#1079)
* Implemented  which uses the overpass API to download power features for individual countries.

* Extended  rule by input.

* Bug fixes and improvements to clean_osm_data.py. Added  in retrieve_osm_data.py.

* Updated clean_osm_data and retrieve_osm_data to create clean substations.

* Finished clean_osm_data function.

* Added check whether line is a circle. If so, drop it.

* Extended build_electricity.smk by build_osm_network.py

* Added build_osm_network

* Working osm-network-fast

* Bug fixes.

* Finalised and cleaned  including docstrings.

* Added try catch to retrieve_osm_data. Allows for parallelisation of downloads.

* Updated cleaning process.

* Set maximum number of threads for retrieving to 4, wrt. fair usage policy and potential request errors.

* Intermediate update on clean_osm_data.py. Added docstrings.

* Bug fix.

* Bug fix.

* Bug fixes in data types out of clean_osm_data

* Significant improvements to retrieve_osm_data, clean_osm_data. Cleaned code. Speed improvements

* Cleaned config.

* Fixes.

* Bug fixes.

* Updated default config

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Removed overpass from required packages. Not needed anymore.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added links_relations (route = power, frequency = 0) to retrieval. This will change how HVDC links are extracted in the near future.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Work-in-progress clean_osm_data

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added clean links output to clean_osm_data. Script uses OSM relations to retrieve clean HVDC links.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* New code for integrating HVDC links. Using relations. Base network implementation functioning.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* removed manual line dropping.

* Updated clean script

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* reverted Snakefile to default: sync settings

* added prebuilt functionality.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updated build_electricity.smk to work with scenario management.

* removed commented-out code.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* removed commented-out code.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fixed bug in pdf export by substituting pdf export with svg.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Bug-fix Snakefile

* dropped not needed columns from build_osm_network.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updated build_shapes, config.default and clean_osm_data.

* pre-commit changes.

* test

* Added initial prepare_osm_network_release.py script

* Finalised prepare_osm_network_release script to build clean and stable OSM base_network input files.

* Added new rules/development.smk

* Updated clean_osm_data to add substation_centroid to linestrings

* Updated clean_osm_data to add substation_centroid to linestrings

* Updated clean_osm_data to add substation_centroid to linestrings

* Updated clean_osm_data to add substation_centroid to linestrings

* Added osm-prebuilt functionality and zenodo sandbox repository.

* Updated clean_osm_data to geopandas v.1.01

* Made base_network and build_osm_network function more robust for empty links.

* Made base_network and build_osm_network function more robust for empty links.

* Bug fix in base_network. Voltage level null is now kept (relevant e.g. for Corsica)

* Merge with hcanges in upstream PR 1146. Fixing UA and MD.

* Updated Zenodo and fixed prepare_osm_network_release

* Updated osm network release.

* Updated prepare osm network release.

* Updated MD, UA scripts.

* Cleaned determine_availability_matrix_MD_UA.py, removed redundant code

* Bug fixes.

* Bug fixes for UA MD scripts.

* Rename of build script.

* Bug fix: only distribute load to buses with substation.

* Updated zenodo sandbox repository.

* Updated config.default

* Cleaned config.default.yaml: Related settings grouped together and redundant voltage settings aggregated.

* Cleaned config.default.yaml: Related settings grouped together and redundant voltage settings aggregated. Added release notes.

* Updated Zenodo repositories for OSM-prebuilt to offcial publication.

* Updated configtables

* Updated links.csv: Under_construction lines to in commission.

* Updated link 8394 and parameter_corrections: Continuation of North-Sea-Link.

* Major update: fix simplify_network, fix Corsica, updated build_osm_network to include lines overpassing nodes.

* remove config backup

* Bug fix: Carrier type of all supernodes corrected to 'AC'

* Bug fix: Carrier type of all supernodes corrected to 'AC'

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updated rules and base_network for compatibility with TYNDP projects.

* Updated Zenodo repository and prebuilt network to include 150 kV HVDC connections.

* Removed outdated config backup.

* Implemented all comments from PR #1079. Cleaned up OSM implementation.

* Bug fix: Added all voltages, 200 kV-750 kV, to default config.

* Cleaning and bugfixes.

* Updated Zenodo repository to https://zenodo.org/records/13358976. Added converter voltages, 'underground' property for DC lines/cables, and included Konti-Skan HVDC (DK-SE). Added compatibility with https://github.com/PyPSA/pypsa-eur/pull/1079 and https://github.com/PyPSA/pypsa-eur/pull/1085

* Apply suggestions from code review

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* simplify_network: handle complicated transformer topologies

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* syntax fix

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2024-08-22 15:01:20 +02:00
lisazeyen
7490647084 update release notes and doc 2024-08-20 15:06:02 +02:00
pre-commit-ci[bot]
2682c313ba [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-08-20 12:04:47 +00:00
lisazeyen
3ad83c2097 Merge remote-tracking branch 'origin' 2024-08-20 13:51:44 +02:00
lisazeyen
f4819d0b4c co2 sequestration depending on period 2024-08-20 13:51:33 +02:00
pre-commit-ci[bot]
8b0474e41c [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-08-20 08:16:00 +00:00
lisazeyen
50c2fee6ce
avoid infinity in sector rations 2024-08-20 10:14:59 +02:00
pre-commit-ci[bot]
54a2d24517 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-08-19 11:39:36 +00:00
Fabian
ca47becc87 build_transport_data: select swiss years according to data selection
use index.unique instead of index.levels where needed
2024-08-19 13:33:28 +02:00
lisazeyen
885a881e78
bug fix for reference year<2018 for ammonia 2024-08-16 14:39:04 +02:00
lisazeyen
ad100c494b
increase CH transport data until 2022 2024-08-16 13:57:02 +02:00
pre-commit-ci[bot]
18759053f0 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-08-16 11:42:20 +00:00
lisazeyen
02fec080ee
Merge branch 'master' into country-specific-dh-forward-temperatures 2024-08-16 13:40:59 +02:00
Markus Millinger
ebbb75491b
Solid biomass to hydrogen process (#1213)
* Added solid biomass to hydrogen, bioH2, process

* Add doc for new config

* add release notes

* enable bioH2 also without biomass spatially resolved

* Update doc/release_notes.rst

Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>

* Added overnight cost

---------

Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2024-08-16 13:18:10 +02:00
Philipp Glaum
8480d62d33
fix bug in simplyfy network (#1222) 2024-08-16 13:16:52 +02:00
lisazeyen
b3f85d750f
Merge branch 'master' into country-specific-dh-forward-temperatures 2024-08-16 11:28:46 +02:00
Bobby Xiong
55cca6a0f9
Bug fix: Carrier type of added supernodes in simplify_network need to be set to "AC" (#1221)
* Fixed simplify_network.py to handle more complex topologies, i.e. if two links are connected to nearby AC buses separated by an AC line.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added fix for Corsica node: If region containing Corsica is modelled, include substation on Corsica as supernode (end point).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Bug fix: Carrier type of all supernodes corrected to 'AC'

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Bug fix: Carrier type of all supernodes corrected to 'AC'

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Bug fix: Carrier type of all supernodes corrected to 'AC'

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-15 19:45:49 +02:00
cpschau
0221372b49
modified environments to include tabula-py update and fixed PDF econding (#1219) 2024-08-15 14:52:33 +02:00
Philipp Glaum
a9f67b313f
handle new and upgraded TYNDP&NEP lines/links in base network (OSM compatible) (#1085)
* add general implementation to add tyndp and nep. TODO: Remove duplicate?

* pre_osm_merge

* clean and update base_network.py
update default config settings

* clean and update base_network.py
update default config settings

* base_network.py:remove adding of transmission projects
add_transmission_project.py: add new script for creating lines and link csv from transmission projects
add_electricity.py: add new projects from created csv files

* cluster_network.py: do not allow to group lines with different build years together-> requires pypsa update
simplify_network.py: fix bug of simplify links

* remove legacies of removing transmission projects from base_network

* restructure folders:new folder for tranmission project csv
add_transmission_projects: improve logging, cleanup

* update lines csvs and add default line type for upgraded and new lines

* remove duplicate lines which were already in gridkit

* allow to connect ac lines only to ac buses

* add manual links csv (adjusted links_tyndp.csv) and update default config

* add realease note

* remove links_tyndp.csv file, references in build_elec.smk and function in base_network.py to add them

* add configuration options for transmission projects to documentation and add template folder for transmission projects

* update pypsa version in environments

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* integrate Fabian's review comments 2

* add electricity:add import pathlib and duplicate printing out of adding line projects

* update NEP line csv

* address Fabian's comments

* build_transmission_projects: address Fabian's final comments
simplify_network: use modus to get line type which appears most often

* build_transmission_project: change from .geometry to ["geometry"]

* build_transmission_projects: remove redundanty line

* build_transmission_projects: remove buffer for europe shape because of higher resolution
default config: fix wrong key for skip argument in transmission_projects

* update configtables and default config

* update manual links csv and delete undetected duplicate links in tyndp2020

* final adjustments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2024-08-15 11:42:21 +02:00
Fabian Neumann
f72afe7d10 bugfix: make sure to include Ukraine offshore shapes with new EEZ files 2024-08-14 16:27:06 +02:00
Bobby Xiong
1cf85b0be8
Fix for Corsica in simplify_network: Include local substation (#1215)
* Fixed simplify_network.py to handle more complex topologies, i.e. if two links are connected to nearby AC buses separated by an AC line.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added fix for Corsica node: If region containing Corsica is modelled, include substation on Corsica as supernode (end point).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-14 14:22:03 +02:00
Bobby Xiong
3ef6d519d7
Fix simplify_network.py to handle more complex topologies (#1211)
* Fixed simplify_network.py to handle more complex topologies, i.e. if two links are connected to nearby AC buses separated by an AC line.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-14 10:24:38 +02:00
pre-commit-ci[bot]
9243567872 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-08-13 15:08:49 +00:00
AmosSchledorn
a017802976 Merge remote-tracking branch 'origin/master' into country-specific-dh-forward-temperatures 2024-08-13 17:07:29 +02:00
AmosSchledorn
7542d57820 style: move get_country_from_node_name function from _helpers tu run 2024-08-13 17:07:20 +02:00
pre-commit-ci[bot]
156eccefb7 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-08-09 12:24:39 +00:00
lisazeyen
9b8139abbe add docstring 2024-08-09 14:24:12 +02:00
lisazeyen
c5256c1f37 add coke oven transformation output 2024-08-09 14:24:12 +02:00
lisazeyen
fe9185ef3f
Merge branch 'master' into jrc-idees-2020 2024-08-09 14:17:25 +02:00
lisazeyen
70db93fd32 correct sheet name chemical feedstocks 2024-08-09 13:14:47 +02:00
AmosSchledorn
6c9bcae6f0 Merge remote-tracking branch 'origin/master' into country-specific-dh-forward-temperatures 2024-08-08 17:20:23 +02:00
pre-commit-ci[bot]
a914366391 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-08-08 15:13:34 +00:00
AmosSchledorn
b31767cb20 Merge branch 'country-specific-dh-forward-temperatures' of https://github.com/PyPSA/pypsa-eur into country-specific-dh-forward-temperatures 2024-08-08 17:00:09 +02:00
AmosSchledorn
130ee49dbd add option to make central heating forward/return temperatures country-specific 2024-08-08 16:57:41 +02:00