0c36de9bf8
* Implemented which uses the overpass API to download power features for individual countries. * Extended rule by input. * Bug fixes and improvements to clean_osm_data.py. Added in retrieve_osm_data.py. * Updated clean_osm_data and retrieve_osm_data to create clean substations. * Finished clean_osm_data function. * Added check whether line is a circle. If so, drop it. * Extended build_electricity.smk by build_osm_network.py * Added build_osm_network * Working osm-network-fast * Bug fixes. * Finalised and cleaned including docstrings. * Added try catch to retrieve_osm_data. Allows for parallelisation of downloads. * Updated cleaning process. * Set maximum number of threads for retrieving to 4, wrt. fair usage policy and potential request errors. * Intermediate update on clean_osm_data.py. Added docstrings. * Bug fix. * Bug fix. * Bug fixes in data types out of clean_osm_data * Significant improvements to retrieve_osm_data, clean_osm_data. Cleaned code. Speed improvements * Cleaned config. * Fixes. * Bug fixes. * Updated default config * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Removed overpass from required packages. Not needed anymore. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Added links_relations (route = power, frequency = 0) to retrieval. This will change how HVDC links are extracted in the near future. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Work-in-progress clean_osm_data * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Added clean links output to clean_osm_data. Script uses OSM relations to retrieve clean HVDC links. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * New code for integrating HVDC links. Using relations. Base network implementation functioning. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * removed manual line dropping. * Updated clean script * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * reverted Snakefile to default: sync settings * added prebuilt functionality. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updated build_electricity.smk to work with scenario management. * removed commented-out code. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * removed commented-out code. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug in pdf export by substituting pdf export with svg. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Bug-fix Snakefile * dropped not needed columns from build_osm_network. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updated build_shapes, config.default and clean_osm_data. * pre-commit changes. * test * Added initial prepare_osm_network_release.py script * Finalised prepare_osm_network_release script to build clean and stable OSM base_network input files. * Added new rules/development.smk * Updated clean_osm_data to add substation_centroid to linestrings * Updated clean_osm_data to add substation_centroid to linestrings * Updated clean_osm_data to add substation_centroid to linestrings * Updated clean_osm_data to add substation_centroid to linestrings * Added osm-prebuilt functionality and zenodo sandbox repository. * Updated clean_osm_data to geopandas v.1.01 * Made base_network and build_osm_network function more robust for empty links. * Made base_network and build_osm_network function more robust for empty links. * Bug fix in base_network. Voltage level null is now kept (relevant e.g. for Corsica) * Merge with hcanges in upstream PR 1146. Fixing UA and MD. * Updated Zenodo and fixed prepare_osm_network_release * Updated osm network release. * Updated prepare osm network release. * Updated MD, UA scripts. * Cleaned determine_availability_matrix_MD_UA.py, removed redundant code * Bug fixes. * Bug fixes for UA MD scripts. * Rename of build script. * Bug fix: only distribute load to buses with substation. * Updated zenodo sandbox repository. * Updated config.default * Cleaned config.default.yaml: Related settings grouped together and redundant voltage settings aggregated. * Cleaned config.default.yaml: Related settings grouped together and redundant voltage settings aggregated. Added release notes. * Updated Zenodo repositories for OSM-prebuilt to offcial publication. * Updated configtables * Updated links.csv: Under_construction lines to in commission. * Updated link 8394 and parameter_corrections: Continuation of North-Sea-Link. * Major update: fix simplify_network, fix Corsica, updated build_osm_network to include lines overpassing nodes. * remove config backup * Bug fix: Carrier type of all supernodes corrected to 'AC' * Bug fix: Carrier type of all supernodes corrected to 'AC' * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updated rules and base_network for compatibility with TYNDP projects. * Updated Zenodo repository and prebuilt network to include 150 kV HVDC connections. * Removed outdated config backup. * Implemented all comments from PR #1079. Cleaned up OSM implementation. * Bug fix: Added all voltages, 200 kV-750 kV, to default config. * Cleaning and bugfixes. * Updated Zenodo repository to https://zenodo.org/records/13358976. Added converter voltages, 'underground' property for DC lines/cables, and included Konti-Skan HVDC (DK-SE). Added compatibility with https://github.com/PyPSA/pypsa-eur/pull/1079 and https://github.com/PyPSA/pypsa-eur/pull/1085 * Apply suggestions from code review * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * simplify_network: handle complicated transformer topologies * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * syntax fix --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
147 lines
3.5 KiB
Python
147 lines
3.5 KiB
Python
# -*- coding: utf-8 -*-
|
|
# SPDX-FileCopyrightText: : 2020-2024 The PyPSA-Eur Authors
|
|
#
|
|
# SPDX-License-Identifier: MIT
|
|
|
|
import logging
|
|
import os
|
|
|
|
import pandas as pd
|
|
import pypsa
|
|
from _helpers import configure_logging, set_scenario_config
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
|
|
BUSES_COLUMNS = [
|
|
"bus_id",
|
|
"voltage",
|
|
"dc",
|
|
"symbol",
|
|
"under_construction",
|
|
"x",
|
|
"y",
|
|
"country",
|
|
"geometry",
|
|
]
|
|
LINES_COLUMNS = [
|
|
"line_id",
|
|
"bus0",
|
|
"bus1",
|
|
"voltage",
|
|
"circuits",
|
|
"length",
|
|
"underground",
|
|
"under_construction",
|
|
"geometry",
|
|
]
|
|
LINKS_COLUMNS = [
|
|
"link_id",
|
|
"bus0",
|
|
"bus1",
|
|
"voltage",
|
|
"p_nom",
|
|
"length",
|
|
"underground",
|
|
"under_construction",
|
|
"geometry",
|
|
]
|
|
TRANSFORMERS_COLUMNS = [
|
|
"transformer_id",
|
|
"bus0",
|
|
"bus1",
|
|
"voltage_bus0",
|
|
"voltage_bus1",
|
|
"geometry",
|
|
]
|
|
CONVERTERS_COLUMNS = [
|
|
"converter_id",
|
|
"bus0",
|
|
"bus1",
|
|
"voltage",
|
|
"geometry",
|
|
]
|
|
|
|
|
|
def export_clean_csv(df, columns, output_file):
|
|
"""
|
|
Export a cleaned DataFrame to a CSV file.
|
|
|
|
Args:
|
|
df (pandas.DataFrame): The DataFrame to be exported.
|
|
columns (list): A list of column names to include in the exported CSV file.
|
|
output_file (str): The path to the output CSV file.
|
|
|
|
Returns:
|
|
None
|
|
"""
|
|
rename_dict = {
|
|
"Bus": "bus_id",
|
|
"Line": "line_id",
|
|
"Link": "link_id",
|
|
"Transformer": "transformer_id",
|
|
"v_nom": "voltage",
|
|
"num_parallel": "circuits",
|
|
}
|
|
|
|
if "converter_id" in columns:
|
|
rename_dict["Link"] = "converter_id"
|
|
|
|
df.reset_index().rename(columns=rename_dict).loc[:, columns].replace(
|
|
{True: "t", False: "f"}
|
|
).to_csv(output_file, index=False, quotechar="'")
|
|
|
|
return None
|
|
|
|
|
|
if __name__ == "__main__":
|
|
if "snakemake" not in globals():
|
|
from _helpers import mock_snakemake
|
|
|
|
snakemake = mock_snakemake("prepare_osm_network_release")
|
|
|
|
configure_logging(snakemake)
|
|
set_scenario_config(snakemake)
|
|
|
|
network = pypsa.Network(snakemake.input.base_network)
|
|
|
|
network.buses["dc"] = network.buses.pop("carrier").map({"DC": "t", "AC": "f"})
|
|
network.lines.length = network.lines.length * 1e3
|
|
network.links.length = network.links.length * 1e3
|
|
|
|
# Export to clean csv for release
|
|
logger.info(f"Exporting {len(network.buses)} buses to %s", snakemake.output.buses)
|
|
export_clean_csv(network.buses, BUSES_COLUMNS, snakemake.output.buses)
|
|
|
|
logger.info(
|
|
f"Exporting {len(network.transformers)} transformers to %s",
|
|
snakemake.output.transformers,
|
|
)
|
|
export_clean_csv(
|
|
network.transformers, TRANSFORMERS_COLUMNS, snakemake.output.transformers
|
|
)
|
|
|
|
logger.info(f"Exporting {len(network.lines)} lines to %s", snakemake.output.lines)
|
|
export_clean_csv(network.lines, LINES_COLUMNS, snakemake.output.lines)
|
|
|
|
# Boolean that specifies if link element is a converter
|
|
is_converter = network.links.index.str.startswith("conv") == True
|
|
|
|
logger.info(
|
|
f"Exporting {len(network.links[~is_converter])} links to %s",
|
|
snakemake.output.links,
|
|
)
|
|
export_clean_csv(
|
|
network.links[~is_converter], LINKS_COLUMNS, snakemake.output.links
|
|
)
|
|
|
|
logger.info(
|
|
f"Exporting {len(network.links[is_converter])} converters to %s",
|
|
snakemake.output.converters,
|
|
)
|
|
export_clean_csv(
|
|
network.links[is_converter], CONVERTERS_COLUMNS, snakemake.output.converters
|
|
)
|
|
|
|
logger.info("Export of OSM network for release complete.")
|