diff --git a/.gitignore b/.gitignore index a86ca9f3..c5a0ce0d 100644 --- a/.gitignore +++ b/.gitignore @@ -12,4 +12,8 @@ gurobi.log /notebooks /data /data/links_p_nom.csv -/cutouts \ No newline at end of file +/cutouts + +doc/_build + +config.yaml diff --git a/.readthedocs.yml b/.readthedocs.yml new file mode 100644 index 00000000..301e4829 --- /dev/null +++ b/.readthedocs.yml @@ -0,0 +1,4 @@ +version: 2 + +conda: + environment: environment.docs.yaml \ No newline at end of file diff --git a/README.md b/README.md index 3f708d58..10e9075f 100644 --- a/README.md +++ b/README.md @@ -1,21 +1,28 @@ +![Version](https://img.shields.io/github/tag-date/pypsa/pypsa-eur) +[![Documentation](https://readthedocs.org/projects/pypsa-eur/badge/?version=latest)](https://pypsa-eur.readthedocs.io/en/latest/?badge=latest) +![GitHub](https://img.shields.io/github/license/pypsa/pypsa-eur) +![Size](https://img.shields.io/github/repo-size/pypsa/pypsa-eur) +[![Zenodo](https://zenodo.org/badge/DOI/10.5281/zenodo.1246852.svg)](https://doi.org/10.5281/zenodo.1246852) +[![Gitter](https://badges.gitter.im/PyPSA/community.svg)](https://gitter.im/PyPSA/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge) + # PyPSA-Eur: An Open Optimisation Model of the European Transmission System + PyPSA-Eur is an open model dataset of the European power system at the transmission network level that covers the full ENTSO-E area. +The model is suitable both for operational studies and generation and transmission expansion planning studies. +The continental scope and highly resolved spatial scale enables a proper description of the long-range +smoothing effects for renewable power generation and their varying resource availability. -![PyPSA-Eur Grid Model](https://raw.githubusercontent.com/PyPSA/pypsa-eur/master/img/pypsa-eur-grid.png) - - -The model is described and partially validated in the paper +The model is described in the [documentation](https://pypsa-eur.readthedocs.io) +and in the paper [PyPSA-Eur: An Open Optimisation Model of the European Transmission System](https://arxiv.org/abs/1806.01613), 2018, [arXiv:1806.01613](https://arxiv.org/abs/1806.01613). -This repository contains the scripts and some of the data required to -automatically build the dataset from openly-available sources. +![PyPSA-Eur Grid Model](doc/img/base.png) -Already-built versions of the model can be found in the accompanying [Zenodo -repository](https://zenodo.org/record/1246851). +![PyPSA-Eur Grid Model Simplified](doc/img/elec_s_X.png) The model is designed to be imported into the open toolbox [PyPSA](https://github.com/PyPSA/PyPSA) for operational studies as @@ -33,168 +40,7 @@ The dataset consists of: - Electrical demand time series from the [OPSD project](https://open-power-system-data.org/). - Renewable time series based on ERA5 and SARAH, assembled using the [atlite tool](https://github.com/FRESNA/atlite). -- Geographical potentials for wind and solar generators based on land use (CORINE) and excluding nature reserves (Natura2000) are computed with the [vresutils library](https://github.com/FRESNA/vresutils). +- Geographical potentials for wind and solar generators based on land use (CORINE) and excluding nature reserves (Natura2000) are computed with the [vresutils library](https://github.com/FRESNA/vresutils) and the [glaes library](https://github.com/FZJ-IEK3-VSA/glaes). -Building the model with the scripts in this repository uses up to 20GB of -memory. Computing optimal investment and operation scenarios requires a strong -interior-point solver compatible with the modelling library -[PYOMO](https://github.com/Pyomo/pyomo) like Gurobi or CPLEX with up to 100GB of -memory (for the 356-bus approximation). - -This project is maintained by the [Energy System Modelling -group](https://www.iai.kit.edu/english/2338.php) at the [Institute for -Automation and Applied -Informatics](https://www.iai.kit.edu/english/index.php) at the -[Karlsruhe Institute of -Technology](http://www.kit.edu/english/index.php). It is currently -funded by the [Helmholtz -Association](https://www.helmholtz.de/en/). Previous versions were -developed by the [Renewable Energy -Group](https://fias.uni-frankfurt.de/physics/schramm/renewable-energy-system-and-network-analysis/) -at [FIAS](https://fias.uni-frankfurt.de/) to carry out simulations for -the [CoNDyNet project](http://condynet.de/), financed by the [German -Federal Ministry for Education and Research -(BMBF)](https://www.bmbf.de/en/index.html) as part of the [Stromnetze -Research -Initiative](http://forschung-stromnetze.info/projekte/grundlagen-und-konzepte-fuer-effiziente-dezentrale-stromnetze/). - -# Installation - -The steps are demonstrated as shell commands, where the path before the `%` sign denotes the -directory in which the commands following the `%` should be entered. - -Clone the repository using `git` (**to a directory without any spaces in the path**) -```shell -/some/other/path % cd /some/path/without/spaces -/some/path/without/spaces % git clone https://github.com/PyPSA/pypsa-eur.git -``` - -## Python dependencies -The python package requirements are curated in the conda [environment.yaml](environment.yaml) file. -The environment can be installed and activated using -```shell -.../pypsa-eur % conda env create -f environment.yaml -.../pypsa-eur % conda activate pypsa-eur # or source activate pypsa-eur on older linux installations -``` - -**Note that activation is local to the currently open shell! After opening a new terminal window, one needs to reissue the second command!** - -## Data dependencies -Not all data dependencies are shipped with the git repository (since git is not suited for handling large changing files). Instead we provide two separate data bundles: -1. [pypsa-eur-data-bundle.tar.xz](https://vfs.fias.science/d/0a0ca1e2fb/files/?p=/pypsa-eur-data-bundle.tar.xz) contains common GIS datasets like NUTS3 shapes, EEZ shapes, CORINE Landcover, Natura 2000 and also electricity specific summary statistics like historic per country yearly totals of hydro generation, GDP and POP on NUTS3 levels and per-country load time-series. It should be extracted in the `data` subdirectory (so that all files are in the `data/bundle` subdirectory) -```shell -.../pypsa-eur/data % curl -OL "https://vfs.fias.science/d/0a0ca1e2fb/files/?dl=1&p=/pypsa-eur-data-bundle.tar.xz" -.../pypsa-eur/data % tar xJf pypsa-eur-data-bundle.tar.xz -``` -2. [pypsa-eur-cutouts.tar.xz](https://vfs.fias.science/d/0a0ca1e2fb/files/?p=/pypsa-eur-cutouts.tar.xz) are spatiotemporal subsets of the European weather data from the [ECMWF ERA5](https://software.ecmwf.int/wiki/display/CKB/ERA5+data+documentation) reanalysis dataset and the [CMSAF SARAH-2](https://wui.cmsaf.eu/safira/action/viewDoiDetails?acronym=SARAH_V002) solar surface radiation dataset for the year 2013. They have been prepared by and are for use with the [atlite](https://github.com/FRESNA/atlite) tool. You can either generate them yourself using the `build_cutouts` snakemake rule or extract them directly in the `pypsa-eur` directory (extracting the bundle is recommended, since procuring the source weather data files for atlite is not properly documented at the moment): -```shell -.../pypsa-eur % curl -OL "https://vfs.fias.science/d/0a0ca1e2fb/files/?dl=1&p=/pypsa-eur-cutouts.tar.xz" -.../pypsa-eur % tar xJf pypsa-eur-cutouts.tar.xz -``` - -3. Optionally, you can download a rasterized version of the NATURA dataset [natura.tiff](https://vfs.fias.science/d/0a0ca1e2fb/files/?p=/natura.tiff&dl=1) and put it into the `resources` sub-directory. If you don't, it will be generated automatically, which takes several hours. - -```shell -.../pypsa-eur % curl -L "https://vfs.fias.science/d/0a0ca1e2fb/files/?p=/natura.tiff&dl=1" -o "resources/natura.tiff" -``` - -4. Optionally, if you want to save disk space, you can delete `data/pypsa-eur-data-bundle.tar.xz` and `pypsa-eur-cutouts.tar.xz` once extracting the bundles is complete. E.g. - -```shell -.../pypsa-eur % rm -rf data/pypsa-eur-data-bundle.tar.xz pypsa-eur-cutouts.tar.xz -``` - -# Script overview - -The model has several configuration options collected in the [config.yaml](config.yaml) file -located in the root directory. - -## Model workflow -The generation of the model is controlled by the workflow management system -[Snakemake](https://snakemake.bitbucket.io/). In a nutshell, one declares in the -`Snakefile` for each python script in the `scripts` directory a rule which -describes which files the scripts consume and produce. `snakemake` then runs the -scripts in the correct order and is able to track, what parts of the workflow -have to be regenerated, when a data file or script is updated. For instance, -with the [Snakefile of pypsa-eur](Snakefile), an invocation to -```shell -snakemake networks/elec_s_128.nc -``` -follows the dependency graph -![Dependency graph for network elec_s_128](img/dependencies-elec_s_128.png) - -## Building the network -In detail this means it has to run the independent scripts, -- `build_shapes` to generate GeoJSON files with country, exclusive economic zones and nuts3 shapes -- `build_cutout` to prepare smaller weather data portions from ERA5 for cutout `europe-2013-era5` and SARAH for cutout `europe-2013-sarah`. - -With these and the externally extracted `ENTSO-E online map topology`, it can build the PyPSA basis model -- `base_network` stored at `networks/base.nc` with all `buses`, HVAC `lines` and HVDC `links`, and in -- `build_bus_regions` determine the Voronoi cell of each substation. - -Then it hands these over to the scripts for generating renewable and hydro feedin data, -- `build_hydro_profile` for the hourly hydro energy availability, -- `build_renewable_potentials` for the landuse/natura2000 constrained installation potentials for PV and wind, -- `build_renewable_profiles` for the PV and wind hourly capacity factors in each Voronoi cell. -- `build_powerplants` uses [powerplantmatching](https://github.com/FRESNA/powerplantmatching) to determine today's thermal power plant capacities and then locates the closest substation for each powerplant. - -The central rule `add_electricity` then ties all the different data inputs together to a detailed PyPSA model stored in `networks/elec.nc`, containing: - -- Today's transmission topology and capacities (optionally including lines which are under construction according to the config settings `lines: under_construction` and `links: under_construction`) -- Today's thermal and hydro generation capacities (for the technologies listed in the config setting `electricity: conventional_carriers`) -- Today's load time-series (upsampled according to population and gross domestic product) - -It further adds extendable `generators` and `storage_units` with *zero* capacity for -- wind and pv installations with today's locational, hourly wind and solar pv capacity factors (but **no** capacities) -- long-term hydrogen and short-term battery storage units (if listed in `electricity: extendable_carriers`) -- additional open-cycle gas turbines (if `OCGT` is listed in `electricity: extendable_carriers`) - -The additional rules prepare approximations of the full model, in which generation, storage and transmission capacities can be co-optimized -- `simplify_network` transforms the transmission grid to a 380 kV-only equivalent network, while -- `cluster_network` uses a kmeans based clustering technique to partition the network into a certain number of zones and then reduce the network to a representation with one bus per zone. - -The simplification and clustering steps are described in detail in the paper -[The role of spatial scale in joint optimisations of generation and transmission for European highly renewable scenarios](https://arxiv.org/abs/1705.07617), 2017, [arXiv:1705.07617](https://arxiv.org/abs/1705.07617), [doi:10.1109/EEM.2017.7982024](https://doi.org/10.1109/EEM.2017.7982024). - -## Solving the network -After generating the network it can be solved by using 'solve_all_elec_networks'. This runs the following rules: -- 'cluster_network' -- 'prepare_network' -- 'solve_all_elec_networks' -- 'solve_network' - -## Summarising the results and making plots -The following rule can be used to summarize the results in seperate .csv files: -``` -snakemake results/summaries/elec_s_all_lall_Co2L-3H_all - ^ clusters - ^ line volume or cost cap - ^- options - ^- all countries -``` -the line volume/cost cap field can be set to one of the following: -* `lv1.25` for a particular line volume extension by 25% -* `lc1.25` for a line cost extension by 25 % -* `lall` for all evalutated caps -* `lvall` for all line volume caps -* `lcall` for all line cost caps - -Replacing '/summaries/' with '/plots/' creates nice colored maps of the results. - -# Solver choice -Default choice for the solver is Gurobi (freely available under academic license) or CPLEX. If you want to go fully opensource the CBC solver (https://projects.coin-or.org/Cbc) can be used. To install CBC run 'conda install -c conda-forge coincbc'. - -# Hints - -For the use of `snakemake`, it makes sense to familiarize oneself quickly with its [basic tutorial](https://snakemake.readthedocs.io/en/stable/tutorial/basics.html) and then read carefully through the section [Executing Snakemake](https://snakemake.readthedocs.io/en/stable/executable.html), noting the arguments `-n`, `-r`, but also `--dag`, `-R` and `-t`. - -The dependency graph shown above was generated using -```shell -snakemake --dag networks/elec_s_128.nc | dot -Tpng > dependency-graph-elec_s_128.png -``` - -# License - -The code in PyPSA-Eur is released as free software under the -[GPLv3](http://www.gnu.org/licenses/gpl-3.0.en.html), see -[LICENSE](LICENSE.txt). +Already-built versions of the model can be found in the accompanying [Zenodo +repository](https://zenodo.org/record/1246851). diff --git a/Snakefile b/Snakefile index c24bd250..b8cba3b9 100644 --- a/Snakefile +++ b/Snakefile @@ -93,7 +93,7 @@ rule build_bus_regions: script: "scripts/build_bus_regions.py" rule build_cutout: - output: "cutouts/{cutout}" + output: directory("cutouts/{cutout}") resources: mem=config['atlite'].get('nprocesses', 4) * 1000 threads: config['atlite'].get('nprocesses', 4) benchmark: "benchmarks/build_cutout_{cutout}" @@ -110,7 +110,9 @@ rule build_renewable_profiles: base_network="networks/base.nc", corine="data/bundle/corine/g250_clc06_V18_5.tif", natura="resources/natura.tiff", - gebco="data/bundle/GEBCO_2014_2D.nc", + gebco=lambda wildcards: ("data/bundle/GEBCO_2014_2D.nc" + if "max_depth" in config["renewable"][wildcards.technology].keys() + else []), country_shapes='resources/country_shapes.geojson', offshore_shapes='resources/offshore_shapes.geojson', regions=lambda wildcards: ("resources/regions_onshore.geojson" @@ -124,15 +126,16 @@ rule build_renewable_profiles: # group: 'feedin_preparation' script: "scripts/build_renewable_profiles.py" -rule build_hydro_profile: - input: - country_shapes='resources/country_shapes.geojson', - eia_hydro_generation='data/bundle/EIA_hydro_generation_2000_2014.csv', - cutout="cutouts/" + config["renewable"]['hydro']['cutout'] - output: 'resources/profile_hydro.nc' - resources: mem=5000 - # group: 'feedin_preparation' - script: 'scripts/build_hydro_profile.py' +if 'hydro' in config['renewable'].keys(): + rule build_hydro_profile: + input: + country_shapes='resources/country_shapes.geojson', + eia_hydro_generation='data/bundle/EIA_hydro_generation_2000_2014.csv', + cutout="cutouts/" + config["renewable"]['hydro']['cutout'] + output: 'resources/profile_hydro.nc' + resources: mem=5000 + # group: 'feedin_preparation' + script: 'scripts/build_hydro_profile.py' rule add_electricity: input: @@ -305,7 +308,9 @@ rule build_country_flh: base_network="networks/base.nc", corine="data/bundle/corine/g250_clc06_V18_5.tif", natura="resources/natura.tiff", - gebco="data/bundle/GEBCO_2014_2D.nc", + gebco=lambda wildcards: ("data/bundle/GEBCO_2014_2D.nc" + if "max_depth" in config["renewable"][wildcards.technology].keys() + else []), country_shapes='resources/country_shapes.geojson', offshore_shapes='resources/offshore_shapes.geojson', pietzker="data/pietzker2014.xlsx", diff --git a/config.yaml b/config.default.yaml similarity index 72% rename from config.yaml rename to config.default.yaml index 3900c50b..936dadea 100644 --- a/config.yaml +++ b/config.default.yaml @@ -4,18 +4,15 @@ logging_level: INFO summary_dir: results scenario: - sectors: [E] # ,E+EV,E+BEV,E+BEV+V2G] # [ E+EV, E+BEV, E+BEV+V2G ] + sectors: [E] simpl: [''] - #ll: ['v1.0', 'v1.09', 'v1.125', 'v1.18', 'v1.25', 'v1.35', 'v1.5', 'v1.7', 'v2.0', 'vopt'] # line limit a 'v' prefix means volume - ll: ['vopt', 'copt'] #['v1.0', 'v1.125', 'v1.25', 'v1.5', 'v2.0', 'vopt'] # line limit a 'v' prefix means volume - #ll: ['c1.0', 'v1.125', 'v1.25', 'v1.5', 'v2.0', 'vopt'] # line limit a 'v' prefix means volume - clusters: [37, 45, 64, 90, 128, 181, 256, 362, 512] # (2**np.r_[5.5:9:.5]).astype(int) - opts: [Co2L-3H] #, LC-FL, LC-T, Ep-T, Co2L-T] + ll: ['copt'] + clusters: [37, 100] + opts: [Co2L-3H] countries: ['AL', 'AT', 'BA', 'BE', 'BG', 'CH', 'CZ', 'DE', 'DK', 'EE', 'ES', 'FI', 'FR', 'GB', 'GR', 'HR', 'HU', 'IE', 'IT', 'LT', 'LU', 'LV', 'ME', 'MK', 'NL', 'NO', 'PL', 'PT', 'RO', 'RS', 'SE', 'SI', 'SK'] snapshots: - # arguments to pd.date_range start: "2013-01-01" end: "2014-01-01" closed: 'left' # end is not inclusive @@ -31,18 +28,13 @@ electricity: extendable_carriers: Generator: [OCGT] - StorageUnit: [battery, H2] # [CAES] + StorageUnit: [battery, H2] max_hours: battery: 6 H2: 168 - # estimate_renewable_capacities_from_capacity_stats: - # # Wind is the Fueltype in ppm.data.Capacity_stats, onwind, offwind-{ac,dc} the carrier in PyPSA-Eur - # Wind: [onwind, offwind-ac, offwind-dc] - # Solar: [solar] - - conventional_carriers: [] # nuclear, oil, OCGT, CCGT, coal, lignite, geothermal, biomass] + conventional_carriers: [] # [nuclear, oil, OCGT, CCGT, coal, lignite, geothermal, biomass] atlite: nprocesses: 4 @@ -65,15 +57,11 @@ renewable: resource: method: wind turbine: Vestas_V112_3MW - # ScholzPhd Tab 4.3.1: 10MW/km^2 - capacity_per_sqkm: 3 + capacity_per_sqkm: 3 # ScholzPhd Tab 4.3.1: 10MW/km^2 # correction_factor: 0.93 corine: - #The selection of CORINE Land Cover [1] types that are allowed for wind and solar are based on [2] p.42 / p.28 - # - #[1] https://www.eea.europa.eu/ds_resolveuid/C9RK15EA06 - # - #[2] Scholz, Y. (2012). Renewable energy based electricity supply at low costs: development of the REMix model and application for Europe. + # Scholz, Y. (2012). Renewable energy based electricity supply at low costs: + # development of the REMix model and application for Europe. ( p.42 / p.28) grid_codes: [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 31, 32] distance: 1000 @@ -91,7 +79,7 @@ renewable: corine: [44, 255] natura: true max_depth: 50 - max_shore_distance: 80000 + max_shore_distance: 30000 potential: simple # or conservative clip_p_max_pu: 1.e-2 offwind-dc: @@ -105,7 +93,7 @@ renewable: corine: [44, 255] natura: true max_depth: 50 - min_shore_distance: 80000 + min_shore_distance: 30000 potential: simple # or conservative clip_p_max_pu: 1.e-2 solar: @@ -116,16 +104,13 @@ renewable: orientation: slope: 35. azimuth: 180. - # ScholzPhd Tab 4.3.1: 170 MW/km^2 - capacity_per_sqkm: 1.7 - correction_factor: 0.854337 + capacity_per_sqkm: 1.7 # ScholzPhd Tab 4.3.1: 170 MW/km^2 # Determined by comparing uncorrected area-weighted full-load hours to those # published in Supplementary Data to # Pietzcker, Robert Carl, et al. "Using the sun to decarbonize the power # sector: The economic potential of photovoltaics and concentrating solar # power." Applied Energy 135 (2014): 704-720. - # Comparison is shown in resources/country_flh_aggregated_solar.csv - + correction_factor: 0.854337 corine: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 26, 31, 32] natura: true @@ -138,12 +123,12 @@ renewable: hydro_max_hours: "energy_capacity_totals_by_country" # one of energy_capacity_totals_by_country, # estimate_by_large_installations or a float clip_min_inflow: 1.0 + lines: types: 220.: "Al/St 240/40 2-bundle 220.0" 300.: "Al/St 240/40 3-bundle 300.0" 380.: "Al/St 240/40 4-bundle 380.0" - s_max_pu: 0.7 length_factor: 1.25 under_construction: 'zero' # 'zero': set capacity to zero, 'remove': remove, 'keep': with full capacity @@ -163,35 +148,27 @@ load: costs: year: 2030 - - # From a Lion Hirth paper, also reflects average of Noothout et al 2016 - discountrate: 0.07 - # [EUR/USD] ECB: https://www.ecb.europa.eu/stats/exchange/eurofxref/html/eurofxref-graph-usd.en.html # noqa: E501 - USD2013_to_EUR2013: 0.7532 - - # Marginal and capital costs can be overwritten - # capital_cost: - # Wind: Bla - marginal_cost: # + discountrate: 0.07 # From a Lion Hirth paper, also reflects average of Noothout et al 2016 + USD2013_to_EUR2013: 0.7532 # [EUR/USD] ECB: https://www.ecb.europa.eu/stats/exchange/eurofxref/html/eurofxref-graph-usd.en.html + marginal_cost: solar: 0.01 onwind: 0.015 offwind: 0.015 hydro: 0. H2: 0. battery: 0. - - emission_prices: # only used with the option Ep (emission prices) + emission_prices: # only used with the option Ep co2: 0. solving: options: formulation: kirchhoff - #load_shedding: true + load_shedding: false noisy_costs: true min_iterations: 3 max_iterations: 5 - #nhours: 10 clip_p_max_pu: 0.01 + #nhours: 10 solver: name: gurobi threads: 4 @@ -216,28 +193,21 @@ plotting: boundaries: [-10.2, 29, 35, 72] p_nom: bus_size_factor: 5.e+4 - linewidth_factor: 3.e+3 # 1.e+3 #3.e+3 + linewidth_factor: 3.e+3 costs_max: 800 costs_threshold: 1 - energy_max: 15000. energy_min: -10000. energy_threshold: 50. - vre_techs: ["onwind", "offwind-ac", "offwind-dc", "solar", "ror"] conv_techs: ["OCGT", "CCGT", "Nuclear", "Coal"] storage_techs: ["hydro+PHS", "battery", "H2"] - # store_techs: ["Li ion", "water tanks"] - load_carriers: ["AC load"] #, "heat load", "Li ion load"] + load_carriers: ["AC load"] AC_carriers: ["AC line", "AC transformer"] link_carriers: ["DC line", "Converter AC-DC"] - heat_links: ["heat pump", "resistive heater", "CHP heat", "CHP electric", - "gas boiler", "central heat pump", "central resistive heater", "central CHP heat", - "central CHP electric", "central gas boiler"] - heat_generators: ["gas boiler", "central gas boiler", "solar thermal collector", "central solar thermal collector"] tech_colors: "onwind" : "#235ebc" "onshore wind" : "#235ebc" @@ -332,22 +302,24 @@ plotting: "HVDC links" : "#8a1caf" "DC-DC" : "#8a1caf" "DC link" : "#8a1caf" - # _helpers.load_network requirements nice_names: - # OCGT: "Gas" - # OCGT marginal: "Gas (marginal)" - offwind: "offshore wind" - onwind: "onshore wind" - battery: "Battery storage" + OCGT: "Open-Cycle Gas" + CCGT: "Combined-Cycle Gas" + offwind-ac: "Offshore Wind (AC)" + offwind-dc: "Offshore Wind (DC)" + onwind: "Onshore Wind" + battery: "Battery Storage" + H2: "Hydrogen Storage" lines: "Transmission lines" - AC line: "AC lines" - AC-AC: "DC lines" ror: "Run of river" nice_names_n: - offwind: "offshore\nwind" - onwind: "onshore\nwind" - # OCGT: "Gas" - H2: "Hydrogen\nstorage" - # OCGT marginal: "Gas (marginal)" - lines: "transmission\nlines" - ror: "run of river" + OCGT: "Open-Cycle\nGas" + CCGT: "Combined-Cycle\nGas" + offwind-ac: "Offshore\nWind (AC)" + offwind-dc: "Offshore\nWind (DC)" + onwind: "Onshore\nWind" + battery: "Battery\nStorage" + H2: "Hydrogen\nStorage" + lines: "Transmission\nlines" + ror: "Run of\nriver" + \ No newline at end of file diff --git a/config.tutorial.yaml b/config.tutorial.yaml new file mode 100644 index 00000000..381293a9 --- /dev/null +++ b/config.tutorial.yaml @@ -0,0 +1,311 @@ +version: 0.1 +logging_level: INFO + +summary_dir: results + +scenario: + sectors: [E] + simpl: [''] + ll: ['copt'] + clusters: [2,6] + opts: [Co2L-24H] + +countries: ['DE'] + +snapshots: + start: "2013-03-01" + end: "2014-04-01" + closed: 'left' # end is not inclusive + +enable: + powerplantmatching: false + prepare_links_p_nom: false + +electricity: + voltages: [220., 300., 380.] + co2limit: 100.e+6 + + extendable_carriers: + Generator: [OCGT] + StorageUnit: [battery, H2] + + max_hours: + battery: 6 + H2: 168 + + conventional_carriers: [coal, CCGT] # [nuclear, oil, OCGT, CCGT, coal, lignite, geothermal, biomass] + +atlite: + nprocesses: 4 + cutouts: + europe-2013-era5: + module: era5 + xs: [4., 15.] + ys: [56., 46.] + months: [3, 3] + years: [2013, 2013] + +renewable: + onwind: + cutout: europe-2013-era5 + resource: + method: wind + turbine: Vestas_V112_3MW + capacity_per_sqkm: 3 # ScholzPhd Tab 4.3.1: 10MW/km^2 + # correction_factor: 0.93 + corine: + # Scholz, Y. (2012). Renewable energy based electricity supply at low costs: + # development of the REMix model and application for Europe. ( p.42 / p.28) + grid_codes: [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, + 24, 25, 26, 27, 28, 29, 31, 32] + distance: 1000 + distance_grid_codes: [1, 2, 3, 4, 5, 6] + natura: true + potential: simple # or conservative + clip_p_max_pu: 1.e-2 + offwind-ac: + cutout: europe-2013-era5 + resource: + method: wind + turbine: NREL_ReferenceTurbine_5MW_offshore + capacity_per_sqkm: 3 + # correction_factor: 0.93 + corine: [44, 255] + natura: true + max_shore_distance: 30000 + potential: simple # or conservative + clip_p_max_pu: 1.e-2 + offwind-dc: + cutout: europe-2013-era5 + resource: + method: wind + turbine: NREL_ReferenceTurbine_5MW_offshore + # ScholzPhd Tab 4.3.1: 10MW/km^2 + capacity_per_sqkm: 3 + # correction_factor: 0.93 + corine: [44, 255] + natura: true + min_shore_distance: 30000 + potential: simple # or conservative + clip_p_max_pu: 1.e-2 + solar: + cutout: europe-2013-era5 + resource: + method: pv + panel: CSi + orientation: + slope: 35. + azimuth: 180. + capacity_per_sqkm: 1.7 # ScholzPhd Tab 4.3.1: 170 MW/km^2 + # Determined by comparing uncorrected area-weighted full-load hours to those + # published in Supplementary Data to + # Pietzcker, Robert Carl, et al. "Using the sun to decarbonize the power + # sector: The economic potential of photovoltaics and concentrating solar + # power." Applied Energy 135 (2014): 704-720. + correction_factor: 0.854337 + corine: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, + 14, 15, 16, 17, 18, 19, 20, 26, 31, 32] + natura: true + potential: simple # or conservative + clip_p_max_pu: 1.e-2 + +lines: + types: + 220.: "Al/St 240/40 2-bundle 220.0" + 300.: "Al/St 240/40 3-bundle 300.0" + 380.: "Al/St 240/40 4-bundle 380.0" + s_max_pu: 0.7 + length_factor: 1.25 + under_construction: 'zero' # 'zero': set capacity to zero, 'remove': remove, 'keep': with full capacity + +links: + p_max_pu: 1.0 + include_tyndp: true + under_construction: 'zero' # 'zero': set capacity to zero, 'remove': remove, 'keep': with full capacity + +transformers: + x: 0.1 + s_nom: 2000. + type: '' + +load: + scaling_factor: 1.0 + +costs: + year: 2030 + discountrate: 0.07 # From a Lion Hirth paper, also reflects average of Noothout et al 2016 + USD2013_to_EUR2013: 0.7532 # [EUR/USD] ECB: https://www.ecb.europa.eu/stats/exchange/eurofxref/html/eurofxref-graph-usd.en.html + marginal_cost: + solar: 0.01 + onwind: 0.015 + offwind: 0.015 + H2: 0. + battery: 0. + emission_prices: # only used with the option Ep + co2: 0. + +solving: + options: + formulation: kirchhoff + load_shedding: false + noisy_costs: true + min_iterations: 1 + max_iterations: 1 + clip_p_max_pu: 0.01 + #nhours: 10 + solver: + name: ipopt + # solver: + # name: gurobi + # threads: 4 + # method: 2 # barrier + # crossover: 0 + # BarConvTol: 1.e-5 + # FeasibilityTol: 1.e-6 + # AggFill: 0 + # PreDual: 0 + # GURO_PAR_BARDENSETHRESH: 200 + # solver: + # name: cplex + # threads: 4 + # lpmethod: 4 # barrier + # solutiontype: 2 # non basic solution, ie no crossover + # barrier_convergetol: 1.e-5 + # feasopt_tolerance: 1.e-6 + +plotting: + map: + figsize: [7, 7] + boundaries: [-10.2, 29, 35, 72] + p_nom: + bus_size_factor: 5.e+4 + linewidth_factor: 3.e+3 + + costs_max: 800 + costs_threshold: 1 + + energy_max: 15000. + energy_min: -10000. + energy_threshold: 50. + + vre_techs: ["onwind", "offwind-ac", "offwind-dc", "solar", "ror"] + conv_techs: ["OCGT", "CCGT", "Nuclear", "Coal"] + storage_techs: ["hydro+PHS", "battery", "H2"] + load_carriers: ["AC load"] + AC_carriers: ["AC line", "AC transformer"] + link_carriers: ["DC line", "Converter AC-DC"] + tech_colors: + "onwind" : "#235ebc" + "onshore wind" : "#235ebc" + 'offwind' : "#6895dd" + 'offwind-ac' : "#6895dd" + 'offshore wind' : "#6895dd" + 'offshore wind ac' : "#6895dd" + 'offwind-dc' : "#74c6f2" + 'offshore wind dc' : "#74c6f2" + "hydro" : "#08ad97" + "hydro+PHS" : "#08ad97" + "PHS" : "#08ad97" + "hydro reservoir" : "#08ad97" + 'hydroelectricity' : '#08ad97' + "ror" : "#4adbc8" + "run of river" : "#4adbc8" + 'solar' : "#f9d002" + 'solar PV' : "#f9d002" + 'solar thermal' : '#ffef60' + 'biomass' : '#0c6013' + 'solid biomass' : '#06540d' + 'biogas' : '#23932d' + 'waste' : '#68896b' + 'geothermal' : '#ba91b1' + "OCGT" : "#d35050" + "OCGT marginal" : "#d35050" + "OCGT-heat" : "#d35050" + "gas boiler" : "#d35050" + "gas boilers" : "#d35050" + "gas boiler marginal" : "#d35050" + "gas-to-power/heat" : "#d35050" + "gas" : "#d35050" + "natural gas" : "#d35050" + "CCGT" : "#b20101" + "CCGT marginal" : "#b20101" + "Nuclear" : "#ff9000" + "Nuclear marginal" : "#ff9000" + "nuclear" : "#ff9000" + "coal" : "#707070" + "Coal" : "#707070" + "Coal marginal" : "#707070" + "lignite" : "#9e5a01" + "Lignite" : "#9e5a01" + "Lignite marginal" : "#9e5a01" + "Oil" : "#262626" + "oil" : "#262626" + "H2" : "#ea048a" + "hydrogen storage" : "#ea048a" + "Sabatier" : "#a31597" + "methanation" : "#a31597" + "helmeth" : "#a31597" + "DAC" : "#d284ff" + "co2 stored" : "#e5e5e5" + "CO2 sequestration" : "#e5e5e5" + "battery" : "#b8ea04" + "battery storage" : "#b8ea04" + "Li ion" : "#b8ea04" + "BEV charger" : "#e2ff7c" + "V2G" : "#7a9618" + "transport fuel cell" : "#e884be" + "retrofitting" : "#e0d6a8" + "building retrofitting" : "#e0d6a8" + "heat pumps" : "#ff9768" + "heat pump" : "#ff9768" + "air heat pump" : "#ffbea0" + "ground heat pump" : "#ff7a3d" + "power-to-heat" : "#a59e7c" + "power-to-gas" : "#db8585" + "power-to-liquid" : "#a9acd1" + "Fischer-Tropsch" : "#a9acd1" + "resistive heater" : "#aa4925" + "water tanks" : "#401f75" + "hot water storage" : "#401f75" + "hot water charging" : "#351c5e" + "hot water discharging" : "#683ab2" + "CHP" : "#d80a56" + "CHP heat" : "#d80a56" + "CHP electric" : "#d80a56" + "district heating" : "#93864b" + "Ambient" : "#262626" + "Electric load" : "#f9d002" + "electricity" : "#f9d002" + "Heat load" : "#d35050" + "heat" : "#d35050" + "Transport load" : "#235ebc" + "transport" : "#235ebc" + "lines" : "#70af1d" + "transmission lines" : "#70af1d" + "AC-AC" : "#70af1d" + "AC line" : "#70af1d" + "links" : "#8a1caf" + "HVDC links" : "#8a1caf" + "DC-DC" : "#8a1caf" + "DC link" : "#8a1caf" + nice_names: + OCGT: "Open-Cycle Gas" + CCGT: "Combined-Cycle Gas" + offwind-ac: "Offshore Wind (AC)" + offwind-dc: "Offshore Wind (DC)" + onwind: "Onshore Wind" + battery: "Battery Storage" + H2: "Hydrogen Storage" + lines: "Transmission lines" + ror: "Run of river" + nice_names_n: + OCGT: "Open-Cycle\nGas" + CCGT: "Combined-Cycle\nGas" + offwind-ac: "Offshore\nWind (AC)" + offwind-dc: "Offshore\nWind (DC)" + onwind: "Onshore\nWind" + battery: "Battery\nStorage" + H2: "Hydrogen\nStorage" + lines: "Transmission\nlines" + ror: "Run of\nriver" + \ No newline at end of file diff --git a/doc/Makefile b/doc/Makefile new file mode 100644 index 00000000..be901185 --- /dev/null +++ b/doc/Makefile @@ -0,0 +1,192 @@ +# Makefile for Sphinx documentation +# + +# You can set these variables from the command line. +SPHINXOPTS = +SPHINXBUILD = sphinx-build +PAPER = +BUILDDIR = _build + +# User-friendly check for sphinx-build +ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) +$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) +endif + +# Internal variables. +PAPEROPT_a4 = -D latex_paper_size=a4 +PAPEROPT_letter = -D latex_paper_size=letter +ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . +# the i18n builder cannot share the environment and doctrees with the others +I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . + +.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest coverage gettext + +help: + @echo "Please use \`make ' where is one of" + @echo " html to make standalone HTML files" + @echo " dirhtml to make HTML files named index.html in directories" + @echo " singlehtml to make a single large HTML file" + @echo " pickle to make pickle files" + @echo " json to make JSON files" + @echo " htmlhelp to make HTML files and a HTML help project" + @echo " qthelp to make HTML files and a qthelp project" + @echo " applehelp to make an Apple Help Book" + @echo " devhelp to make HTML files and a Devhelp project" + @echo " epub to make an epub" + @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" + @echo " latexpdf to make LaTeX files and run them through pdflatex" + @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" + @echo " text to make text files" + @echo " man to make manual pages" + @echo " texinfo to make Texinfo files" + @echo " info to make Texinfo files and run them through makeinfo" + @echo " gettext to make PO message catalogs" + @echo " changes to make an overview of all changed/added/deprecated items" + @echo " xml to make Docutils-native XML files" + @echo " pseudoxml to make pseudoxml-XML files for display purposes" + @echo " linkcheck to check all external links for integrity" + @echo " doctest to run all doctests embedded in the documentation (if enabled)" + @echo " coverage to run coverage check of the documentation (if enabled)" + +clean: + rm -rf $(BUILDDIR)/* + +html: + $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html + @echo + @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." + +dirhtml: + $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml + @echo + @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." + +singlehtml: + $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml + @echo + @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." + +pickle: + $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle + @echo + @echo "Build finished; now you can process the pickle files." + +json: + $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json + @echo + @echo "Build finished; now you can process the JSON files." + +htmlhelp: + $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp + @echo + @echo "Build finished; now you can run HTML Help Workshop with the" \ + ".hhp project file in $(BUILDDIR)/htmlhelp." + +qthelp: + $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp + @echo + @echo "Build finished; now you can run "qcollectiongenerator" with the" \ + ".qhcp project file in $(BUILDDIR)/qthelp, like this:" + @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/PyPSA.qhcp" + @echo "To view the help file:" + @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/PyPSA.qhc" + +applehelp: + $(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp + @echo + @echo "Build finished. The help book is in $(BUILDDIR)/applehelp." + @echo "N.B. You won't be able to view it unless you put it in" \ + "~/Library/Documentation/Help or install it in your application" \ + "bundle." + +devhelp: + $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp + @echo + @echo "Build finished." + @echo "To view the help file:" + @echo "# mkdir -p $$HOME/.local/share/devhelp/PyPSA" + @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/PyPSA" + @echo "# devhelp" + +epub: + $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub + @echo + @echo "Build finished. The epub file is in $(BUILDDIR)/epub." + +latex: + $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex + @echo + @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." + @echo "Run \`make' in that directory to run these through (pdf)latex" \ + "(use \`make latexpdf' here to do that automatically)." + +latexpdf: + $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex + @echo "Running LaTeX files through pdflatex..." + $(MAKE) -C $(BUILDDIR)/latex all-pdf + @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." + +latexpdfja: + $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex + @echo "Running LaTeX files through platex and dvipdfmx..." + $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja + @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." + +text: + $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text + @echo + @echo "Build finished. The text files are in $(BUILDDIR)/text." + +man: + $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man + @echo + @echo "Build finished. The manual pages are in $(BUILDDIR)/man." + +texinfo: + $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo + @echo + @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." + @echo "Run \`make' in that directory to run these through makeinfo" \ + "(use \`make info' here to do that automatically)." + +info: + $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo + @echo "Running Texinfo files through makeinfo..." + make -C $(BUILDDIR)/texinfo info + @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." + +gettext: + $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale + @echo + @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." + +changes: + $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes + @echo + @echo "The overview file is in $(BUILDDIR)/changes." + +linkcheck: + $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck + @echo + @echo "Link check complete; look for any errors in the above output " \ + "or in $(BUILDDIR)/linkcheck/output.txt." + +doctest: + $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest + @echo "Testing of doctests in the sources finished, look at the " \ + "results in $(BUILDDIR)/doctest/output.txt." + +coverage: + $(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage + @echo "Testing of coverage in the sources finished, look at the " \ + "results in $(BUILDDIR)/coverage/python.txt." + +xml: + $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml + @echo + @echo "Build finished. The XML files are in $(BUILDDIR)/xml." + +pseudoxml: + $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml + @echo + @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." diff --git a/doc/_static/theme_overrides.css b/doc/_static/theme_overrides.css new file mode 100644 index 00000000..c413554e --- /dev/null +++ b/doc/_static/theme_overrides.css @@ -0,0 +1,19 @@ +/* override table width restrictions */ +@media screen and (min-width: 767px) { + + .wy-table-responsive table td { + /* !important prevents the common CSS stylesheets from overriding + this as on RTD they are loaded after this stylesheet */ + white-space: normal !important; + /* background: #eeeeee !important; */ + } + + .wy-table-responsive { + max-width: 100%; + overflow: visible !important; + } + + .wy-nav-content { + max-width: 910px !important; + } + } \ No newline at end of file diff --git a/doc/conf.py b/doc/conf.py new file mode 100644 index 00000000..867b9f53 --- /dev/null +++ b/doc/conf.py @@ -0,0 +1,321 @@ +# -*- coding: utf-8 -*- +# +# PyPSA documentation build configuration file, created by +# sphinx-quickstart on Tue Jan 5 10:04:42 2016. +# +# This file is execfile()d with the current directory set to its +# containing dir. +# +# Note that not all possible configuration values are present in this +# autogenerated file. +# +# All configuration values have a default; values that are commented out +# serve to show the default. + +import sys +import os +import shlex + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +sys.path.insert(0, os.path.abspath('../scripts')) + +# -- General configuration ------------------------------------------------ + +# If your documentation needs a minimal Sphinx version, state it here. +#needs_sphinx = '1.0' + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [ + #'sphinx.ext.autodoc', + #'sphinx.ext.autosummary', + 'sphinx.ext.intersphinx', + 'sphinx.ext.todo', + 'sphinx.ext.mathjax', + 'sphinx.ext.napoleon', + 'sphinx.ext.graphviz', + #'sphinx.ext.pngmath', + #'sphinxcontrib.tikz', + #'rinoh.frontend.sphinx', + 'sphinx.ext.imgconverter', # for SVG conversion +] + +autodoc_default_flags = ['members'] +autosummary_generate = True + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# The suffix(es) of source filenames. +# You can specify multiple suffix as a list of string: +# source_suffix = ['.rst', '.md'] +source_suffix = '.rst' + +# The encoding of source files. +#source_encoding = 'utf-8-sig' + +# The master toctree document. +master_doc = 'index' + +# General information about the project. +project = u'PyPSA-Eur' +copyright = u'2017-2019 Jonas Hoersch (KIT, FIAS), Fabian Hofmann (FIAS), David Schlachtberger (FIAS), Tom Brown (KIT, FIAS); 2019 Fabian Neumann (KIT)' +author = u'Jonas Hoersch (KIT, FIAS), Fabian Hofmann (FIAS), David Schlachtberger (FIAS), Tom Brown (KIT, FIAS), Fabian Neumann (KIT)' + +# The version info for the project you're documenting, acts as replacement for +# |version| and |release|, also used in various other places throughout the +# built documents. +# +# The short X.Y version. +version = u'0.1.0' +# The full version, including alpha/beta/rc tags. +release = u'0.1.0' + +# The language for content autogenerated by Sphinx. Refer to documentation +# for a list of supported languages. +# +# This is also used if you do content translation via gettext catalogs. +# Usually you set "language" from the command line for these cases. +language = None + +# There are two options for replacing |today|: either, you set today to some +# non-false value, then it is used: +#today = '' +# Else, today_fmt is used as the format for a strftime call. +#today_fmt = '%B %d, %Y' + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +exclude_patterns = ['_build'] + +# The reST default role (used for this markup: `text`) to use for all +# documents. +#default_role = None + +# If true, '()' will be appended to :func: etc. cross-reference text. +#add_function_parentheses = True + +# If true, the current module name will be prepended to all description +# unit titles (such as .. function::). +#add_module_names = True + +# If true, sectionauthor and moduleauthor directives will be shown in the +# output. They are ignored by default. +#show_authors = False + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = 'sphinx' + +# A list of ignored prefixes for module index sorting. +#modindex_common_prefix = [] + +# If true, keep warnings as "system message" paragraphs in the built documents. +#keep_warnings = False + +# If true, `todo` and `todoList` produce output, else they produce nothing. +todo_include_todos = True + + +# -- Options for HTML output ---------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +html_theme = 'sphinx_rtd_theme' + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. +html_theme_options = { + 'display_version': True, + 'sticky_navigation': True, +} + + +# Add any paths that contain custom themes here, relative to this directory. +#html_theme_path = [] + +# The name for this set of Sphinx documents. If None, it defaults to +# " v documentation". +#html_title = None + +# A shorter title for the navigation bar. Default is the same as html_title. +#html_short_title = None + +# The name of an image file (relative to this directory) to place at the top +# of the sidebar. +#html_logo = None + +# The name of an image file (within the static path) to use as favicon of the +# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 +# pixels large. +#html_favicon = None + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] + +html_context = { + 'css_files': [ + '_static/theme_overrides.css', # override wide tables in RTD theme + ], +} + +# Add any extra paths that contain custom files (such as robots.txt or +# .htaccess) here, relative to this directory. These files are copied +# directly to the root of the documentation. +#html_extra_path = [] + +# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, +# using the given strftime format. +#html_last_updated_fmt = '%b %d, %Y' + +# If true, SmartyPants will be used to convert quotes and dashes to +# typographically correct entities. +#html_use_smartypants = True + +# Custom sidebar templates, maps document names to template names. +#html_sidebars = {} + +# Additional templates that should be rendered to pages, maps page names to +# template names. +#html_additional_pages = {} + +# If false, no module index is generated. +#html_domain_indices = True + +# If false, no index is generated. +#html_use_index = True + +# If true, the index is split into individual pages for each letter. +#html_split_index = False + +# If true, links to the reST sources are added to the pages. +#html_show_sourcelink = True + +# If true, "Created using Sphinx" is shown in the HTML footer. Default is True. +#html_show_sphinx = True + +# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. +#html_show_copyright = True + +# If true, an OpenSearch description file will be output, and all pages will +# contain a tag referring to it. The value of this option must be the +# base URL from which the finished HTML is served. +#html_use_opensearch = '' + +# This is the file name suffix for HTML files (e.g. ".xhtml"). +#html_file_suffix = None + +# Language to be used for generating the HTML full-text search index. +# Sphinx supports the following languages: +# 'da', 'de', 'en', 'es', 'fi', 'fr', 'hu', 'it', 'ja' +# 'nl', 'no', 'pt', 'ro', 'ru', 'sv', 'tr' +#html_search_language = 'en' + +# A dictionary with options for the search language support, empty by default. +# Now only 'ja' uses this config value +#html_search_options = {'type': 'default'} + +# The name of a javascript file (relative to the configuration directory) that +# implements a search results scorer. If empty, the default will be used. +#html_search_scorer = 'scorer.js' + +# Output file base name for HTML help builder. +htmlhelp_basename = 'PyPSAEurdoc' + +# -- Options for LaTeX output --------------------------------------------- + +latex_elements = { +# The paper size ('letterpaper' or 'a4paper'). +#'papersize': 'letterpaper', + +# The font size ('10pt', '11pt' or '12pt'). +#'pointsize': '10pt', + +# Additional stuff for the LaTeX preamble. +#'preamble': '', + +# Latex figure (float) alignment +#'figure_align': 'htbp', +} + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, +# author, documentclass [howto, manual, or own class]). +latex_documents = [ + (master_doc, 'PyPSA-Eur.tex', u'PyPSA-Eur Documentation', + u'author', 'manual'), +] + + +#Added for rinoh http://www.mos6581.org/rinohtype/quickstart.html +rinoh_documents = [(master_doc, # top-level file (index.rst) + 'PyPSA-Eur', # output (target.pdf) + 'PyPSA-Eur Documentation', # document title + 'author')] # document author + + +# The name of an image file (relative to this directory) to place at the top of +# the title page. +#latex_logo = None + +# For "manual" documents, if this is true, then toplevel headings are parts, +# not chapters. +#latex_use_parts = False + +# If true, show page references after internal links. +#latex_show_pagerefs = False + +# If true, show URL addresses after external links. +#latex_show_urls = False + +# Documents to append as an appendix to all manuals. +#latex_appendices = [] + +# If false, no module index is generated. +#latex_domain_indices = True + + +# -- Options for manual page output --------------------------------------- + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). +man_pages = [ + (master_doc, 'pypsa-eur', u'PyPSA-Eur Documentation', + [author], 1) +] + +# If true, show URL addresses after external links. +#man_show_urls = False + + +# -- Options for Texinfo output ------------------------------------------- + +# Grouping the document tree into Texinfo files. List of tuples +# (source start file, target name, title, author, +# dir menu entry, description, category) +texinfo_documents = [ + (master_doc, 'PyPSA-Eur', u'PyPSA-Eur Documentation', + author, 'PyPSA-Eur', 'One line description of project.', + 'Miscellaneous'), +] + +# Documents to append as an appendix to all manuals. +#texinfo_appendices = [] + +# If false, no module index is generated. +#texinfo_domain_indices = True + +# How to display URL addresses: 'footnote', 'no', or 'inline'. +#texinfo_show_urls = 'footnote' + +# If true, do not generate a @detailmenu in the "Top" node's menu. +#texinfo_no_detailmenu = False + + +# Example configuration for intersphinx: refer to the Python standard library. +intersphinx_mapping = {'https://docs.python.org/': None} diff --git a/doc/configtables/atlite.csv b/doc/configtables/atlite.csv new file mode 100644 index 00000000..b60b15e5 --- /dev/null +++ b/doc/configtables/atlite.csv @@ -0,0 +1,8 @@ +,Unit,Values,Description +nprocesses,--,int,"Number of parallel processes in cutout preparation" +cutouts,,, +-- {name},--,"Convention is to name cutouts like ``--`` (e.g. ``europe-2013-era5``).","Directory to write cutout data to. The user may specify multiple cutouts under configuration ``atlite: cutouts:``. Reference is used in configuration ``renewable: {technology}: cutout:``" +-- -- module,--,"One of {'era5','sarah'}","Source of the reanalysis weather dataset (e.g. `ERA5 `_ or `SARAH-2 `_)" +-- -- xs,°,"Float interval within [-180, 180]","Range of longitudes to download weather data for." +-- -- ys,°,"Float interval within [-90, 90]","Range of latitudes to download weather data for." +-- -- years,--,"Integer interval within [1979,2018]","Range of years to download weather data for." \ No newline at end of file diff --git a/doc/configtables/costs.csv b/doc/configtables/costs.csv new file mode 100644 index 00000000..c35b83dc --- /dev/null +++ b/doc/configtables/costs.csv @@ -0,0 +1,8 @@ +,Unit,Values,Description +year,--,"YYYY; e.g. '2030'","Year for which to retrieve cost assumptions of ``data/costs.csv``." +discountrate,--,float,"Default discount rate if not specified for a technology in ``data/costs.csv``." +USD2013_to_EUR2013,--,float,"Exchange rate from USD :math:`_{2013}` to EUR :math:`_{2013}` from `ECB `_" +capital_cost,EUR/MW,"Keys should be in the 'technology' column of ``data/costs.csv``. Values can be any float.","For the given technologies, assumptions about their capital investment costs are set to the corresponding value. Optional; overwrites cost assumptions from ``data/costs.csv``." +marginal_cost,EUR/MWh,"Keys should be in the 'technology' column of ``data/costs.csv``. Values can be any float.","For the given technologies, assumptions about their marginal operating costs are set to the corresponding value. Optional; overwrites cost assumptions from ``data/costs.csv``." +emission_prices,,, +-- co2,EUR/t,float,"Exogenous price of carbon-dioxide added to the marginal costs of fossil-fuelled generators according to their carbon intensity." \ No newline at end of file diff --git a/doc/configtables/electricity.csv b/doc/configtables/electricity.csv new file mode 100644 index 00000000..03052835 --- /dev/null +++ b/doc/configtables/electricity.csv @@ -0,0 +1,10 @@ +,Unit,Values,Description +voltages,kV,"Any subset of {220., 300., 380.}","Voltage levels to consider when" +co2limit,:math:`t_{CO_2-eq}/a`,float,"Cap on total annual system carbon dioxide emissions" +extendable_carriers,,, +-- Generator,--,"Any subset of {'OCGT','CCGT'}","Places extendable conventional power plants (OCGT and/or CCGT) where gas power plants are located today without capacity limits." +-- StorageUnit,--,"Any subset of {'battery','H2'}","Places extendable storage units (battery and/or hydrogen) at every node/bus without capacity limits." +max_hours,,, +-- battery,h,float,"Maximum state of charge capacity of the battery in terms of hours at full output capacity ``p_nom``. Cf. `PyPSA documentation `_." +-- H2,h,float,"Maximum state of charge capacity of the hydrogen storage in terms of hours at full output capacity ``p_nom``. Cf. `PyPSA documentation `_." +conventional_carriers,--,"Any subset of {nuclear, oil, OCGT, CCGT, coal, lignite, geothermal, biomass}","List of conventional power plants to include in the model from ``resources/powerplants.csv``." diff --git a/doc/configtables/hydro.csv b/doc/configtables/hydro.csv new file mode 100644 index 00000000..801b27ff --- /dev/null +++ b/doc/configtables/hydro.csv @@ -0,0 +1,6 @@ +,Unit,Values,Description +cutout,--,"Must be 'europe-2013-era5'","Specifies the directory where the relevant weather data ist stored." +carriers,--,"Any subset of {'ror', 'PHS', 'hydro'}","Specifies the types of hydro power plants to build per-unit availability time series for. 'ror' stands for run-of-river plants, 'PHS' represents pumped-hydro storage, and 'hydro' stands for hydroelectric dams." +PHS_max_hours,h,float,"Maximum state of charge capacity of the pumped-hydro storage (PHS) in terms of hours at full output capacity ``p_nom``. Cf. `PyPSA documentation `_." +hydro_max_hours,h,"Any of {float, 'energy_capacity_totals_by_country', 'estimate_by_large_installations'}","Maximum state of charge capacity of the pumped-hydro storage (PHS) in terms of hours at full output capacity ``p_nom`` or heuristically determined. Cf. `PyPSA documentation `_." +clip_min_inflow,MW,float,"To avoid too small values in the inflow time series, values below this threshold are set to zero." \ No newline at end of file diff --git a/doc/configtables/licenses.csv b/doc/configtables/licenses.csv new file mode 100644 index 00000000..3e25f5df --- /dev/null +++ b/doc/configtables/licenses.csv @@ -0,0 +1,14 @@ +"Files","BY","NC","SA","Mark Changes",Detail +"corine/*","x",,,"x",https://land.copernicus.eu/pan-european/corine-land-cover/clc-2012?tab=metadata +"eez/*","x","x","x",,http://www.marineregions.org/disclaimer.php +"natura/*","x",,,,https://www.eea.europa.eu/data-and-maps/data/natura-10#tab-metadata +"naturalearth/*",,,,,http://www.naturalearthdata.com/about/terms-of-use/ +"NUTS_2013 _60M_SH/*","x","x",,"x",https://ec.europa.eu/eurostat/web/gisco/geodata/reference-data/administrative-units-statistical-units +"cantons.csv","x",,"x",,https://en.wikipedia.org/wiki/Data_codes_for_Switzerland +"EIA_hydro_generation _2000_2014.csv","x",,,,https://www.eia.gov/about/copyrights_reuse.php +"GEBCO_2014_2D.nc","x",,,,https://www.gebco.net/data_and_products/gridded_bathymetry_data/documents/gebco_2014_historic.pdf +"hydro_capacities.csv","x",,,, +"je-e-21.03.02.xls","x","x",,,https://www.bfs.admin.ch/bfs/en/home/fso/swiss-federal-statistical-office/terms-of-use.html +"nama_10r_3 gdp.tsv.gz","x",,,"x",https://ec.europa.eu/eurostat/about/policies/copyright +"nama_10r_3 popgdp.tsv.gz","x",,,"x",https://ec.europa.eu/eurostat/about/policies/copyright +"time_series_60min _singleindex_filtered.csv","x",,,,https://data.open-power-system-data.org/time_series/2019-06-05/README.md diff --git a/doc/configtables/lines.csv b/doc/configtables/lines.csv new file mode 100644 index 00000000..e5067867 --- /dev/null +++ b/doc/configtables/lines.csv @@ -0,0 +1,5 @@ +,Unit,Values,Description +types,--,"Values should specify a `line type in PyPSA `_. Keys should specify the corresponding voltage level (e.g. 220., 300. and 380. kV)","Specifies line types to assume for the different voltage levels of the ENTSO-E grid extraction. Should normally handle voltage levels 220, 300, and 380 kV" +s_max_pu,--,"Value in [0.,1.]","Correction factor for line capacities (``s_nom``) to approximate :math:`N-1` security and reserve capacity for reactive power flows" +length_factor,--,float,"Correction factor to account for the fact that buses are *not* connected by lines through air-line distance." +under_construction,--,"One of {'zero': set capacity to zero, 'remove': remove completely, 'keep': keep with full capacity}","Specifies how to handle lines which are currently under construction." \ No newline at end of file diff --git a/doc/configtables/links.csv b/doc/configtables/links.csv new file mode 100644 index 00000000..d77c9ddd --- /dev/null +++ b/doc/configtables/links.csv @@ -0,0 +1,4 @@ +,Unit,Values,Description +p_max_pu,--,"Value in [0.,1.]","Correction factor for link capacities ``p_nom``." +include_tyndp,bool,"{'true', 'false'}","Specifies whether to add HVDC link projects from the `TYNDP 2018 `_ which are at least in permitting." +under_construction,--,"One of {'zero': set capacity to zero, 'remove': remove completely, 'keep': keep with full capacity}","Specifies how to handle lines which are currently under construction." \ No newline at end of file diff --git a/doc/configtables/load.csv b/doc/configtables/load.csv new file mode 100644 index 00000000..035b27a1 --- /dev/null +++ b/doc/configtables/load.csv @@ -0,0 +1,2 @@ +,Unit,Values,Description +scaling_factor,--,float,"Global correction factor for the load time series." \ No newline at end of file diff --git a/doc/configtables/offwind-ac.csv b/doc/configtables/offwind-ac.csv new file mode 100644 index 00000000..e34e8831 --- /dev/null +++ b/doc/configtables/offwind-ac.csv @@ -0,0 +1,12 @@ +,Unit,Values,Description +cutout,--,"Should be a folder listed in the configuration ``atlite: cutouts:`` (e.g. 'europe-2013-era5') or reference an existing folder in the directory ``cutouts``. Source module must be ERA5.","Specifies the directory where the relevant weather data ist stored." +resource,,, +-- method,--,"Must be 'wind'","A superordinate technology type." +-- turbine,--,"One of turbine types included in `atlite `_","Specifies the turbine type and its characteristic power curve." +capacity_per_sqkm,:math:`MW/km^2`,float,"Allowable density of wind turbine placement." +corine,--,"Any *realistic* subset of the `CORINE Land Cover code list `_","Specifies areas according to CORINE Land Cover codes which are generally eligible for AC-connected offshore wind turbine placement." +natura,bool,"{true, false}","Switch to exclude `Natura 2000 `_ natural protection areas. Area is excluded if ``true``." +max_depth,m,float,"Maximum sea water depth at which wind turbines can be build. Maritime areas with deeper waters are excluded in the process of calculating the AC-connected offshore wind potential." +min_shore_distance,m,float,"Minimum distance to the shore below which wind turbines cannot be build. Such areas close to the shore are excluded in the process of calculating the AC-connected offshore wind potential." +potential,--,"One of {'simple', 'conservative'}","Method to compute the maximal installable potential for a node; confer :ref:`renewableprofiles`" +clip_p_max_pu,p.u.,float,"To avoid too small values in the renewables` per-unit availability time series values below this threshold are set to zero." \ No newline at end of file diff --git a/doc/configtables/offwind-dc.csv b/doc/configtables/offwind-dc.csv new file mode 100644 index 00000000..e34e8831 --- /dev/null +++ b/doc/configtables/offwind-dc.csv @@ -0,0 +1,12 @@ +,Unit,Values,Description +cutout,--,"Should be a folder listed in the configuration ``atlite: cutouts:`` (e.g. 'europe-2013-era5') or reference an existing folder in the directory ``cutouts``. Source module must be ERA5.","Specifies the directory where the relevant weather data ist stored." +resource,,, +-- method,--,"Must be 'wind'","A superordinate technology type." +-- turbine,--,"One of turbine types included in `atlite `_","Specifies the turbine type and its characteristic power curve." +capacity_per_sqkm,:math:`MW/km^2`,float,"Allowable density of wind turbine placement." +corine,--,"Any *realistic* subset of the `CORINE Land Cover code list `_","Specifies areas according to CORINE Land Cover codes which are generally eligible for AC-connected offshore wind turbine placement." +natura,bool,"{true, false}","Switch to exclude `Natura 2000 `_ natural protection areas. Area is excluded if ``true``." +max_depth,m,float,"Maximum sea water depth at which wind turbines can be build. Maritime areas with deeper waters are excluded in the process of calculating the AC-connected offshore wind potential." +min_shore_distance,m,float,"Minimum distance to the shore below which wind turbines cannot be build. Such areas close to the shore are excluded in the process of calculating the AC-connected offshore wind potential." +potential,--,"One of {'simple', 'conservative'}","Method to compute the maximal installable potential for a node; confer :ref:`renewableprofiles`" +clip_p_max_pu,p.u.,float,"To avoid too small values in the renewables` per-unit availability time series values below this threshold are set to zero." \ No newline at end of file diff --git a/doc/configtables/onwind.csv b/doc/configtables/onwind.csv new file mode 100644 index 00000000..5b7b6797 --- /dev/null +++ b/doc/configtables/onwind.csv @@ -0,0 +1,13 @@ +,Unit,Values,Description +cutout,--,"Should be a folder listed in the configuration ``atlite: cutouts:`` (e.g. 'europe-2013-era5') or reference an existing folder in the directory ``cutouts``. Source module must be ERA5.","Specifies the directory where the relevant weather data ist stored." +resource,,, +-- method,--,"Must be 'wind'","A superordinate technology type." +-- turbine,--,"One of turbine types included in `atlite `_","Specifies the turbine type and its characteristic power curve." +capacity_per_sqkm,:math:`MW/km^2`,float,"Allowable density of wind turbine placement." +corine,,, +-- grid_codes,--,"Any subset of the `CORINE Land Cover code list `_","Specifies areas according to CORINE Land Cover codes which are generally eligible for wind turbine placement." +-- distance,m,float,"Distance to keep from areas specified in ``distance_grid_codes``" +-- distance_grid_codes,--,"Any subset of the `CORINE Land Cover code list `_","Specifies areas according to CORINE Land Cover codes to which wind turbines must maintain a distance specified in the setting ``distance``." +natura,bool,"{true, false}","Switch to exclude `Natura 2000 `_ natural protection areas. Area is excluded if ``true``." +potential,--,"One of {'simple', 'conservative'}","Method to compute the maximal installable potential for a node; confer :ref:`renewableprofiles`" +clip_p_max_pu,p.u.,float,"To avoid too small values in the renewables` per-unit availability time series values below this threshold are set to zero." \ No newline at end of file diff --git a/doc/configtables/opts.csv b/doc/configtables/opts.csv new file mode 100644 index 00000000..9a0fcc93 --- /dev/null +++ b/doc/configtables/opts.csv @@ -0,0 +1,6 @@ +Trigger, Description, Definition, Status +``Co2L``, Add an overall carbon-dioxide emissions limit configured in ``electricity: co2limit``, ``prepare_network``: `add_co2limit() `_ and its `caller `_, In active use +``nH``; i.e. ``2H``-``6H``, Resample the time-resolution by averaging over every ``n`` snapshots, ``prepare_network``: `average_every_nhours() `_ and its `caller `_), In active use +``Ep``, Add cost for a carbon-dioxide price configured in ``costs: emission_prices: co2`` to ``marginal_cost`` of generators, ``prepare_network``: `add_emission_prices() `_ and its `caller `_, Broken +``BAU``, Add a per-``carrier`` minimal overall capacity; i.e. at least ``40GW`` of ``OCGT`` in Europe; configured in ``electricity: BAU_mincapacities``, ``solve_network``: `add_opts_constraints() `_, Untested +``SAFE``, Add a capacity reserve margin of a certain fraction above the peak demand to which renewable generators and storage do *not* contribute. Ignores network., ``solve_network`` `add_opts_constraints() `_, Untested diff --git a/doc/configtables/plotting.csv b/doc/configtables/plotting.csv new file mode 100644 index 00000000..0f21c9a8 --- /dev/null +++ b/doc/configtables/plotting.csv @@ -0,0 +1,15 @@ +,Unit,Values,Description +map,,, +-- figsize,--,"[width, height]; e.g. [7,7]","Figure size in inches." +-- boundaries,°,"[x1,x2,y1,y2]","Boundaries of the map plots in degrees latitude (y) and longitude (x)" +-- p_nom,,, +-- -- bus_size_factor,--,float,"Factor by which values determining bus sizes are scaled to fit well in the plot." +-- -- linewidth_factor,--,float,"Factor by which values determining bus sizes are scaled to fit well in the plot." +costs_max,bn Euro,float,"Upper y-axis limit in cost bar plots." +costs_threshold,bn Euro,float,"Threshold below which technologies will not be shown in cost bar plots." +energy_max,TWh,float,"Upper y-axis limit in energy bar plots." +energy_min,TWh,float,"Lower y-axis limit in energy bar plots." +energy_threshold,TWh,float,"Threshold below which technologies will not be shown in energy bar plots." +tech_colors,--,"carrier -> HEX colour code","Mapping from network ``carrier`` to a colour (`HEX colour code `_)." +nice_names,--,"str -> str","Mapping from network ``carrier`` to a more readable name." +nice_names_n,--,"str -> str","Same as nice_names, but with linebreaks." \ No newline at end of file diff --git a/doc/configtables/scenario.csv b/doc/configtables/scenario.csv new file mode 100644 index 00000000..52dafa56 --- /dev/null +++ b/doc/configtables/scenario.csv @@ -0,0 +1,6 @@ +,Unit,Values,Description +sectors,--,"Must be 'elec'","Placeholder for integration of other energy sectors." +simpl,--,cf. :ref:`simpl`,"List of ``{simpl}`` wildcards to run." +ll,--,cf. :ref:`ll`,"List of ``{ll}`` wildcards to run." +clusters,--,cf. :ref:`clusters`,"List of ``{clusters}`` wildcards to run." +opts,--,cf. :ref:`opts`,"List of ``{opts}`` wildcards to run." \ No newline at end of file diff --git a/doc/configtables/snapshots.csv b/doc/configtables/snapshots.csv new file mode 100644 index 00000000..14fd8001 --- /dev/null +++ b/doc/configtables/snapshots.csv @@ -0,0 +1,4 @@ +,Unit,Values,Description +start,--,"str or datetime-like; e.g. YYYY-MM-DD","Left bound of date range" +end,--,"str or datetime-like; e.g. YYYY-MM-DD","Right bound of date range" +closed,--,"One of {None, ‘left’, ‘right’}","Make the time interval closed to the ``left``, ``right``, or both sides ``None``." \ No newline at end of file diff --git a/doc/configtables/solar.csv b/doc/configtables/solar.csv new file mode 100644 index 00000000..c7153db4 --- /dev/null +++ b/doc/configtables/solar.csv @@ -0,0 +1,14 @@ +,Unit,Values,Description +cutout,--,"Should be a folder listed in the configuration ``atlite: cutouts:`` (e.g. 'europe-2013-era5') or reference an existing folder in the directory ``cutouts``. Source module can be ERA5 or SARAH-2.","Specifies the directory where the relevant weather data ist stored that is specified at ``atlite/cutouts`` configuration. Both ``sarah`` and ``era5`` work." +resource,,, +-- method,--,"Must be 'pv'","A superordinate technology type." +-- panel,--,"One of {'Csi', 'CdTe', 'KANENA'} as defined in `atlite `_","Specifies the solar panel technology and its characteristic attributes." +-- orientation,,, +-- -- slope,°,"Realistically any angle in [0., 90.]","Specifies the tilt angle (or slope) of the solar panel. A slope of zero corresponds to the face of the panel aiming directly overhead. A positive tilt angle steers the panel towards the equator." +-- -- azimuth,°,"Any angle in [0., 360.]","Specifies the `azimuth `_ orientation of the solar panel. South corresponds to 180.°." +capacity_per_sqkm,:math:`MW/km^2`,float,"Allowable density of solar panel placement." +correction_factor,--,float,"A correction factor for the capacity factor (availability) time series." +corine,--,"Any subset of the `CORINE Land Cover code list `_","Specifies areas according to CORINE Land Cover codes which are generally eligible for solar panel placement." +natura,bool,"{true, false}","Switch to exclude `Natura 2000 `_ natural protection areas. Area is excluded if ``true``." +potential,--,"One of {'simple', 'conservative'}","Method to compute the maximal installable potential for a node; confer :ref:`renewableprofiles`" +clip_p_max_pu,p.u.,float,"To avoid too small values in the renewables` per-unit availability time series values below this threshold are set to zero." \ No newline at end of file diff --git a/doc/configtables/solving-options.csv b/doc/configtables/solving-options.csv new file mode 100644 index 00000000..72873c83 --- /dev/null +++ b/doc/configtables/solving-options.csv @@ -0,0 +1,8 @@ +,Unit,Values,Description +formulation,--,"Any of {'angles', 'kirchhoff', 'cycles', 'ptdf'}","Specifies which variant of linearized power flow formulations to use in the optimisation problem. Recommended is 'kirchhoff'. Explained in `this article `_." +load_shedding,bool,"{'true','false'}","Add generators with a prohibitively high marginal cost to simulate load shedding and avoid problem infeasibilities." +noisy_costs,bool,"{'true','false'}","Add random noise to marginal cost of generators by :math:`\mathcal{U}(0.009,0,011)` and capital cost of lines and links by :math:`\mathcal{U}(0.09,0,11)`." +min_iterations,--,int,"Minimum number of solving iterations in between which resistance and reactence (``x/r``) are updated for branches according to ``s_nom_opt`` of the previous run." +max_iterations,--,int,"Maximum number of solving iterations in between which resistance and reactence (``x/r``) are updated for branches according to ``s_nom_opt`` of the previous run." +nhours,--,int,"Specifies the :math:`n` first snapshots to take into account. Must be less than the total number of snapshots. Rather recommended only for debugging." +clip_p_max_pu,p.u.,float,"To avoid too small values in the renewables` per-unit availability time series values below this threshold are set to zero." \ No newline at end of file diff --git a/doc/configtables/solving-solver.csv b/doc/configtables/solving-solver.csv new file mode 100644 index 00000000..db16d867 --- /dev/null +++ b/doc/configtables/solving-solver.csv @@ -0,0 +1,3 @@ +,Unit,Values,Description +name,--,"One of {'gurobi', 'cplex', 'cbc', 'glpk', 'ipopt'}; potentially more possible","Solver to use for optimisation problems in the workflow; e.g. clustering and linear optimal power flow." +opts,--,"Parameter list for `Gurobi `_ and `CPLEX `_","Solver specific parameter settings." \ No newline at end of file diff --git a/doc/configtables/toplevel.csv b/doc/configtables/toplevel.csv new file mode 100644 index 00000000..dfc4a0bb --- /dev/null +++ b/doc/configtables/toplevel.csv @@ -0,0 +1,8 @@ +,Unit,Values,Description +version,--,0.1,"Version of PyPSA-Eur" +logging_level,--,"Any of {'INFO', 'WARNING', 'ERROR'}","Restrict console outputs to all infos, warning or errors only" +summary_dir,--,"e.g. 'results'","Directory into which results are written." +countries,--,"Subset of {'AL', 'AT', 'BA', 'BE', 'BG', 'CH', 'CZ', 'DE', 'DK', 'EE', 'ES', 'FI', 'FR', 'GB', 'GR', 'HR', 'HU', 'IE', 'IT', 'LT', 'LU', 'LV', 'ME', 'MK', 'NL', 'NO', 'PL', 'PT', 'RO', 'RS', 'SE', 'SI', 'SK'}","European countries defined by their `Two-letter country codes (ISO 3166-1) `_ which should be included in the energy system model." +enable,,, +-- powerplantmatching,bool,"{true, false}","Switch to retrieve currently existing conventional power plant capacities matched from multiple sources by using `powerplantmatching `_." +-- prepare_links_p_nom,bool,"{true, false}","Switch to retrieve current HVDC projects from `Wikipedia `_" \ No newline at end of file diff --git a/doc/configtables/transformers.csv b/doc/configtables/transformers.csv new file mode 100644 index 00000000..b58ae8f3 --- /dev/null +++ b/doc/configtables/transformers.csv @@ -0,0 +1,4 @@ +,Unit,Values,Description +x,p.u.,float,"Series reactance (per unit, using ``s_nom`` as base power of the transformer. Overwritten if ``type`` is specified." +s_nom,MVA,float,"Limit of apparent power which can pass through branch. Overwritten if ``type`` is specified." +type,--,"A `transformer type in PyPSA `_.","Specifies transformer types to assume for the transformers of the ENTSO-E grid extraction." \ No newline at end of file diff --git a/doc/configuration.rst b/doc/configuration.rst new file mode 100644 index 00000000..d0360d59 --- /dev/null +++ b/doc/configuration.rst @@ -0,0 +1,283 @@ +.. _config: + +########################################## +Configuration +########################################## + +PyPSA-Eur has several configuration options which are documented in this section and are collected in a ``config.yaml`` file located in the root directory. Users should copy the provided default configuration (``config.default.yaml``) and amend their own modifications and assumptions in the user-specific configuration file (``config.yaml``); confer installation instructions at :ref:`defaultconfig`. + +.. _toplevel_cf: + +Top-level configuration +======================= + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 1-5,13 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/toplevel.csv + +.. _scenario: + +``scenario`` +============ + +It is common conduct to analyse energy system optimisation models for **multiple scenarios** for a variety of reasons, +e.g. assessing their sensitivity towards changing the temporal and/or geographical resolution or investigating how +investment changes as more ambitious greenhouse-gas emission reduction targets are applied. + +The ``scenario`` section is an extraordinary section of the config file +that is strongly connected to the :ref:`wildcards` and is designed to +facilitate running multiple scenarios through a single command + +.. code:: bash + + snakemake solve_all_elec_networks + +For each wildcard, a **list of values** is provided. The rule ``solve_all_elec_networks`` will trigger the rules for creating ``results/networks/elec_s{simpl}_{clusters}_l{ll}_{opts}.nc`` for **all combinations** of the provided wildcard values as defined by Python's `itertools.product(...) `_ function that snakemake's `expand(...) function `_ uses. + +An exemplary dependency graph (starting from the simplification rules) then looks like this: + +.. image:: img/scenarios.png + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 6-11 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/scenario.csv + +.. _snapshots_cf: + +``snapshots`` +============= + +Specifies the temporal range to build an energy system model for as arguments to `pandas.date_range `_ + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 15-18 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/snapshots.csv + +.. _electricity_cf: + +``electricity`` +=============== + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 24-36 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/electricity.csv + +.. warning:: + Carriers in ``conventional_carriers`` must not also be in ``extendable_carriers``. + +.. _atlite_cf: + +``atlite`` +============= + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 38-51 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/atlite.csv + +.. _renewable_cf: + +``renewable`` +============= + +``onwind`` +---------- + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 53-70 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/onwind.csv + +``offwind-ac`` +-------------- + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 53,71-83 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/offwind-ac.csv + +``offwind-dc`` +--------------- + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 53,84-97 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/offwind-dc.csv + +``solar`` +--------------- + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 53,98-117 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/solar.csv + +``hydro`` +--------------- + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 53,118-124 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/hydro.csv + +.. _lines_cf: + +``lines`` +============= + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 126-133 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/lines.csv + +.. _links_cf: + +``links`` +============= + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 135-138 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/links.csv + +.. _transformers_cf: + +``transformers`` +================ + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 140-143 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/transformers.csv + +.. _load_cf: + +``load`` +============= + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 145-146 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/load.csv + +.. _costs_cf: + +``costs`` +============= + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 148-160 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/costs.csv + +.. note:: + To change cost assumptions in more detail (i.e. other than ``marginal_cost`` and ``capital_cost``), consider modifying cost assumptions directly in ``data/costs.csv`` as this is not yet supported through the config file. + + You can also build multiple different cost databases. Make a renamed copy of ``data/costs.csv`` (e.g. ``data/costs-optimistic.csv``) and set the variable ``COSTS=data/costs-optimistic.csv`` in the ``Snakefile``. + +.. _solving_cf: + +``solving`` +============= + +``options`` +----------- + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 162-170 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/solving-options.csv + +``solver`` +---------- + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 162,171-187 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/solving-solver.csv + +.. _plotting_cf: + +``plotting`` +============= + +.. literalinclude:: ../config.default.yaml + :language: yaml + :lines: 189-323 + +.. csv-table:: + :header-rows: 1 + :widths: 25,7,22,30 + :file: configtables/plotting.csv diff --git a/doc/contributing.rst b/doc/contributing.rst new file mode 100644 index 00000000..4ba0f147 --- /dev/null +++ b/doc/contributing.rst @@ -0,0 +1,10 @@ +####################### +Contributing +####################### + +We strongly welcome anyone interested in contributing to this project, +be it with new ideas, suggestions, by filing bug reports or contributing code. + +You are invited to submit pull requests and file issues to the `GitHub repository `_. + +If you are unfamiliar with pull requests, the GitHub help pages have a nice `guide `_. \ No newline at end of file diff --git a/doc/costs.rst b/doc/costs.rst new file mode 100644 index 00000000..e65ddf27 --- /dev/null +++ b/doc/costs.rst @@ -0,0 +1,45 @@ +################## +Cost Assumptions +################## + +The database of cost assumptions is stored in ``data/costs.csv``. + +It includes cost assumptions for all included technologies for specific +years from various sources, namely for + +- discount rate, +- lifetime, +- investment (CAPEX), +- fixed operation and maintenance (FOM), +- variable operation and maintenance (VOM), +- fuel costs, +- efficiency, and +- carbon-dioxide intensity. + +The given overnight capital costs are annualised to net present costs +with a discount rate of :math:`r` over the economic lifetime :math:`n` using the annuity factor + +.. math:: + + a = \frac{1-(1+r)^{-n}}{r}. + +Based on the parameters above the ``marginal_cost`` and ``capital_cost`` of the system components are calculated. + + +Modifying Cost Assumptions +========================== + +Some cost assumptions (e.g. marginal cost and capital cost) can be directly overwritten in the ``config.yaml`` (cf. Section :ref:`costs_cf` in :ref:`config`). + +To change cost assumptions in more detail, modify cost assumptions directly in ``data/costs.csv`` as this is not yet supported through the config file. + +You can also build multiple different cost databases. Make a renamed copy of ``data/costs.csv`` (e.g. ``data/costs-optimistic.csv``) and set the variable ``COSTS=data/costs-optimistic.csv`` in the ``Snakefile``. + + +Default Cost Assumptions +======================== + +.. csv-table:: + :header-rows: 1 + :widths: 10,3,5,4,6,8 + :file: ../data/costs.csv \ No newline at end of file diff --git a/doc/dconf/user b/doc/dconf/user new file mode 100644 index 00000000..09f370e3 Binary files /dev/null and b/doc/dconf/user differ diff --git a/doc/img/base.png b/doc/img/base.png new file mode 100644 index 00000000..4521142e Binary files /dev/null and b/doc/img/base.png differ diff --git a/doc/img/corine.png b/doc/img/corine.png new file mode 100644 index 00000000..07112e0e Binary files /dev/null and b/doc/img/corine.png differ diff --git a/doc/img/countries.png b/doc/img/countries.png new file mode 100644 index 00000000..e6acaae5 Binary files /dev/null and b/doc/img/countries.png differ diff --git a/doc/img/country_shapes.png b/doc/img/country_shapes.png new file mode 100644 index 00000000..2284e5e0 Binary files /dev/null and b/doc/img/country_shapes.png differ diff --git a/doc/img/distance_hist.png b/doc/img/distance_hist.png new file mode 100644 index 00000000..da202566 Binary files /dev/null and b/doc/img/distance_hist.png differ diff --git a/doc/img/eez.png b/doc/img/eez.png new file mode 100644 index 00000000..c9b54707 Binary files /dev/null and b/doc/img/eez.png differ diff --git a/doc/img/elec.png b/doc/img/elec.png new file mode 100644 index 00000000..52d4f772 Binary files /dev/null and b/doc/img/elec.png differ diff --git a/doc/img/elec_s.png b/doc/img/elec_s.png new file mode 100644 index 00000000..66afdb1d Binary files /dev/null and b/doc/img/elec_s.png differ diff --git a/doc/img/elec_s_X.png b/doc/img/elec_s_X.png new file mode 100644 index 00000000..e0f4f4a3 Binary files /dev/null and b/doc/img/elec_s_X.png differ diff --git a/doc/img/era5.png b/doc/img/era5.png new file mode 100644 index 00000000..a3d98121 Binary files /dev/null and b/doc/img/era5.png differ diff --git a/doc/img/europe_shape.png b/doc/img/europe_shape.png new file mode 100644 index 00000000..20b4cd5b Binary files /dev/null and b/doc/img/europe_shape.png differ diff --git a/doc/img/gebco_2019_grid_image.jpg b/doc/img/gebco_2019_grid_image.jpg new file mode 100644 index 00000000..3dba620a Binary files /dev/null and b/doc/img/gebco_2019_grid_image.jpg differ diff --git a/doc/img/hydrocapacities.png b/doc/img/hydrocapacities.png new file mode 100644 index 00000000..3a41b62b Binary files /dev/null and b/doc/img/hydrocapacities.png differ diff --git a/doc/img/hydrogeneration.png b/doc/img/hydrogeneration.png new file mode 100644 index 00000000..c86ffcfe Binary files /dev/null and b/doc/img/hydrogeneration.png differ diff --git a/doc/img/inflow-box.png b/doc/img/inflow-box.png new file mode 100644 index 00000000..84dcb7cf Binary files /dev/null and b/doc/img/inflow-box.png differ diff --git a/doc/img/inflow-ts.png b/doc/img/inflow-ts.png new file mode 100644 index 00000000..dbef1a15 Binary files /dev/null and b/doc/img/inflow-ts.png differ diff --git a/doc/img/load-box.png b/doc/img/load-box.png new file mode 100644 index 00000000..d95eb2bd Binary files /dev/null and b/doc/img/load-box.png differ diff --git a/doc/img/load-ts.png b/doc/img/load-ts.png new file mode 100644 index 00000000..e1c8a1e1 Binary files /dev/null and b/doc/img/load-ts.png differ diff --git a/doc/img/natura.png b/doc/img/natura.png new file mode 100644 index 00000000..372b92df Binary files /dev/null and b/doc/img/natura.png differ diff --git a/doc/img/nuts3.png b/doc/img/nuts3.png new file mode 100644 index 00000000..e594e10e Binary files /dev/null and b/doc/img/nuts3.png differ diff --git a/doc/img/nuts3_shapes.png b/doc/img/nuts3_shapes.png new file mode 100644 index 00000000..f6829374 Binary files /dev/null and b/doc/img/nuts3_shapes.png differ diff --git a/doc/img/offshore_shapes.png b/doc/img/offshore_shapes.png new file mode 100644 index 00000000..a8c1dfb9 Binary files /dev/null and b/doc/img/offshore_shapes.png differ diff --git a/doc/img/p_nom_max_hist.png b/doc/img/p_nom_max_hist.png new file mode 100644 index 00000000..ea6dfb4f Binary files /dev/null and b/doc/img/p_nom_max_hist.png differ diff --git a/doc/img/potential_heatmap.png b/doc/img/potential_heatmap.png new file mode 100644 index 00000000..98281b4d Binary files /dev/null and b/doc/img/potential_heatmap.png differ diff --git a/doc/img/powerplantmatching.png b/doc/img/powerplantmatching.png new file mode 100644 index 00000000..83061366 Binary files /dev/null and b/doc/img/powerplantmatching.png differ diff --git a/doc/img/profile_ts.png b/doc/img/profile_ts.png new file mode 100644 index 00000000..6cc3fa8d Binary files /dev/null and b/doc/img/profile_ts.png differ diff --git a/img/pypsa-eur-grid.png b/doc/img/pypsa-eur-grid.png similarity index 100% rename from img/pypsa-eur-grid.png rename to doc/img/pypsa-eur-grid.png diff --git a/doc/img/regions_offshore.png b/doc/img/regions_offshore.png new file mode 100644 index 00000000..866eee5f Binary files /dev/null and b/doc/img/regions_offshore.png differ diff --git a/doc/img/regions_offshore_elec_s.png b/doc/img/regions_offshore_elec_s.png new file mode 100644 index 00000000..ed762a6c Binary files /dev/null and b/doc/img/regions_offshore_elec_s.png differ diff --git a/doc/img/regions_offshore_elec_s_X.png b/doc/img/regions_offshore_elec_s_X.png new file mode 100644 index 00000000..b5e63b70 Binary files /dev/null and b/doc/img/regions_offshore_elec_s_X.png differ diff --git a/doc/img/regions_onshore.png b/doc/img/regions_onshore.png new file mode 100644 index 00000000..f7128ea4 Binary files /dev/null and b/doc/img/regions_onshore.png differ diff --git a/doc/img/regions_onshore_elec_s.png b/doc/img/regions_onshore_elec_s.png new file mode 100644 index 00000000..96fb15fa Binary files /dev/null and b/doc/img/regions_onshore_elec_s.png differ diff --git a/doc/img/regions_onshore_elec_s_X.png b/doc/img/regions_onshore_elec_s_X.png new file mode 100644 index 00000000..02d2762e Binary files /dev/null and b/doc/img/regions_onshore_elec_s_X.png differ diff --git a/doc/img/results.png b/doc/img/results.png new file mode 100644 index 00000000..7cd8c1e4 Binary files /dev/null and b/doc/img/results.png differ diff --git a/doc/img/sarah.png b/doc/img/sarah.png new file mode 100644 index 00000000..2b822e31 Binary files /dev/null and b/doc/img/sarah.png differ diff --git a/doc/img/scenarios.png b/doc/img/scenarios.png new file mode 100644 index 00000000..d18164fc Binary files /dev/null and b/doc/img/scenarios.png differ diff --git a/doc/img/tech-colors.png b/doc/img/tech-colors.png new file mode 100644 index 00000000..531ab142 Binary files /dev/null and b/doc/img/tech-colors.png differ diff --git a/doc/img/underwater_hist.png b/doc/img/underwater_hist.png new file mode 100644 index 00000000..c3db34b7 Binary files /dev/null and b/doc/img/underwater_hist.png differ diff --git a/doc/img/workflow.png b/doc/img/workflow.png new file mode 100644 index 00000000..e9603ce5 Binary files /dev/null and b/doc/img/workflow.png differ diff --git a/doc/index.rst b/doc/index.rst new file mode 100644 index 00000000..bc036220 --- /dev/null +++ b/doc/index.rst @@ -0,0 +1,161 @@ +PyPSA-Eur: An Open Optimisation Model of the European Transmission System +========================================================================= + +.. image:: https://img.shields.io/github/tag-date/pypsa/pypsa-eur + :alt: GitHub tag + +.. image:: https://readthedocs.org/projects/pypsa-eur/badge/?version=latest + :target: https://pypsa-eur.readthedocs.io/en/latest/?badge=latest + :alt: Documentation Status + +.. image:: https://img.shields.io/github/license/pypsa/pypsa-eur + :alt: GitHub + +.. image:: https://img.shields.io/github/repo-size/pypsa/pypsa-eur + :alt: GitHub repo size + +.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.1246852.svg + :target: https://doi.org/10.5281/zenodo.1246852 + +.. image:: https://badges.gitter.im/PyPSA/community.svg + :target: https://gitter.im/PyPSA/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge + :alt: Chat on Gitter + +PyPSA-Eur is an open model dataset of the European power system at the +transmission network level that covers the full ENTSO-E area. + +It contains alternating current lines at and above 220 kV voltage level and all high voltage direct current lines, substations, an open database of conventional power plants, time series for electrical demand and variable renewable generator availability, and geographic potentials for the expansion of wind and solar power. + +The model is suitable both for operational studies and generation and transmission expansion planning studies. The continental scope and highly resolved spatial scale enables a proper description of the long-range smoothing effects for renewable power generation and their varying resource availability. + +.. image:: img/base.png + +The restriction to freely available and open data encourages the open exchange of model data developments and eases the comparison of model results. It provides a full, automated software pipeline to assemble the load-flow-ready model from the original datasets, which enables easy replacement and improvement of the individual parts. + +PyPSA-Eur is designed to be imported into the open toolbox `PyPSA `_ for which `documentation `_ is available as well. + +This project is maintained by the `Energy System Modelling group `_ at the `Institute for Automation and Applied Informatics `_ at the `Karlsruhe Institute of Technology `_. The group is funded by the `Helmholtz Association `_ until 2024. Previous versions were developed by the `Renewable Energy Group `_ at `FIAS `_ to carry out simulations for the `CoNDyNet project `_, financed by the `German Federal Ministry for Education and Research (BMBF) `_ as part of the `Stromnetze Research Initiative `_. + +Documentation +============= + +**Getting Started** + +* :doc:`introduction` +* :doc:`installation` +* :doc:`tutorial` + +.. toctree:: + :hidden: + :maxdepth: 1 + :caption: Getting Started + + introduction + installation + tutorial + +**Configuration** + +* :doc:`wildcards` +* :doc:`configuration` +* :doc:`costs` + +.. toctree:: + :hidden: + :maxdepth: 1 + :caption: Configuration + + wildcards + configuration + costs + +**Rules Overview** + +* :doc:`preparation` +* :doc:`simplification` +* :doc:`solving` +* :doc:`plotting` + +.. toctree:: + :hidden: + :maxdepth: 1 + :caption: Rules Overview + + preparation + simplification + solving + plotting + +**References** + +* :doc:`release_notes` +* :doc:`limitations` +* :doc:`contributing` + +.. toctree:: + :hidden: + :maxdepth: 1 + :caption: References + + release_notes + limitations + contributing + +Learning Energy System Modelling +================================ + +If you are (relatively) new to energy system modelling and optimisation +and plan to use PyPSA-Eur, the following resources are *one way* to get started +in addition to reading this documentation. + +- Documentation of `PyPSA `_, the package for + simulating and optimising modern power systems which PyPSA-Eur uses under the hood. +- Course on `Energy System Modelling `_, + Karlsruhe Institute of Technology (KIT), `Dr. Tom Brown `_ + +Citing PyPSA-Eur +================ + +If you use PyPSA-Eur for your research, we would appreciate it if you would cite the following paper: + +- Jonas Hörsch, Fabian Hofmann, David Schlachtberger, and Tom Brown. `PyPSA-Eur: An open optimisation model of the European transmission system `_. Energy Strategy Reviews, 22:207-215, 2018. `arXiv:1806.01613 `_, `doi:10.1016/j.esr.2018.08.012 `_. + +Please use the following BibTeX: :: + + @article{PyPSAEur, + author = "Jonas Hoersch and Fabian Hofmann and David Schlachtberger and Tom Brown", + title = "PyPSA-Eur: An open optimisation model of the European transmission system", + journal = "Energy Strategy Reviews", + volume = "22", + pages = "207 - 215", + year = "2018", + issn = "2211-467X", + doi = "10.1016/j.esr.2018.08.012", + eprint = "1806.01613" + } + + +If you want to cite a specific PyPSA-Eur version, each release of PyPSA-Eur is stored on Zenodo with a release-specific DOI. +This can be found linked from the overall PyPSA-Eur Zenodo DOI: + +.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.1246852.svg + :target: https://doi.org/10.5281/zenodo.1246852 + + +Licence +======= + +The code in PyPSA-Eur is released as free software under the `GPLv3 +`_, see +`LICENSE `_. +However, different licenses and terms of use apply to the various input data, which are summarised below. +More details are included in +`the description of the data bundles on zenodo `_. + +.. csv-table:: + :header-rows: 1 + :file: configtables/licenses.csv + +* *BY: Attribute Source* +* *NC: Non-Commercial Use Only* +* *SA: Share Alike* \ No newline at end of file diff --git a/doc/installation.rst b/doc/installation.rst new file mode 100644 index 00000000..33dde1d4 --- /dev/null +++ b/doc/installation.rst @@ -0,0 +1,159 @@ +.. _installation: + +########################################## +Installation +########################################## + +The subsequently described installation steps are demonstrated as shell commands, where the path before the ``%`` sign denotes the +directory in which the commands following the ``%`` should be entered. + +Clone the Repository +==================== + +First of all, clone the `PyPSA-Eur repository `_ using the version control system ``git``. +The path to the directory into which the ``git repository`` is cloned, must **not** have any spaces! + +.. code:: bash + + /some/other/path % cd /some/path/without/spaces + + /some/path/without/spaces % git clone https://github.com/PyPSA/pypsa-eur.git + +.. note:: + If you do not have ``git`` installed, follow installation instructions `here `_. + +.. _deps: + +Install Python Dependencies +=============================== + +PyPSA-Eur relies on a set of other Python packages to function. +We recommend using the package manager and environment management system ``conda`` to install them. +Install `miniconda `_, which is a mini version of `Anaconda `_ that includes only ``conda`` and its dependencies or make sure ``conda`` is already installed on your system. +For instructions for your operating system follow the ``conda`` `installation guide `_. + +The python package requirements are curated in the `environment.yaml `_ file. +The environment can be installed and activated using + +.. code:: bash + + .../pypsa-eur % conda env create -f environment.yaml + + .../pypsa-eur % conda activate pypsa-eur + +.. note:: + Note that activation is local to the currently open shell! + After opening a new terminal window, one needs to reissue the second command! + +.. _data: + +Download Data Dependencies +============================== + +Not all data dependencies are shipped with the git repository, +since git is not suited for handling large changing files. +Instead we provide separate data bundles which can be obtained +using the described shell commands or by downloading and +extracting them manually in the locations outlined below. + +.. note:: + + The :ref:`tutorial` uses smaller data bundles than required for the full model. + To start with the tutorial, substitute with the links below using the following alternatives: + + .. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.3517921.svg + :target: https://doi.org/10.5281/zenodo.3517921 + + .. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.3518020.svg + :target: https://doi.org/10.5281/zenodo.3518020 + + - **Data Bundle:** ``https://zenodo.org/record/3517921/files/pypsa-eur-tutorial-data-bundle.tar.xz`` (197 MB) + - **Cutouts:** ``https://zenodo.org/record/3518020/files/pypsa-eur-tutorial-cutouts.tar.xz`` (19 MB) + +.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.3517935.svg + :target: https://doi.org/10.5281/zenodo.3517935 + +1. **Data Bundle:** `pypsa-eur-data-bundle.tar.xz `_ (1.4 GB) contains common GIS datasets like NUTS3 shapes, EEZ shapes, CORINE Landcover, Natura 2000 and also electricity specific summary statistics like historic per country yearly totals of hydro generation, GDP and POP on NUTS3 levels and per-country load time-series. It should be extracted in the ``data`` sub-directory, such that all files of the bundle are stored in the ``data/bundle`` subdirectory) + +.. code:: bash + + .../pypsa-eur/data % curl -OL "https://zenodo.org/record/3517935/files/pypsa-eur-data-bundle.tar.xz" + + .../pypsa-eur/data % tar xJf pypsa-eur-data-bundle.tar.xz + +.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.3517949.svg + :target: https://doi.org/10.5281/zenodo.3517949 + +2. **Cutouts:** `pypsa-eur-cutouts.tar.xz `_ (3.9 GB) are spatiotemporal subsets of the European weather data from the `ECMWF ERA5 `_ reanalysis dataset and the `CMSAF SARAH-2 `_ solar surface radiation dataset for the year 2013. They have been prepared by and are for use with the `atlite `_ tool. You can either generate them yourself using the ``build_cutouts`` rule or extract them directly into the ``pypsa-eur`` directory. To download cutouts yourself you need to `set up the CDS API `_. For more details read the `atlite documentation `_. For beginners, extracting the bundle is recommended: + +.. code:: bash + + .../pypsa-eur % curl -OL "https://zenodo.org/record/3517949/files/pypsa-eur-cutouts.tar.xz" + + .../pypsa-eur % tar xJf pypsa-eur-cutouts.tar.xz + +.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.3518215.svg + :target: https://doi.org/10.5281/zenodo.3518215 + +3. **Natura:** Optionally, you can download a rasterized version of the NATURA dataset `natura.tiff `_ and put it into the ``resources`` sub-directory. If you don't, it will be generated automatically, which is a time-consuming process. + +.. code:: bash + + .../pypsa-eur % curl -L "https://zenodo.org/record/3518215/files/natura.tiff" -o "resources/natura.tiff" + + +4. **Remove Archives:** Optionally, if you want to save disk space, you can delete ``data/pypsa-eur-data-bundle.tar.xz`` and ``pypsa-eur-cutouts.tar.xz`` once extracting the bundles is complete. E.g. + +.. code:: bash + + .../pypsa-eur % rm -rf data/pypsa-eur-data-bundle.tar.xz pypsa-eur-cutouts.tar.xz + +Install a Solver +================ + +PyPSA passes the PyPSA-Eur network model to an external solver for performing a total annual system cost minimization with optimal power flow. +PyPSA is known to work with the free software + +- `Ipopt `_ +- `Cbc `_ +- `GLPK `_ (`WinGLKP `_) + +and the non-free, commercial software (for which free academic licenses are available) + +- `Gurobi `_ +- `CPLEX `_ + +and any other solver that works with the underlying modelling framework `Pyomo `_. For installation instructions of these solvers for your operating system, follow the links above. + +.. seealso:: + `Getting a solver in the PyPSA documentation `_ + +.. note:: + Commercial solvers such as Gurobi and CPLEX currently significantly outperform open-source solvers for large-scale problems. + It might be the case that you can only retrieve solutions by using a commercial solver. + +.. _defaultconfig: + +Set Up the Default Configuration +================================ + +PyPSA-Eur has several configuration options that must be specified in a ``config.yaml`` file located in the root directory. +An example configuration ``config.default.yaml`` is maintained in the repository. +More details on the configuration options are in :ref:`config`. + +Before first use, create a ``config.yaml`` by copying the example. + +.. code:: bash + + .../pypsa-eur % cp config.default.yaml config.yaml + +Users are advised to regularly check their own ``config.yaml`` against changes in the ``config.default.yaml`` +when pulling a new version from the remote repository. + +.. Using PyPSA-Eur with Docker Images +.. ================================== + +.. If docker. Optional. +.. To run on cloud computing. +.. Gurobi license - floating token server - license must not be tied to a particular machine +.. Provide ``Dockerfile``. diff --git a/doc/introduction.rst b/doc/introduction.rst new file mode 100644 index 00000000..43dcae40 --- /dev/null +++ b/doc/introduction.rst @@ -0,0 +1,56 @@ +.. _intro: + +########################################## + Introduction +########################################## + +Workflow +========= + +The generation of the model is controlled by the workflow management system +`Snakemake `_. +In a nutshell, the ``Snakefile`` declares for each python script in the ``scripts`` directory a rule which describes which files the scripts consume and produce (their corresponding input and output files). +The ``snakemake`` tool then runs the scripts in the correct order according to the rules' input/output dependencies. +Moreover, it is able to track, what parts of the workflow have to be regenerated, when a data file or a script is modified/updated. + +For instance an invocation to + +.. code:: bash + + .../pypsa-eur % snakemake networks/elec_s_128.nc + +follows this dependency graph: + +.. image:: img/workflow.png + +The **blocks** represent the individual rules which are required to create the file ``networks/elec_s_128.nc``. The **arrows** indicate the outputs from preceding rules which a particular rule takes as input data. + +.. note:: + The dependency graph shown above was generated using + ``snakemake --dag networks/elec_s_128.nc | dot -Tpng > workflow.png`` + +For the use of ``snakemake``, it makes sense to familiarize oneself quickly with its `basic tutorial `_ and then read carefully through the section `Executing Snakemake `_, noting the arguments ``-n``, ``-r``, but also ``--dag``, ``-R`` and ``-t``. + +Scenarios, Configuration and Modification +========================================= + +It is easy to run PyPSA-Eur for multiple scenarios using the `wildcards feature `_ of ``snakemake``. Wildcards allow to generalise a rule to produce all files that follow a `regular expression `_ pattern, which e.g. defines one particular scenario. One can think of a wildcard as a parameter that shows up in the input/output file names and thereby determines which rules to run, what data to retrieve and what files to produce. **Details are explained in** :ref:`wildcards` **and** :ref:`scenario`. + +The model also has several further configuration options collected in the ``config.yaml`` file +located in the root directory, which that are not part of the scenarios. **All options are explained in detail in** :ref:`config`. + +Folder Structure +================ + +- ``data``: Includes input data that is not produced by any ``snakemake`` rule. +- ``scripts``: Includes all the Python scripts executed by the ``snakemake`` rules. +- ``resources``: Stores intermediate results of the workflow which can be picked up again by subsequent rules. +- ``networks``: Stores intermediate, unsolved stages of the PyPSA network that describes the energy system model. +- ``results``: Stores the solved PyPSA network data, summary files and plots. +- ``benchmarks``: Stores ``snakemake`` benchmarks. +- ``logs``: Stores log files about solving, including the solver output, console output and the output of a memory logger. + +System Requirements +=================== + +Building the model with the scripts in this repository uses up to 20 GB of memory. Computing optimal investment and operation scenarios requires a strong interior-point solver compatible with the modelling library `Pyomo `_ like `Gurobi `_ or `CPLEX `_ with up to 100 GB of memory. \ No newline at end of file diff --git a/doc/limitations.rst b/doc/limitations.rst new file mode 100644 index 00000000..0820c953 --- /dev/null +++ b/doc/limitations.rst @@ -0,0 +1,54 @@ +########################################## +Limitations +########################################## + + +While the benefit of an openly available, functional and partially validated +model of the European transmission system is high, many approximations have +been made due to missing data. +The limitations of the dataset are listed below, +both as a warning to the user and as an encouragement to assist in +improving the approximations. + +- **Network topology:** + The grid data is based on a map of the ENTSO-E area that is known + to contain small distortions to improve readability. Since the exact impedances + of the lines are unknown, approximations based on line lengths and standard + line parameters were made that ignore specific conductoring choices for + particular lines. There is no openly available data on busbar configurations, switch + locations, transformers or reactive power compensation assets. + +- **Distribution networks:** + Using Voronoi cells to aggregate load and generator data to transmission + network substations ignores the topology of the underlying distribution network, + meaning that assets may be connected to the wrong substation. + +- **Power Demand:** + Assumptions + have been made about the distribution of load in each country proportional to + population and GDP that may not reflect local circumstances. + Openly available + data on load time series may not correspond to the true vertical load and is + not spatially disaggregated; assuming, as we have done, that the load time series + shape is the same at each node within each country ignores local differences. + +- **Currently installed renewable capacities:** + Information on existing wind, solar and small hydro, geothermal, marine and + biomass power plants are excluded from the dataset because of a lack of data + availability in many countries. Approximate distributions of wind and solar + plants in each country can be generated that are proportional to the capacity + factor at each location. + +- **Hydro-electric power plants:** + The database of hydro-electric power plants does not include plant-specific + energy storage information, so that blanket values based on country storage + totals have been used. Inflow time series are based on country-wide approximations, + ignoring local topography and basin drainage; in principle a full + hydrological model should be used. + +- **International interactions:** + Border connections and power flows to Russia, + Belarus, Ukraine, Turkey and Morocco have not been taken into account; + islands which are not connected to the main European system, such as Malta, + Crete and Cyprus, are also excluded from the model. + \ No newline at end of file diff --git a/doc/make.bat b/doc/make.bat new file mode 100644 index 00000000..25b82245 --- /dev/null +++ b/doc/make.bat @@ -0,0 +1,263 @@ +@ECHO OFF + +REM Command file for Sphinx documentation + +if "%SPHINXBUILD%" == "" ( + set SPHINXBUILD=sphinx-build +) +set BUILDDIR=_build +set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% . +set I18NSPHINXOPTS=%SPHINXOPTS% . +if NOT "%PAPER%" == "" ( + set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% + set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% +) + +if "%1" == "" goto help + +if "%1" == "help" ( + :help + echo.Please use `make ^` where ^ is one of + echo. html to make standalone HTML files + echo. dirhtml to make HTML files named index.html in directories + echo. singlehtml to make a single large HTML file + echo. pickle to make pickle files + echo. json to make JSON files + echo. htmlhelp to make HTML files and a HTML help project + echo. qthelp to make HTML files and a qthelp project + echo. devhelp to make HTML files and a Devhelp project + echo. epub to make an epub + echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter + echo. text to make text files + echo. man to make manual pages + echo. texinfo to make Texinfo files + echo. gettext to make PO message catalogs + echo. changes to make an overview over all changed/added/deprecated items + echo. xml to make Docutils-native XML files + echo. pseudoxml to make pseudoxml-XML files for display purposes + echo. linkcheck to check all external links for integrity + echo. doctest to run all doctests embedded in the documentation if enabled + echo. coverage to run coverage check of the documentation if enabled + goto end +) + +if "%1" == "clean" ( + for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i + del /q /s %BUILDDIR%\* + goto end +) + + +REM Check if sphinx-build is available and fallback to Python version if any +%SPHINXBUILD% 1>NUL 2>NUL +if errorlevel 9009 goto sphinx_python +goto sphinx_ok + +:sphinx_python + +set SPHINXBUILD=python -m sphinx.__init__ +%SPHINXBUILD% 2> nul +if errorlevel 9009 ( + echo. + echo.The 'sphinx-build' command was not found. Make sure you have Sphinx + echo.installed, then set the SPHINXBUILD environment variable to point + echo.to the full path of the 'sphinx-build' executable. Alternatively you + echo.may add the Sphinx directory to PATH. + echo. + echo.If you don't have Sphinx installed, grab it from + echo.http://sphinx-doc.org/ + exit /b 1 +) + +:sphinx_ok + + +if "%1" == "html" ( + %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html + if errorlevel 1 exit /b 1 + echo. + echo.Build finished. The HTML pages are in %BUILDDIR%/html. + goto end +) + +if "%1" == "dirhtml" ( + %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml + if errorlevel 1 exit /b 1 + echo. + echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. + goto end +) + +if "%1" == "singlehtml" ( + %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml + if errorlevel 1 exit /b 1 + echo. + echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. + goto end +) + +if "%1" == "pickle" ( + %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle + if errorlevel 1 exit /b 1 + echo. + echo.Build finished; now you can process the pickle files. + goto end +) + +if "%1" == "json" ( + %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json + if errorlevel 1 exit /b 1 + echo. + echo.Build finished; now you can process the JSON files. + goto end +) + +if "%1" == "htmlhelp" ( + %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp + if errorlevel 1 exit /b 1 + echo. + echo.Build finished; now you can run HTML Help Workshop with the ^ +.hhp project file in %BUILDDIR%/htmlhelp. + goto end +) + +if "%1" == "qthelp" ( + %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp + if errorlevel 1 exit /b 1 + echo. + echo.Build finished; now you can run "qcollectiongenerator" with the ^ +.qhcp project file in %BUILDDIR%/qthelp, like this: + echo.^> qcollectiongenerator %BUILDDIR%\qthelp\PyPSA.qhcp + echo.To view the help file: + echo.^> assistant -collectionFile %BUILDDIR%\qthelp\PyPSA.ghc + goto end +) + +if "%1" == "devhelp" ( + %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp + if errorlevel 1 exit /b 1 + echo. + echo.Build finished. + goto end +) + +if "%1" == "epub" ( + %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub + if errorlevel 1 exit /b 1 + echo. + echo.Build finished. The epub file is in %BUILDDIR%/epub. + goto end +) + +if "%1" == "latex" ( + %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex + if errorlevel 1 exit /b 1 + echo. + echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. + goto end +) + +if "%1" == "latexpdf" ( + %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex + cd %BUILDDIR%/latex + make all-pdf + cd %~dp0 + echo. + echo.Build finished; the PDF files are in %BUILDDIR%/latex. + goto end +) + +if "%1" == "latexpdfja" ( + %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex + cd %BUILDDIR%/latex + make all-pdf-ja + cd %~dp0 + echo. + echo.Build finished; the PDF files are in %BUILDDIR%/latex. + goto end +) + +if "%1" == "text" ( + %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text + if errorlevel 1 exit /b 1 + echo. + echo.Build finished. The text files are in %BUILDDIR%/text. + goto end +) + +if "%1" == "man" ( + %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man + if errorlevel 1 exit /b 1 + echo. + echo.Build finished. The manual pages are in %BUILDDIR%/man. + goto end +) + +if "%1" == "texinfo" ( + %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo + if errorlevel 1 exit /b 1 + echo. + echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. + goto end +) + +if "%1" == "gettext" ( + %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale + if errorlevel 1 exit /b 1 + echo. + echo.Build finished. The message catalogs are in %BUILDDIR%/locale. + goto end +) + +if "%1" == "changes" ( + %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes + if errorlevel 1 exit /b 1 + echo. + echo.The overview file is in %BUILDDIR%/changes. + goto end +) + +if "%1" == "linkcheck" ( + %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck + if errorlevel 1 exit /b 1 + echo. + echo.Link check complete; look for any errors in the above output ^ +or in %BUILDDIR%/linkcheck/output.txt. + goto end +) + +if "%1" == "doctest" ( + %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest + if errorlevel 1 exit /b 1 + echo. + echo.Testing of doctests in the sources finished, look at the ^ +results in %BUILDDIR%/doctest/output.txt. + goto end +) + +if "%1" == "coverage" ( + %SPHINXBUILD% -b coverage %ALLSPHINXOPTS% %BUILDDIR%/coverage + if errorlevel 1 exit /b 1 + echo. + echo.Testing of coverage in the sources finished, look at the ^ +results in %BUILDDIR%/coverage/python.txt. + goto end +) + +if "%1" == "xml" ( + %SPHINXBUILD% -b xml %ALLSPHINXOPTS% %BUILDDIR%/xml + if errorlevel 1 exit /b 1 + echo. + echo.Build finished. The XML files are in %BUILDDIR%/xml. + goto end +) + +if "%1" == "pseudoxml" ( + %SPHINXBUILD% -b pseudoxml %ALLSPHINXOPTS% %BUILDDIR%/pseudoxml + if errorlevel 1 exit /b 1 + echo. + echo.Build finished. The pseudo-XML files are in %BUILDDIR%/pseudoxml. + goto end +) + +:end diff --git a/doc/plotting.rst b/doc/plotting.rst new file mode 100644 index 00000000..283e3dce --- /dev/null +++ b/doc/plotting.rst @@ -0,0 +1,171 @@ +########################################## +Plotting and Summary +########################################## + +.. warning:: The corresponding code is currently under revision and has only minimal documentation. + +.. _flh: + +Rule ``build_country_flh`` +============================= + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 0 [color="0.31 0.6 0.85", + fillcolor=gray, + label=build_country_flh, + style=filled]; + 1 [color="0.06 0.6 0.85", + label=base_network]; + 1 -> 0; + 2 [color="0.42 0.6 0.85", + label=build_natura_raster]; + 2 -> 0; + 3 [color="0.58 0.6 0.85", + label=build_shapes]; + 3 -> 0; + 4 [color="0.14 0.6 0.85", + label=build_cutout]; + 4 -> 0; + } + +| + +.. automodule:: build_country_flh + +.. _plot_potentials: + +Rule ``plot_p_nom_max`` +========================== + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 0 [color="0.42 0.6 0.85", + fillcolor=gray, + label=plot_p_nom_max, + style=filled]; + 1 [color="0.58 0.6 0.85", + label=cluster_network]; + 1 -> 0; + } + +| + +.. automodule:: plot_p_nom_max + +.. _summary: + +Rule ``make_summary`` +======================== + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 0 [color="0.47 0.6 0.85", + fillcolor=gray, + label=make_summary, + style=filled]; + 1 [color="0.11 0.6 0.85", + label=solve_network]; + 1 -> 0; + } + +| + +.. automodule:: make_summary + +.. _summary_plot: + +Rule ``plot_summary`` +======================== + +.. graphviz:: + :align: center + + + +| + +.. automodule:: plot_summary + +.. _map_plot: + +Rule ``plot_network`` +======================== + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 0 [color="0.00 0.6 0.85", + fillcolor=gray, + label=plot_network, + style=filled]; + 1 [color="0.50 0.6 0.85", + label=solve_network]; + 1 -> 0; + } + +| + +.. automodule:: plot_network + +.. image:: img/tech-colors.png + :align: center \ No newline at end of file diff --git a/doc/preparation.rst b/doc/preparation.rst new file mode 100644 index 00000000..5fb891dc --- /dev/null +++ b/doc/preparation.rst @@ -0,0 +1,38 @@ +########################################## +Preparing Networks +########################################## + +The preparation process of the PyPSA-Eur energy system model consists of a group of ``snakemake`` +rules which are briefly outlined and explained in detail in the sections below: + +- :mod:`build_shapes` generates GeoJSON files with shapes of the countries, exclusive economic zones and `NUTS3 `_ areas. +- :mod:`build_cutout` prepares smaller weather data portions from `ERA5 `_ for cutout ``europe-2013-era5`` and SARAH for cutout ``europe-2013-sarah``. + +With these and the externally extracted ENTSO-E online map topology +(``data/entsoegridkit``), it can build a base PyPSA network with the following rules: + +- :mod:`base_network` builds and stores the base network with all buses, HVAC lines and HVDC links, while +- :mod:`build_bus_regions` determines `Voronoi cells `_ for all substations. + +Then the process continues by calculating conventional power plant capacities, potentials, and per-unit availability time series for variable renewable energy carriers and hydro power plants with the following rules: + +- :mod:`build_powerplants` for today's thermal power plant capacities using `powerplantmatching `_ allocating these to the closest substation for each powerplant, +- :mod:`build_renewable_profiles` for the hourly capacity factors and installation potentials constrained by land-use in each substation's Voronoi cell for PV, onshore and offshore wind, and +- :mod:`build_hydro_profile` for the hourly per-unit hydro power availability time series. + +The central rule :mod:`add_electricity` then ties all the different data inputs +together into a detailed PyPSA network stored in ``networks/elec.nc``. + +.. toctree:: + :caption: Overview + + preparation/build_shapes + preparation/build_cutout + preparation/prepare_links_p_nom + preparation/base_network + preparation/build_bus_regions + preparation/build_natura_raster + preparation/build_powerplants + preparation/build_renewable_profiles + preparation/build_hydro_profile + preparation/add_electricity diff --git a/doc/preparation/add_electricity.rst b/doc/preparation/add_electricity.rst new file mode 100644 index 00000000..c6937260 --- /dev/null +++ b/doc/preparation/add_electricity.rst @@ -0,0 +1,52 @@ +.. _electricity: + +Rule ``add_electricity`` +============================= + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 3 [color="0.25 0.6 0.85", + label=simplify_network]; + 4 [color="0.50 0.6 0.85", + fillcolor=gray, + label=add_electricity, + style=filled]; + 4 -> 3; + 5 [color="0.36 0.6 0.85", + label=build_bus_regions]; + 5 -> 4; + 6 [color="0.58 0.6 0.85", + label=base_network]; + 6 -> 4; + 7 [color="0.31 0.6 0.85", + label=build_powerplants]; + 7 -> 4; + 8 [color="0.28 0.6 0.85", + label=build_shapes]; + 8 -> 4; + 9 [color="0.22 0.6 0.85", + label=build_renewable_profiles]; + 9 -> 4; + 10 [color="0.44 0.6 0.85", + label=build_hydro_profile]; + 10 -> 4; + } + +| + +.. automodule:: add_electricity diff --git a/doc/preparation/base_network.rst b/doc/preparation/base_network.rst new file mode 100644 index 00000000..432dabaf --- /dev/null +++ b/doc/preparation/base_network.rst @@ -0,0 +1,49 @@ +.. _base: + +Rule ``base_network`` +============================= + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 4 [color="0.50 0.6 0.85", + label=add_electricity]; + 5 [color="0.36 0.6 0.85", + label=build_bus_regions]; + 6 [color="0.58 0.6 0.85", + fillcolor=gray, + label=base_network, + style=filled]; + 6 -> 4; + 6 -> 5; + 7 [color="0.31 0.6 0.85", + label=build_powerplants]; + 6 -> 7; + 9 [color="0.22 0.6 0.85", + label=build_renewable_profiles]; + 6 -> 9; + 8 [color="0.28 0.6 0.85", + label=build_shapes]; + 8 -> 6; + 11 [color="0.03 0.6 0.85", + label=prepare_links_p_nom]; + 11 -> 6; + } + +| + +.. automodule:: base_network \ No newline at end of file diff --git a/doc/preparation/build_bus_regions.rst b/doc/preparation/build_bus_regions.rst new file mode 100644 index 00000000..f30108b4 --- /dev/null +++ b/doc/preparation/build_bus_regions.rst @@ -0,0 +1,46 @@ +.. _busregions: + +Rule ``build_bus_regions`` +============================= + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 3 [color="0.25 0.6 0.85", + label=simplify_network]; + 4 [color="0.50 0.6 0.85", + label=add_electricity]; + 5 [color="0.36 0.6 0.85", + fillcolor=gray, + label=build_bus_regions, + style=filled]; + 5 -> 3; + 5 -> 4; + 9 [color="0.22 0.6 0.85", + label=build_renewable_profiles]; + 5 -> 9; + 6 [color="0.58 0.6 0.85", + label=base_network]; + 6 -> 5; + 8 [color="0.28 0.6 0.85", + label=build_shapes]; + 8 -> 5; + } + +| + +.. automodule:: build_bus_regions \ No newline at end of file diff --git a/doc/preparation/build_cutout.rst b/doc/preparation/build_cutout.rst new file mode 100644 index 00000000..c4c88fa0 --- /dev/null +++ b/doc/preparation/build_cutout.rst @@ -0,0 +1,37 @@ +.. _cutout: + +Rule ``build_cutout`` +============================= + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 9 [color="0.22 0.6 0.85", + label=build_renewable_profiles]; + 10 [color="0.44 0.6 0.85", + label=build_hydro_profile]; + 13 [color="0.17 0.6 0.85", + fillcolor=gray, + label=build_cutout, + style=filled]; + 13 -> 9; + 13 -> 10; + } + +| + +.. automodule:: build_cutout \ No newline at end of file diff --git a/doc/preparation/build_hydro_profile.rst b/doc/preparation/build_hydro_profile.rst new file mode 100644 index 00000000..34435908 --- /dev/null +++ b/doc/preparation/build_hydro_profile.rst @@ -0,0 +1,40 @@ +.. _hydroprofiles: + +Rule ``build_hydro_profile`` +=============================== + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 4 [color="0.61 0.6 0.85", + label=add_electricity]; + 8 [color="0.00 0.6 0.85", + label=build_shapes]; + 10 [color="0.11 0.6 0.85", + fillcolor=gray, + label=build_hydro_profile, + style=filled]; + 8 -> 10; + 10 -> 4; + 13 [color="0.56 0.6 0.85", + label=build_cutout]; + 13 -> 10; + } + +| + +.. automodule:: build_hydro_profile diff --git a/doc/preparation/build_natura_raster.rst b/doc/preparation/build_natura_raster.rst new file mode 100644 index 00000000..5d1fb6c8 --- /dev/null +++ b/doc/preparation/build_natura_raster.rst @@ -0,0 +1,34 @@ +.. _natura: + +Rule ``build_natura_raster`` +=============================== + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 9 [color="0.22 0.6 0.85", + label=build_renewable_profiles]; + 12 [color="0.31 0.6 0.85", + fillcolor=gray, + label=build_natura_raster, + style=filled]; + 12 -> 9; + } + +| + +.. automodule:: build_natura_raster diff --git a/doc/preparation/build_powerplants.rst b/doc/preparation/build_powerplants.rst new file mode 100644 index 00000000..310375df --- /dev/null +++ b/doc/preparation/build_powerplants.rst @@ -0,0 +1,37 @@ +.. _powerplants: + +Rule ``build_powerplants`` +============================= + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 4 [color="0.61 0.6 0.85", + label=add_electricity]; + 6 [color="0.17 0.6 0.85", + label=base_network]; + 7 [color="0.58 0.6 0.85", + fillcolor=gray, + label=build_powerplants, + style=filled]; + 6 -> 7; + 7 -> 4; + } + +| + +.. automodule:: build_powerplants diff --git a/doc/preparation/build_renewable_profiles.rst b/doc/preparation/build_renewable_profiles.rst new file mode 100644 index 00000000..2971cbf3 --- /dev/null +++ b/doc/preparation/build_renewable_profiles.rst @@ -0,0 +1,49 @@ +.. _renewableprofiles: + +Rule ``build_renewable_profiles`` +==================================== + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 4 [color="0.61 0.6 0.85", + label=add_electricity]; + 5 [color="0.19 0.6 0.85", + label=build_bus_regions]; + 9 [color="0.22 0.6 0.85", + fillcolor=gray, + label=build_renewable_profiles, + style=filled]; + 5 -> 9; + 9 -> 4; + 6 [color="0.17 0.6 0.85", + label=base_network]; + 6 -> 9; + 8 [color="0.00 0.6 0.85", + label=build_shapes]; + 8 -> 9; + 12 [color="0.31 0.6 0.85", + label=build_natura_raster]; + 12 -> 9; + 13 [color="0.56 0.6 0.85", + label=build_cutout]; + 13 -> 9; + } + +| + +.. automodule:: build_renewable_profiles diff --git a/doc/preparation/build_shapes.rst b/doc/preparation/build_shapes.rst new file mode 100644 index 00000000..3b6dd5c5 --- /dev/null +++ b/doc/preparation/build_shapes.rst @@ -0,0 +1,46 @@ +.. _shapes: + +Rule ``build_shapes`` +============================= + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 4 [color="0.61 0.6 0.85", + label=add_electricity]; + 5 [color="0.19 0.6 0.85", + label=build_bus_regions]; + 6 [color="0.17 0.6 0.85", + label=base_network]; + 8 [color="0.00 0.6 0.85", + fillcolor=gray, + label=build_shapes, + style=filled]; + 8 -> 4; + 8 -> 5; + 8 -> 6; + 9 [color="0.22 0.6 0.85", + label=build_renewable_profiles]; + 8 -> 9; + 10 [color="0.11 0.6 0.85", + label=build_hydro_profile]; + 8 -> 10; + } + +| + +.. automodule:: build_shapes diff --git a/doc/preparation/prepare_links_p_nom.rst b/doc/preparation/prepare_links_p_nom.rst new file mode 100644 index 00000000..288e6f78 --- /dev/null +++ b/doc/preparation/prepare_links_p_nom.rst @@ -0,0 +1,34 @@ +.. _links: + +Rule ``prepare_links_p_nom`` +=============================== + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 6 [color="0.17 0.6 0.85", + label=base_network]; + 11 [color="0.39 0.6 0.85", + fillcolor=gray, + label=prepare_links_p_nom, + style=filled]; + 11 -> 6; + } + +| + +.. automodule:: prepare_links_p_nom diff --git a/doc/references.bib b/doc/references.bib new file mode 100644 index 00000000..8ee3658b --- /dev/null +++ b/doc/references.bib @@ -0,0 +1,115 @@ +@article{PyPSAEur, + author = "Jonas Hoersch and Fabian Hofmann and David Schlachtberger and Tom Brown", + title = "PyPSA-Eur: An open optimisation model of the European transmission system", + journal = "Energy Strategy Reviews", + volume = "22", + pages = "207 - 215", + year = "2018", + issn = "2211-467X", + doi = "10.1016/j.esr.2018.08.012", + eprint = "1806.01613" + } + +@article{brown2019sectoral, + title={Sectoral Interactions as Carbon Dioxide Emissions Approach Zero in a Highly-Renewable European Energy System}, + author={Brown, Tom and Sch{\"a}fer, Mirko and Greiner, Martin}, + journal={Energies}, + volume={12}, + number={6}, + pages={1032}, + year={2019}, + publisher={Multidisciplinary Digital Publishing Institute} +} + +@article{neumann2019heuristics, + title={Heuristics for Transmission Expansion Planning in Low-Carbon Energy System Models}, + author={Neumann, Fabian and Brown, Tom}, + journal={arXiv preprint arXiv:1907.10548}, + year={2019} +} + +@article{weber2019counter, + title={Counter-intuitive behaviour of energy system models under CO2 caps and prices}, + author={Weber, Juliane and Heinrichs, Heidi Ursula and Gillessen, Bastian and Schumann, Diana and H{\"o}rsch, Jonas and Brown, Tom and Witthaut, Dirk}, + journal={Energy}, + volume={170}, + pages={22--30}, + year={2019}, + publisher={Elsevier} +} + +@phdthesis{horsch2018spatial, + title={Spatial Scaling in Renewable Energy Networks}, + author={H{\"o}rsch, Jonas}, + year={2018}, + school={Johann Wolfgang Goethe-Universit{\"a}t Frankfurt am Main} +} + +@inproceedings{tranberg2018flow, + title={Flow-based analysis of storage usage in a low-carbon European electricity scenario}, + author={Tranberg, Bo and Sch{\"a}fer, Mirko and Brown, Tom and H{\"o}rsch, Jonas and Greiner, Martin}, + booktitle={2018 15th International Conference on the European Energy Market (EEM)}, + pages={1--5}, + year={2018}, + organization={IEEE} +} + +@article{pagnier2018disturbance, + title={Disturbance propagation, inertia location and slow modes in large-scale high voltage power grids}, + author={Pagnier, Laurent and Jacquod, Philippe}, + journal={arXiv preprint arXiv:1810.04982}, + year={2018} +} + +@article{victoria2019role, + title={The role of storage technologies throughout the decarbonisation of the sector-coupled European energy system}, + author={Victoria, Marta and Zhu, Kun and Brown, Tom and Andresen, Gorm B and Greiner, Martin}, + journal={arXiv preprint arXiv:1906.06936}, + year={2019} +} + +@article{brownasynergies, + title={Synergies of sector coupling and transmission extension in a cost-optimised, highly renewable European energy system}, + author={Browna, T and Schlachtbergera, D and Kiesa, A and Schramma, S and Greinerb, M} +} + +@article{gardumi2019representation, + title={Representation of Balancing Options for Variable Renewables in Long-Term Energy System Models: An Application to OSeMOSYS}, + author={Gardumi, Francesco and Welsch, Manuel and Howells, Mark and Colombo, Emanuela}, + journal={Energies}, + volume={12}, + number={12}, + pages={2366}, + year={2019}, + publisher={Multidisciplinary Digital Publishing Institute} +} + +@article{zhu2019impact, + title={Impact of CO2 prices on the design of a highly decarbonised coupled electricity and heating system in Europe}, + author={Zhu, Kun and Victoria, Marta and Brown, Tom and Andresen, Gorm B and Greiner, Martin}, + journal={Applied energy}, + volume={236}, + pages={622--634}, + year={2019}, + publisher={Elsevier} +} + +@article{schlott2018impact, + title={The impact of climate change on a cost-optimal highly renewable European electricity network}, + author={Schlott, Markus and Kies, Alexander and Brown, Tom and Schramm, Stefan and Greiner, Martin}, + journal={Applied energy}, + volume={230}, + pages={1645--1659}, + year={2018}, + publisher={Elsevier} +} + +@article{gotzens2019performing, + title={Performing energy modelling exercises in a transparent way-The issue of data quality in power plant databases}, + author={Gotzens, Fabian and Heinrichs, Heidi and H{\"o}rsch, Jonas and Hofmann, Fabian}, + journal={Energy Strategy Reviews}, + volume={23}, + pages={1--12}, + year={2019}, + publisher={Elsevier} +} diff --git a/doc/release_notes.rst b/doc/release_notes.rst new file mode 100644 index 00000000..5aa24fd7 --- /dev/null +++ b/doc/release_notes.rst @@ -0,0 +1,10 @@ +########################################## +Release Notes +########################################## + +PyPSA-Eur 0.1.0 (DATE) +====================== + +This is the first release of PyPSA-Eur. It now features: + +* Documentation on installation, workflows and configuration settings. diff --git a/doc/simplification.rst b/doc/simplification.rst new file mode 100644 index 00000000..37407bd2 --- /dev/null +++ b/doc/simplification.rst @@ -0,0 +1,21 @@ + + +########################################## +Simplifying Networks +########################################## + +The simplification ``snakemake`` rules prepare **approximations** of the full model, for which it is computationally viable to co-optimize generation, storage and transmission capacities. + +- :mod:`simplify_network` transforms the transmission grid to a 380 kV only equivalent network, while +- :mod:`cluster_network` uses a `k-means `_ based clustering technique to partition the network into a given number of zones and then reduce the network to a representation with one bus per zone. + +The simplification and clustering steps are described in detail in the paper + +- Jonas Hörsch and Tom Brown. `The role of spatial scale in joint optimisations of generation and transmission for European highly renewable scenarios `_), *14th International Conference on the European Energy Market*, 2017. `arXiv:1705.07617 `_, `doi:10.1109/EEM.2017.7982024 `_. + +.. toctree:: + :caption: Overview + + simplification/simplify_network + simplification/cluster_network + simplification/prepare_network diff --git a/doc/simplification/cluster_network.rst b/doc/simplification/cluster_network.rst new file mode 100644 index 00000000..12ec434f --- /dev/null +++ b/doc/simplification/cluster_network.rst @@ -0,0 +1,38 @@ +.. _cluster: + +Rule ``cluster_network`` +=========================== + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 1 [color="0.50 0.6 0.85", + label=prepare_network]; + 2 [color="0.36 0.6 0.85", + fillcolor=gray, + label=cluster_network, + style=filled]; + 2 -> 1; + 3 [color="0.14 0.6 0.85", + label=simplify_network]; + 3 -> 2; + } + + +| + +.. automodule:: cluster_network diff --git a/doc/simplification/prepare_network.rst b/doc/simplification/prepare_network.rst new file mode 100644 index 00000000..d6ccfa12 --- /dev/null +++ b/doc/simplification/prepare_network.rst @@ -0,0 +1,37 @@ +.. _prepare: + +Rule ``prepare_network`` +=========================== + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 0 [color="0.53 0.6 0.85", + label=solve_network]; + 1 [color="0.50 0.6 0.85", + fillcolor=gray, + label=prepare_network, + style=filled]; + 1 -> 0; + 2 [color="0.36 0.6 0.85", + label=cluster_network]; + 2 -> 1; + } + +| + +.. automodule:: prepare_network diff --git a/doc/simplification/simplify_network.rst b/doc/simplification/simplify_network.rst new file mode 100644 index 00000000..0b3152e4 --- /dev/null +++ b/doc/simplification/simplify_network.rst @@ -0,0 +1,40 @@ +.. _simplify: + +Rule ``simplify_network`` +============================ + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 2 [color="0.36 0.6 0.85", + label=cluster_network]; + 3 [color="0.14 0.6 0.85", + fillcolor=gray, + label=simplify_network, + style=filled]; + 3 -> 2; + 4 [color="0.61 0.6 0.85", + label=add_electricity]; + 4 -> 3; + 5 [color="0.19 0.6 0.85", + label=build_bus_regions]; + 5 -> 3; + } + +| + +.. automodule:: simplify_network diff --git a/doc/solving.rst b/doc/solving.rst new file mode 100644 index 00000000..717aa2df --- /dev/null +++ b/doc/solving.rst @@ -0,0 +1,12 @@ +########################################## +Solving Networks +########################################## + +After generating and simplifying the networks they can be solved through the rule :mod:`solve_network` by using the collection rule :mod:`solve_all_elec_networks`. Moreover, networks can be solved for another focus with the derivative rules :mod:`solve_network` by using the collection rule :mod:`trace_solve_network` to log changes during iterations and :mod:`solve_network` by using the collection rule :mod:`solve_operations_network` for dispatch-only analyses on an already solved network. + +.. toctree:: + :caption: Overview + + solving/solve_network + solving/trace_solve_network + solving/solve_operations_network diff --git a/doc/solving/solve_network.rst b/doc/solving/solve_network.rst new file mode 100644 index 00000000..8c1ead82 --- /dev/null +++ b/doc/solving/solve_network.rst @@ -0,0 +1,34 @@ +.. _solve: + +Rule ``solve_network`` +========================= + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="3,3" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 0 [color="0.64 0.6 0.85", + fillcolor=gray, + label=solve_network, + style=filled]; + 1 [color="0.33 0.6 0.85", + label=prepare_network]; + 1 -> 0; + } + +| + +.. automodule:: solve_network diff --git a/doc/solving/solve_operations_network.rst b/doc/solving/solve_operations_network.rst new file mode 100644 index 00000000..0ff0d021 --- /dev/null +++ b/doc/solving/solve_operations_network.rst @@ -0,0 +1,37 @@ +.. _solve_operations: + +Rule ``solve_operations_network`` +==================================== + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 0 [color="0.06 0.6 0.85", + fillcolor=gray, + label=solve_operations_network, + style=filled]; + 1 [color="0.00 0.6 0.85", + label=cluster_network]; + 1 -> 0; + 2 [color="0.19 0.6 0.85", + label=solve_network]; + 2 -> 0; + } + +| + +.. automodule:: solve_operations_network diff --git a/doc/solving/trace_solve_network.rst b/doc/solving/trace_solve_network.rst new file mode 100644 index 00000000..79eec832 --- /dev/null +++ b/doc/solving/trace_solve_network.rst @@ -0,0 +1,34 @@ +.. _trace_solve: + +Rule ``trace_solve_network`` +=============================== + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph [bgcolor=white, + margin=0, + size="8,5" + ]; + node [fontname=sans, + fontsize=10, + penwidth=2, + shape=box, + style=rounded + ]; + edge [color=grey, + penwidth=2 + ]; + 0 [color="0.17 0.6 0.85", + fillcolor=gray, + label=trace_solve_network, + style=filled]; + 1 [color="0.58 0.6 0.85", + label=prepare_network]; + 1 -> 0; + } + +| + +.. automodule:: trace_solve_network diff --git a/doc/tutorial.rst b/doc/tutorial.rst new file mode 100644 index 00000000..cd5cf67c --- /dev/null +++ b/doc/tutorial.rst @@ -0,0 +1,288 @@ +.. _tutorial: + +##################### +Tutorial +##################### + +Before getting started with **PyPSA-Eur** it makes sense to be familiar +with its general modelling framework `PyPSA `_. + +Running the tutorial requires limited computational resources compared to the full model, +which allows the user to explore most of its functionalities on a local machine. +It takes approximately five minutes to complete and +requires 3 GB of memory along with 1 GB free disk space. + +The tutorial will cover examples on how to + +- configure and customise the PyPSA-Eur model and +- run the ``snakemake`` workflow step by step from network creation to the solved network. + +If not yet completed, follow the :ref:`installation` steps but +substitute the links for the **data bundle** and the **cutouts** with the following lightweight alternatives: + + .. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.3517921.svg + :target: https://doi.org/10.5281/zenodo.3517921 + + .. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.3518020.svg + :target: https://doi.org/10.5281/zenodo.3518020 + + - **Data Bundle:** ``https://zenodo.org/record/3517921/files/pypsa-eur-tutorial-data-bundle.tar.xz`` (197 MB) + + + - **Cutouts:** ``https://zenodo.org/record/3518020/files/pypsa-eur-tutorial-cutouts.tar.xz`` (19 MB) + +The configuration of the tutorial is included in the ``config.tutorial.yaml``. +To run the tutorial, use this as your configuration file ``config.yaml``. + +.. code:: bash + + .../pypsa-eur % cp config.tutorial.yaml config.yaml + +How to customise PyPSA-Eur? +=========================== + +The model can be adapted to only include selected countries (e.g. Germany) instead of all European countries to limit the spatial scope. + +.. literalinclude:: ../config.tutorial.yaml + :language: yaml + :lines: 13 + +Likewise, the example's temporal scope can be restricted (e.g. to a single month). + +.. literalinclude:: ../config.tutorial.yaml + :language: yaml + :lines: 15-18 + +It is also possible to allow less or more carbon-dioxide emissions. Here, we limit the emissions of Germany 100 Megatonnes per year. + +.. literalinclude:: ../config.tutorial.yaml + :language: yaml + :lines: 26 + +PyPSA-Eur also includes a database of existing conventional powerplants. +We can select which types of powerplants we like to be included with fixed capacities: + +.. literalinclude:: ../config.tutorial.yaml + :language: yaml + :lines: 36 + +To accurately model the temporal and spatial availability of renewables such as wind and solar energy, we rely on historical weather data. +It is advisable to adapt the required range of coordinates to the selection of countries. + +.. literalinclude:: ../config.tutorial.yaml + :language: yaml + :lines: 38-47 + +We can also decide which weather data source should be used to calculate potentials and capacity factor time-series for each carrier. +For example, we may want to use the ERA-5 dataset for solar and not the default SARAH-2 dataset. + +.. literalinclude:: ../config.tutorial.yaml + :language: yaml + :lines: 48,91-92 + +Finally, it is possible to pick a solver. For instance, this tutorial uses the open-source solvers CBC and Ipopt and does not rely +on the commercial solvers Gurobi or CPLEX (for which free academic licenses are available). + +.. literalinclude:: ../config.tutorial.yaml + :language: yaml + :lines: 156-157 + +.. note:: + + To run the tutorial, either install CBC and Ipopt (see instructions for :ref:`installation`). + + Alternatively, choose another installed solver in the ``config.yaml`` at ``solving: solver:``. + +Note, that we only note major changes to the provided default configuration that is comprehensibly documented in :ref:`config`. +There are many more configuration options beyond what is adapted for the tutorial! + +How to use the ``snakemake`` rules? +=================================== + +Open a terminal, go into the PyPSA-Eur directory, and activate the ``pypsa-eur`` environment with + +.. code:: bash + + .../pypsa-eur % conda activate pypsa-eur + +Let's say based on the modifications above we would like to solve a very simplified model +clustered down to 6 buses and every 24 hours aggregated to one snapshot. The command + +.. code:: bash + + .../pypsa-eur % snakemake results/networks/elec_s_6_lcopt_Co2L-24H. + +orders ``snakemake`` to run the script ``solve_network`` that produces the solved network and stores it in ``.../pypsa-eur/results/networks`` with the name ``elec_s_6_lcopt_Co2L-24H.nc``: + +.. code:: + + rule solve_network: + input: "networks/{network}_s{simpl}_{clusters}_l{ll}_{opts}.nc" + output: "results/networks/{network}_s{simpl}_{clusters}_l{ll}_{opts}.nc" + [...] + script: "scripts/solve_network.py" + +This triggers a workflow of multiple preceding jobs that depend on each rule's inputs and outputs: + +.. graphviz:: + :align: center + + digraph snakemake_dag { + graph[bgcolor=white, margin=0]; + node[shape=box, style=rounded, fontname=sans, fontsize=10, penwidth=2]; + edge[penwidth=2, color=grey]; + 0[label = "solve_network", color = "0.10 0.6 0.85", style="rounded"]; + 1[label = "prepare_network\nll: copt\nopts: Co2L-24H", color = "0.13 0.6 0.85", style="rounded"]; + 2[label = "cluster_network\nclusters: 6", color = "0.51 0.6 0.85", style="rounded"]; + 3[label = "simplify_network\nnetwork: elec\nsimpl: ", color = "0.00 0.6 0.85", style="rounded"]; + 4[label = "add_electricity", color = "0.60 0.6 0.85", style="rounded"]; + 5[label = "build_bus_regions", color = "0.19 0.6 0.85", style="rounded"]; + 6[label = "base_network", color = "0.38 0.6 0.85", style="rounded"]; + 7[label = "build_shapes", color = "0.03 0.6 0.85", style="rounded"]; + 8[label = "build_renewable_profiles\ntechnology: onwind", color = "0.48 0.6 0.85", style="rounded"]; + 9[label = "build_renewable_profiles\ntechnology: offwind-ac", color = "0.48 0.6 0.85", style="rounded"]; + 10[label = "build_renewable_profiles\ntechnology: offwind-dc", color = "0.48 0.6 0.85", style="rounded"]; + 11[label = "build_renewable_profiles\ntechnology: solar", color = "0.48 0.6 0.85", style="rounded"]; + 12[label = "build_cutout\ncutout: europe-2013-era5", color = "0.35 0.6 0.85", style="rounded,dashed"]; + 1 -> 0 + 2 -> 1 + 3 -> 2 + 4 -> 3 + 5 -> 3 + 6 -> 4 + 5 -> 4 + 7 -> 4 + 8 -> 4 + 9 -> 4 + 10 -> 4 + 11 -> 4 + 7 -> 5 + 6 -> 5 + 7 -> 6 + 6 -> 8 + 7 -> 8 + 5 -> 8 + 12 -> 8 + 6 -> 9 + 7 -> 9 + 5 -> 9 + 12 -> 9 + 6 -> 10 + 7 -> 10 + 5 -> 10 + 12 -> 10 + 6 -> 11 + 7 -> 11 + 5 -> 11 + 12 -> 11 + } + +| + +In the terminal, this will show up as a list of jobs to be run: + +.. code:: bash + + Building DAG of jobs... + Using shell: /bin/bash + Provided cores: 1 + Rules claiming more threads will be scaled down. + Unlimited resources: mem + Job counts: + count jobs + 1 add_electricity + 1 base_network + 1 build_bus_regions + 4 build_renewable_profiles + 1 build_shapes + 1 cluster_network + 1 prepare_network + 1 simplify_network + 1 solve_network + 12 + +``snakemake`` then runs these jobs in the correct order. + +A job (here ``simplify_network``) will display its attributes and normally some logs in the terminal: + +.. code:: bash + + [] + rule simplify_network: + input: networks/elec.nc, data/costs.csv, resources/regions_onshore.geojson, resources/regions_offshore.geojson + output: networks/elec_s.nc, resources/regions_onshore_elec_s.geojson, resources/regions_offshore_elec_s.geojson, resources/clustermaps_elec_s.h5 + jobid: 3 + benchmark: benchmarks/simplify_network/elec_s + wildcards: network=elec, simpl= + resources: mem=4000 + + INFO:pypsa.io:Imported network elec.nc has buses, carriers, generators, lines, links, loads, storage_units, transformers + INFO:__main__:Mapping all network lines onto a single 380kV layer + INFO:__main__:Simplifying connected link components + INFO:__main__:Removing stubs + INFO:__main__:Displacing offwind-ac generator(s) and adding connection costs to capital_costs: 20128 Eur/MW/a for `5718 offwind-ac` + INFO:__main__:Displacing offwind-dc generator(s) and adding connection costs to capital_costs: 14994 Eur/MW/a for `5718 offwind-dc`, 26939 Eur/MW/a for `5724 offwind-dc`, 29621 Eur/MW/a for `5725 offwind-dc` + INFO:pypsa.io:Exported network elec_s.nc has lines, carriers, links, storage_units, loads, buses, generators + [] + Finished job 3. + 9 of 12 steps (75%) done + +Once the whole worktree is finished, it should show state so in the terminal: + +.. code:: bash + + Finished job 0. + 12 of 12 steps (100%) done + Complete log: /home/XXXX/pypsa-eur/.snakemake/log/20XX-XX-XXTXX.snakemake.log + snakemake results/networks/elec_s_6_lcopt_Co2L-24H.nc 519,84s user 34,26s system 242% cpu 3:48,83 total + +You will notice that many intermediate stages are saved, namely the outputs of each individual ``snakemake`` rule. + +You can produce any output file occuring in the ``Snakefile`` by running + +.. code:: bash + + .../pypsa-eur % snakemake + +For example, you can explore the evolution of the PyPSA networks by running + +#. ``.../pypsa-eur % snakemake networks/base.nc`` +#. ``.../pypsa-eur % snakemake networks/elec.nc`` +#. ``.../pypsa-eur % snakemake networks/elec_s.nc`` +#. ``.../pypsa-eur % snakemake networks/elec_s_6.nc`` +#. ``.../pypsa-eur % snakemake networks/elec_s_6_lcopt_Co2L-24H.nc`` + +There's a special rule: If you simply run + +.. code:: bash + + .../pypsa-eur % snakemake + +the wildcards given in ``scenario`` in the configuration file ``config.yaml`` are used: + +.. literalinclude:: ../config.tutorial.yaml + :language: yaml + :lines: 6-11 + +In this example we would not only solve a 6-node model of Germany but also a 2-node model. + +How to analyse solved networks? +=============================== + +The solved networks can be analysed just like any other PyPSA network (e.g. in Jupyter Notebooks). + +.. code:: python + + import pypsa + + network = pypsa.Network("results/networks/elec_s_6_lcopt_Co2L-24H.nc") + + ... + +For inspiration, read the `examples section in the PyPSA documentation `_. + +.. note:: + + There are rules for summaries and plotting available in the repository of PyPSA-Eur. + + They are currently under revision and therefore not yet documented. diff --git a/doc/wildcards.rst b/doc/wildcards.rst new file mode 100644 index 00000000..8a177613 --- /dev/null +++ b/doc/wildcards.rst @@ -0,0 +1,191 @@ +.. _wildcards: + +######### +Wildcards +######### + +It is easy to run PyPSA-Eur for multiple scenarios using the wildcards feature of ``snakemake``. +Wildcards allow to generalise a rule to produce all files that follow a regular expression pattern +which e.g. defines one particular scenario. One can think of a wildcard as a parameter that shows +up in the input/output file names of the ``Snakefile`` and thereby determines which rules to run, +what data to retrieve and what files to produce. + +Detailed explanations of how wildcards work in ``snakemake`` can be found in the +`relevant section of the documentation `_. + +.. _network: + +The ``{network}`` wildcard +========================== + +The ``{network}`` wildcard specifies the considered energy sector(s) +and, as currently only ``elec`` (for electricity) is included, +it currently represents rather a placeholder wildcard to facilitate +future extensions including multiple energy sectors at once. + +.. _simpl: + +The ``{simpl}`` wildcard +======================== + +The ``{simpl}`` wildcard specifies number of buses a detailed +network model should be pre-clustered to in the rule +:mod:`simplify_network` (before :mod:`cluster_network`). + +.. seealso:: + :mod:`simplify_network` + +.. _clusters: + +The ``{clusters}`` wildcard +=========================== + +The ``{clusters}`` wildcard specifies the number of buses a detailed +network model should be reduced to in the rule :mod:`cluster_network`. +The number of clusters must be lower than the total number of nodes +and higher than the number of countries. However, a country counts twice if +it has two asynchronous subnetworks (e.g. Denmark or Italy). + +If an `m` is placed behind the number of clusters (e.g. ``100m``), +generators are only moved to the clustered buses but not aggregated +by carrier; i.e. the clustered bus may have more than one e.g. wind generator. + +.. seealso:: + :mod:`cluster_network` + +.. _ll: + +The ``{ll}`` wildcard +===================== + +The ``{ll}`` wildcard specifies what limits on +line expansion are set for the optimisation model. +It is handled in the rule :mod:`prepare_network`. + +The wildcard, in general, consists of two parts: + + 1. The first part can be + ``v`` (for setting a limit on line volume) or + ``c`` (for setting a limit on line cost) + + 2. The second part can be + ``opt`` or a float bigger than one (e.g. 1.25). + + (a) If ``opt`` is chosen line expansion is optimised + according to its capital cost + (where the choice ``v`` only considers overhead costs for HVDC transmission lines, while + ``c`` uses more accurate costs distinguishing between + overhead and underwater sections and including inverter pairs). + + (b) ``v1.25`` will limit the total volume of line expansion + to 25 % of currently installed capacities weighted by + individual line lengths; investment costs are neglected. + + (c) ``c1.25`` will allow to build a transmission network that + costs no more than 25 % more than the current system. + +.. seealso:: + :mod:`prepare_network` + +.. _opts: + +The ``{opts}`` wildcard +======================= + +The ``{opts}`` wildcard triggers optional constraints, which are activated in either +:mod:`prepare_network` or the :mod:`solve_network` step. +It may hold multiple triggers separated by ``-``, i.e. ``Co2L-3H`` contains the +``Co2L`` trigger and the ``3H`` switch. There are currently: + + +.. csv-table:: + :header-rows: 1 + :widths: 10,20,10,10 + :file: configtables/opts.csv + +.. seealso:: + :mod:`prepare_network`, :mod:`solve_network` + +.. _country: + +The ``{country}`` wildcard +========================== + +The rules ``make_summary`` and ``plot_summary`` (generating summaries of all or a subselection +of the solved networks) as well as ``plot_p_nom_max`` (for plotting the cumulative +generation potentials for renewable technologies) can be narrowed to +individual countries using the ``{country}`` wildcard. + +If ``country = all``, then the rule acts on the network for all countries +defined in ``config.yaml``. If otherwise ``country = DE`` or another 2-letter +country code, then the network is narrowed to buses of this country +for the rule. For example to get a summary of the energy generated +in Germany (in the solution for Europe) use: + +.. code:: bash + + snakemake results/summaries/elec_s_all_lall_Co2L-3H_DE + +.. seealso:: + :mod:`make_summary`, :mod:`plot_summary`, :mod:`plot_p_nom_max` + +.. _cutout_wc: + +The ``{cutout}`` wildcard +========================= + +The ``{cutout}`` wildcard facilitates running the rule :mod:`build_cutout` +for all cutout configurations specified under ``atlite: cutouts:``. +These cutouts will be stored in a folder specified by ``{cutout}``. + +.. seealso:: + :mod:`build_cutout`, :ref:`atlite_cf` + +.. _technology: + +The ``{technology}`` wildcard +============================= + +The ``{technology}`` wildcard specifies for which renewable energy technology to produce availablity time +series and potentials using the rule :mod:`build_renewable_profiles`. +It can take the values ``onwind``, ``offwind-ac``, ``offwind-dc``, and ``solar`` but **not** ``hydro`` +(since hydroelectric plant profiles are created by a different rule. + +The wildcard can moreover be used to create technology specific figures and summaries. +For instance ``{technology}`` can be used to plot regionally disaggregated potentials +with the rule :mod:`plot_p_nom_max` or to summarize a particular technology's +full load hours in various countries with the rule :mod:`build_country_flh`. + +.. seealso:: + :mod:`build_renewable_profiles`, :mod:`plot_p_nom_max`, :mod:`build_country_flh` + +.. _attr: + +The ``{attr}`` wildcard +======================= + +The ``{attr}`` wildcard specifies which attribute are used for size +representations of network components on a map plot produced by the rule +``plot_network``. While it might be extended in the future, ``{attr}`` +currently only supports plotting of ``p_nom``. + +.. seealso:: + :mod:`plot_network` + +.. _ext: + +The ``{ext}`` wildcard +====================== + +The ``{ext}`` wildcard specifies the file type of the figures the +rule :mod:`plot_network`, :mod:`plot_summary`, and :mod:`plot_p_nom_max` produce. +Typical examples are ``pdf`` and ``png``. The list of supported file +formats depends on the used backend. To query the supported file types on your system, issue: + +.. code:: python + + import matplotlib.pyplot as plt + plt.gcf().canvas.get_supported_filetypes() + +.. seealso:: + :mod:`plot_network`, :mod:`plot_summary`, :mod:`plot_p_nom_max` diff --git a/environment.docs.yaml b/environment.docs.yaml new file mode 100644 index 00000000..aa2728e3 --- /dev/null +++ b/environment.docs.yaml @@ -0,0 +1,49 @@ +name: pypsa-eur-docs +channels: + - conda-forge + #- bioconda +dependencies: + #- python + - pip + - pypsa>=0.14 + + # Dependencies of the workflow itself + #- xlrd + - scikit-learn + - pycountry + - seaborn + #- snakemake-minimal + - memory_profiler + - yaml + + # Second order dependencies which should really be deps of atlite + - xarray + #- bottleneck + #- toolz + #- dask + - progressbar2 + + # Include ipython so that one does not inadvertently drop out of the conda + # environment by calling ipython + # - ipython + + # GIS dependencies have to come all from conda-forge + - conda-forge::cartopy + - conda-forge::fiona + - conda-forge::pyproj=1.9.5.1 + - conda-forge::pyshp + - conda-forge::geopandas + - conda-forge::rasterio + - conda-forge::shapely + - conda-forge::libgdal + + # The FRESNA/KIT stuff is not packaged for conda yet + - pip: + - vresutils>=0.2.5 + - git+https://github.com/FRESNA/atlite.git#egg=atlite + - git+https://github.com/PyPSA/glaes.git#egg=glaes + - git+https://github.com/PyPSA/geokit.git#egg=geokit + - cdsapi + - powerplantmatching + - sphinx + - sphinx_rtd_theme \ No newline at end of file diff --git a/environment.yaml b/environment.yaml index 0b1d7aee..7684a3eb 100644 --- a/environment.yaml +++ b/environment.yaml @@ -15,6 +15,7 @@ dependencies: - seaborn - snakemake-minimal - memory_profiler + - yaml # Second order dependencies which should really be deps of atlite - xarray @@ -37,10 +38,10 @@ dependencies: - conda-forge::shapely - conda-forge::libgdal - # The FRESNA/KIT stuff is not packaged for conda yet - pip: - vresutils>=0.3 - git+https://github.com/FRESNA/atlite.git#egg=atlite - git+https://github.com/PyPSA/glaes.git#egg=glaes - git+https://github.com/PyPSA/geokit.git#egg=geokit - #- git+https://github.com/FRESNA/powerplantmatching.git#egg=powerplantmatching + - cdsapi + - powerplantmatching diff --git a/img/dependencies-elec_s_128.png b/img/dependencies-elec_s_128.png deleted file mode 100644 index 996f9b73..00000000 Binary files a/img/dependencies-elec_s_128.png and /dev/null differ diff --git a/scripts/add_electricity.py b/scripts/add_electricity.py index c76aee05..7d0dc394 100644 --- a/scripts/add_electricity.py +++ b/scripts/add_electricity.py @@ -1,4 +1,91 @@ # coding: utf-8 +""" +Adds electrical generators and storage units to a base network. + +Relevant Settings +----------------- + +.. code:: yaml + + costs: + year: + USD2013_to_EUR2013: + dicountrate: + emission_prices: + + electricity: + max_hours: + marginal_cost: + capital_cost: + conventional_carriers: + co2limit: + extendable_carriers: + Generator: + StorageUnit: + estimate_renewable_capacities_from_capacity_stats: + + load: + scaling_factor: + + renewable: (keys) + hydro: + carriers: + hydro_max_hours: + hydro_capital_cost: + + lines: + length_factor: + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at :ref:`costs_cf`, :ref:`electricity_cf`, :ref:`load_cf`, :ref:`renewable_cf`, :ref:`lines_cf` + +Inputs +------ + +- ``data/costs.csv``: The database of cost assumptions for all included technologies for specific years from various sources; e.g. discount rate, lifetime, investment (CAPEX), fixed operation and maintenance (FOM), variable operation and maintenance (VOM), fuel costs, efficiency, carbon-dioxide intensity. +- ``data/bundle/hydro_capacities.csv``: Hydropower plant store/discharge power capacities, energy storage capacity, and average hourly inflow by country. + + .. image:: ../img/hydrocapacities.png + :scale: 34 % + +- ``data/geth2015_hydro_capacities.csv``: alternative to capacities above; NOT CURRENTLY USED! +- ``data/bundle/time_series_60min_singleindex_filtered.csv``: Hourly per-country load profiles since 2010 from the `ENTSO-E statistical database `_ + + .. image:: ../img/load-box.png + :scale: 33 % + + .. image:: ../img/load-ts.png + :scale: 33 % + +- ``resources/regions_onshore.geojson``: confer :ref:`busregions` +- ``resources/nuts3_shapes.geojson``: confer :ref:`shapes` +- ``resources/powerplants.csv``: confer :ref:`powerplants` +- ``resources/profile_{}.nc``: all technologies in ``config["renewables"].keys()``, confer :ref:`renewableprofiles`. +- ``networks/base.nc``: confer :ref:`base` + +Outputs +------- + +- ``networks/elec.nc``: + + .. image:: ../img/elec.png + :scale: 33 % + +Description +----------- + +The rule :mod:`add_electricity` ties all the different data inputs from the preceding rules together into a detailed PyPSA network that is stored in ``networks/elec.nc``. It includes: + +- today's transmission topology and transfer capacities (optionally including lines which are under construction according to the config settings ``lines: under_construction`` and ``links: under_construction``), +- today's thermal and hydro power generation capacities (for the technologies listed in the config setting ``electricity: conventional_carriers``), and +- today's load time-series (upsampled in a top-down approach according to population and gross domestic product) + +It further adds extendable ``generators`` and ``storage_units`` with **zero** capacity for + +- photovoltaic, onshore and AC- as well as DC-connected offshore wind installations with today's locational, hourly wind and solar capacity factors (but **no** current capacities), +- long-term hydrogen and short-term battery storage units (if listed in the config setting ``electricity: extendable_carriers``), and +- additional open- and combined-cycle gas turbines (if ``OCGT`` and/or ``CCGT`` is listed in the config setting ``electricity: extendable_carriers``) +""" import logging logger = logging.getLogger(__name__) @@ -513,7 +600,8 @@ if __name__ == "__main__": attach_conventional_generators(n, costs, ppl) attach_wind_and_solar(n, costs) - attach_hydro(n, costs, ppl) + if 'hydro' in snakemake.config['renewable']: + attach_hydro(n, costs, ppl) attach_extendable_generators(n, costs, ppl) attach_storage(n, costs) diff --git a/scripts/base_network.py b/scripts/base_network.py index f2eab220..fb91b5ed 100644 --- a/scripts/base_network.py +++ b/scripts/base_network.py @@ -1,4 +1,62 @@ # coding: utf-8 +""" +Creates the network topology from the ENTSO-E map extracts as a PyPSA network. + +Relevant Settings +----------------- + +.. code:: yaml + + snapshots: + + countries: + + electricity: + voltages: + + lines: + types: + s_max_pu: + under_construction: + + links: + p_max_pu: + under_construction: + include_tyndp: + + transformers: + x: + s_nom: + type: + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at + :ref:`snapshots_cf`, :ref:`toplevel_cf`, :ref:`electricity_cf`, :ref:`load_cf`, + :ref:`lines_cf`, :ref:`links_cf`, :ref:`transformers_cf` + +Inputs +------ + +- ``data/entsoegridkit``: Extract from the geographical vector data of the online `ENTSO-E Interactive Map `_ by the `GridKit `_ toolkit. +- ``data/parameter_corrections.yaml``: Corrections for ``data/entsoegridkit`` +- ``data/links_p_nom.csv``: confer :ref:`links` +- ``data/links_tyndp.csv``: List of projects in the `TYNDP 2018 `_ that are at least *in permitting* with fields for start- and endpoint (names and coordinates), length, capacity, construction status, and project reference ID. +- ``resources/country_shapes.geojson``: confer :ref:`shapes` +- ``resources/offshore_shapes.geojson``: confer :ref:`shapes` +- ``resources/europe_shape.geojson``: confer :ref:`shapes` + +Outputs +------- + +- ``networks/base.nc`` + + .. image:: ../img/base.png + :scale: 33 % + +Description +----------- + +""" import yaml import pandas as pd diff --git a/scripts/build_bus_regions.py b/scripts/build_bus_regions.py index 03b5b041..7ee75ab2 100644 --- a/scripts/build_bus_regions.py +++ b/scripts/build_bus_regions.py @@ -1,3 +1,42 @@ +""" +Creates Voronoi shapes for each bus representing both onshore and offshore regions. + +Relevant Settings +----------------- + +.. code:: yaml + + countries: + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at + :ref:`toplevel_cf` + +Inputs +------ + +- ``resources/country_shapes.geojson``: confer :ref:`shapes` +- ``resources/offshore_shapes.geojson``: confer :ref:`shapes` +- ``networks/base.nc``: confer :ref:`base` + +Outputs +------- + +- ``resources/regions_onshore.geojson``: + + .. image:: ../img/regions_onshore.png + :scale: 33 % + +- ``resources/regions_offshore.geojson``: + + .. image:: ../img/regions_offshore.png + :scale: 33 % + +Description +----------- + +""" + import os from operator import attrgetter @@ -7,49 +46,53 @@ import geopandas as gpd from vresutils.graph import voronoi_partition_pts import pypsa +import logging -countries = snakemake.config['countries'] +if __name__ == "__main__": + logging.basicConfig(level=snakemake.config["logging_level"]) -n = pypsa.Network(snakemake.input.base_network) + countries = snakemake.config['countries'] -country_shapes = gpd.read_file(snakemake.input.country_shapes).set_index('name')['geometry'] -offshore_shapes = gpd.read_file(snakemake.input.offshore_shapes).set_index('name')['geometry'] + n = pypsa.Network(snakemake.input.base_network) -onshore_regions = [] -offshore_regions = [] + country_shapes = gpd.read_file(snakemake.input.country_shapes).set_index('name')['geometry'] + offshore_shapes = gpd.read_file(snakemake.input.offshore_shapes).set_index('name')['geometry'] -for country in countries: - c_b = n.buses.country == country + onshore_regions = [] + offshore_regions = [] - onshore_shape = country_shapes[country] - onshore_locs = n.buses.loc[c_b & n.buses.substation_lv, ["x", "y"]] - onshore_regions.append(gpd.GeoDataFrame({ - 'name': onshore_locs.index, - 'x': onshore_locs['x'], - 'y': onshore_locs['y'], - 'geometry': voronoi_partition_pts(onshore_locs.values, onshore_shape), - 'country': country - })) + for country in countries: + c_b = n.buses.country == country - if country not in offshore_shapes.index: continue - offshore_shape = offshore_shapes[country] - offshore_locs = n.buses.loc[c_b & n.buses.substation_off, ["x", "y"]] - offshore_regions_c = gpd.GeoDataFrame({ - 'name': offshore_locs.index, - 'x': offshore_locs['x'], - 'y': offshore_locs['y'], - 'geometry': voronoi_partition_pts(offshore_locs.values, offshore_shape), - 'country': country - }, index=offshore_locs.index) - offshore_regions_c = offshore_regions_c.loc[offshore_regions_c.area > 1e-2] - offshore_regions.append(offshore_regions_c) + onshore_shape = country_shapes[country] + onshore_locs = n.buses.loc[c_b & n.buses.substation_lv, ["x", "y"]] + onshore_regions.append(gpd.GeoDataFrame({ + 'name': onshore_locs.index, + 'x': onshore_locs['x'], + 'y': onshore_locs['y'], + 'geometry': voronoi_partition_pts(onshore_locs.values, onshore_shape), + 'country': country + })) -def save_to_geojson(s, fn): - if os.path.exists(fn): - os.unlink(fn) - schema = {**gpd.io.file.infer_schema(s), 'geometry': 'Unknown'} - s.to_file(fn, driver='GeoJSON', schema=schema) + if country not in offshore_shapes.index: continue + offshore_shape = offshore_shapes[country] + offshore_locs = n.buses.loc[c_b & n.buses.substation_off, ["x", "y"]] + offshore_regions_c = gpd.GeoDataFrame({ + 'name': offshore_locs.index, + 'x': offshore_locs['x'], + 'y': offshore_locs['y'], + 'geometry': voronoi_partition_pts(offshore_locs.values, offshore_shape), + 'country': country + }, index=offshore_locs.index) + offshore_regions_c = offshore_regions_c.loc[offshore_regions_c.area > 1e-2] + offshore_regions.append(offshore_regions_c) -save_to_geojson(pd.concat(onshore_regions), snakemake.output.regions_onshore) + def save_to_geojson(s, fn): + if os.path.exists(fn): + os.unlink(fn) + schema = {**gpd.io.file.infer_schema(s), 'geometry': 'Unknown'} + s.to_file(fn, driver='GeoJSON', schema=schema) -save_to_geojson(pd.concat(offshore_regions), snakemake.output.regions_offshore) + save_to_geojson(pd.concat(onshore_regions), snakemake.output.regions_onshore) + + save_to_geojson(pd.concat(offshore_regions), snakemake.output.regions_offshore) diff --git a/scripts/build_country_flh.py b/scripts/build_country_flh.py index 3a2de295..ceae4848 100644 --- a/scripts/build_country_flh.py +++ b/scripts/build_country_flh.py @@ -1,4 +1,61 @@ #!/usr/bin/env python +""" +Create ``.csv`` files and plots for comparing per country full load hours of renewable time series. + +Relevant Settings +----------------- + +.. code:: yaml + + snapshots: + + renewable: + {technology}: + cutout: + resource: + correction_factor: + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at + :ref:`snapshots_cf`, :ref:`renewable_cf` + +Inputs +------ + +- ``data/bundle/corine/g250_clc06_V18_5.tif``: `CORINE Land Cover (CLC) `_ inventory on `44 classes `_ of land use (e.g. forests, arable land, industrial, urban areas). + + .. image:: img/corine.png + :scale: 33 % + +- ``data/bundle/GEBCO_2014_2D.nc``: A `bathymetric `_ data set with a global terrain model for ocean and land at 15 arc-second intervals by the `General Bathymetric Chart of the Oceans (GEBCO) `_. + + .. image:: img/gebco_2019_grid_image.jpg + :scale: 50 % + + **Source:** `GEBCO `_ + +- ``data/pietzker2014.xlsx``: `Supplementary material 2 `_ from `Pietzcker et al. `_; not part of the data bundle; download and place here yourself. +- ``resources/natura.tiff``: confer :ref:`natura` +- ``resources/country_shapes.geojson``: confer :ref:`shapes` +- ``resources/offshore_shapes.geojson``: confer :ref:`shapes` +- ``resources/regions_onshore.geojson``: (if not offshore wind), confer :ref:`busregions` +- ``resources/regions_offshore.geojson``: (if offshore wind), :ref:`busregions` +- ``"cutouts/" + config["renewable"][{technology}]['cutout']``: :ref:`cutout` +- ``networks/base.nc``: :ref:`base` + +Outputs +------- + +- ``resources/country_flh_area_{technology}.csv``: +- ``resources/country_flh_aggregated_{technology}.csv``: +- ``resources/country_flh_uncorrected_{technology}.csv``: +- ``resources/country_flh_{technology}.pdf``: +- ``resources/country_exclusion_{technology}``: + +Description +----------- + +""" import os import atlite diff --git a/scripts/build_cutout.py b/scripts/build_cutout.py index f64db0e6..02a69df4 100644 --- a/scripts/build_cutout.py +++ b/scripts/build_cutout.py @@ -1,17 +1,106 @@ +""" +Create cutouts with `atlite `_. + +For this rule to work you must have + +- installed the `Copernicus Climate Data Store `_ ``cdsapi`` package (`install with `pip``) and +- registered and setup your CDS API key as described `on their website `_. + +.. seealso:: + For details on the weather data read the `atlite documentation `_. + If you need help specifically for creating cutouts `the corresponding section in the atlite documentation `_ should be helpful. + +Relevant Settings +----------------- + +.. code:: yaml + + atlite: + nprocesses: + cutouts: + {cutout}: + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at + :ref:`atlite_cf` + +Inputs +------ + +*None* + +Outputs +------- + +- ``cutouts/{cutout}``: weather data from either the `ERA5 `_ + reanalysis weather dataset or `SARAH-2 `_ + satellite-based historic weather data with the following structure: + +**ERA5 cutout:** + + =================== ========== ========== ========================================================= + Field Dimensions Unit Description + =================== ========== ========== ========================================================= + pressure time, y, x Pa Surface pressure + ------------------- ---------- ---------- --------------------------------------------------------- + temperature time, y, x K Air temperature 2 meters above the surface. + ------------------- ---------- ---------- --------------------------------------------------------- + soil temperature time, y, x K Soil temperature between 1 meters and 3 meters + depth (layer 4). + ------------------- ---------- ---------- --------------------------------------------------------- + influx_toa time, y, x Wm**-2 Top of Earth's atmosphere TOA incident solar radiation + ------------------- ---------- ---------- --------------------------------------------------------- + influx_direct time, y, x Wm**-2 Total sky direct solar radiation at surface + ------------------- ---------- ---------- --------------------------------------------------------- + runoff time, y, x m `Runoff `_ + (volume per area) + ------------------- ---------- ---------- --------------------------------------------------------- + roughness y, x m Forecast surface roughness + (`roughness length `_) + ------------------- ---------- ---------- --------------------------------------------------------- + height y, x m Surface elevation above sea level + ------------------- ---------- ---------- --------------------------------------------------------- + albedo time, y, x -- `Albedo `_ + measure of diffuse reflection of solar radiation. + Calculated from relation between surface solar radiation + downwards (Jm**-2) and surface net solar radiation + (Jm**-2). Takes values between 0 and 1. + ------------------- ---------- ---------- --------------------------------------------------------- + influx_diffuse time, y, x Wm**-2 Diffuse solar radiation at surface. + Surface solar radiation downwards minus + direct solar radiation. + ------------------- ---------- ---------- --------------------------------------------------------- + wnd100m time, y, x ms**-1 Wind speeds at 100 meters (regardless of direction) + =================== ========== ========== ========================================================= + + .. image:: ../img/era5.png + :scale: 40 % + +A **SARAH-2 cutout** can be used to amend the fields ``temperature``, ``influx_toa``, ``influx_direct``, ``albedo``, +``influx_diffuse`` of ERA5 using satellite-based radiation observations. + + .. image:: ../img/sarah.png + :scale: 40 % + +Description +----------- + +""" import os import atlite import logging logger = logging.getLogger(__name__) -logging.basicConfig(level=snakemake.config['logging_level']) +if __name__ == "__main__": + logging.basicConfig(level=snakemake.config['logging_level']) -cutout_params = snakemake.config['atlite']['cutouts'][snakemake.wildcards.cutout] -for p in ('xs', 'ys', 'years', 'months'): - if p in cutout_params: - cutout_params[p] = slice(*cutout_params[p]) + cutout_params = snakemake.config['atlite']['cutouts'][snakemake.wildcards.cutout] + for p in ('xs', 'ys', 'years', 'months'): + if p in cutout_params: + cutout_params[p] = slice(*cutout_params[p]) -cutout = atlite.Cutout(snakemake.wildcards.cutout, - cutout_dir=os.path.dirname(snakemake.output[0]), - **cutout_params) + cutout = atlite.Cutout(snakemake.wildcards.cutout, + cutout_dir=os.path.dirname(snakemake.output[0]), + **cutout_params) -cutout.prepare(nprocesses=snakemake.config['atlite'].get('nprocesses', 4)) + cutout.prepare(nprocesses=snakemake.config['atlite'].get('nprocesses', 4)) diff --git a/scripts/build_hydro_profile.py b/scripts/build_hydro_profile.py index d7e9797e..d0bc5665 100644 --- a/scripts/build_hydro_profile.py +++ b/scripts/build_hydro_profile.py @@ -1,4 +1,58 @@ #!/usr/bin/env python +""" +Build hydroelectric inflow time-series for each country. + +Relevant Settings +----------------- + +.. code:: yaml + + countries: + + renewable: + hydro: + cutout: + clip_min_inflow: + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at + :ref:`toplevel_cf`, :ref:`renewable_cf` + +Inputs +------ + +- ``data/bundle/EIA_hydro_generation_2000_2014.csv``: Hydroelectricity net generation per country and year (`EIA `_) + + .. image:: ../img/hydrogeneration.png + :scale: 33 % + +- ``resources/country_shapes.geojson``: confer :ref:`shapes` +- ``"cutouts/" + config["renewable"]['hydro']['cutout']``: confer :ref:`cutout` + +Outputs +------- + +- ``resources/profile_hydro.nc``: + + =================== ================ ========================================================= + Field Dimensions Description + =================== ================ ========================================================= + inflow countries, time Inflow to the state of charge (in MW), + e.g. due to river inflow in hydro reservoir. + =================== ================ ========================================================= + + .. image:: ../img/inflow-ts.png + :scale: 33 % + + .. image:: ../img/inflow-box.png + :scale: 33 % + +Description +----------- + +.. seealso:: + :mod:`build_renewable_profiles` +""" import os import atlite @@ -6,24 +60,26 @@ import pandas as pd import geopandas as gpd from vresutils import hydro as vhydro import logging -logger = logging.getLogger(__name__) -logger.setLevel(level=snakemake.config['logging_level']) -config = snakemake.config['renewable']['hydro'] -cutout = atlite.Cutout(config['cutout'], - cutout_dir=os.path.dirname(snakemake.input.cutout)) -countries = snakemake.config['countries'] -country_shapes = gpd.read_file(snakemake.input.country_shapes).set_index('name')['geometry'].reindex(countries) -country_shapes.index.name = 'countries' +if __name__ == "__main__": + logger.basicConfig(level=snakemake.config['logging_level']) -eia_stats = vhydro.get_eia_annual_hydro_generation(snakemake.input.eia_hydro_generation).reindex(columns=countries) -inflow = cutout.runoff(shapes=country_shapes, - smooth=True, - lower_threshold_quantile=True, - normalize_using_yearly=eia_stats) + config = snakemake.config['renewable']['hydro'] + cutout = atlite.Cutout(config['cutout'], + cutout_dir=os.path.dirname(snakemake.input.cutout)) -if 'clip_min_inflow' in config: - inflow.values[inflow.values < config['clip_min_inflow']] = 0. + countries = snakemake.config['countries'] + country_shapes = gpd.read_file(snakemake.input.country_shapes).set_index('name')['geometry'].reindex(countries) + country_shapes.index.name = 'countries' -inflow.to_netcdf(snakemake.output[0]) + eia_stats = vhydro.get_eia_annual_hydro_generation(snakemake.input.eia_hydro_generation).reindex(columns=countries) + inflow = cutout.runoff(shapes=country_shapes, + smooth=True, + lower_threshold_quantile=True, + normalize_using_yearly=eia_stats) + + if 'clip_min_inflow' in config: + inflow.values[inflow.values < config['clip_min_inflow']] = 0. + + inflow.to_netcdf(snakemake.output[0]) diff --git a/scripts/build_natura_raster.py b/scripts/build_natura_raster.py index d0ad7cdf..78c7e804 100644 --- a/scripts/build_natura_raster.py +++ b/scripts/build_natura_raster.py @@ -1,3 +1,40 @@ +""" +Rasters the vector data of the `Natura 2000 `_ natural protection areas onto all cutout regions. + +Relevant Settings +----------------- + +.. code:: yaml + + renewable: + {technology}: + cutout: + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at + :ref:`renewable_cf` + +Inputs +------ + +- ``data/bundle/natura/Natura2000_end2015.shp``: `Natura 2000 `_ natural protection areas. + + .. image:: ../img/natura.png + :scale: 33 % + +Outputs +------- + +- ``resources/natura.tiff``: Rasterized version of `Natura 2000 `_ natural protection areas to reduce computation times. + + .. image:: ../img/natura.png + :scale: 33 % + +Description +----------- + +""" + import numpy as np import atlite from osgeo import gdal @@ -10,10 +47,11 @@ def determine_cutout_xXyY(cutout_name): dy = (Y - y) / (cutout.shape[0] - 1) return [x - dx/2., X + dx/2., y - dy/2., Y + dy/2.] -cutout_names = np.unique([res['cutout'] for res in snakemake.config['renewable'].values()]) -xs, Xs, ys, Ys = zip(*(determine_cutout_xXyY(cutout) for cutout in cutout_names)) -xXyY = min(xs), max(Xs), min(ys), max(Ys) +if __name__ == "__main__": + cutout_names = np.unique([res['cutout'] for res in snakemake.config['renewable'].values()]) + xs, Xs, ys, Ys = zip(*(determine_cutout_xXyY(cutout) for cutout in cutout_names)) + xXyY = min(xs), max(Xs), min(ys), max(Ys) -natura = gk.vector.loadVector(snakemake.input[0]) -extent = gk.Extent.from_xXyY(xXyY).castTo(3035).fit(100) -extent.rasterize(natura, pixelWidth=100, pixelHeight=100, output=snakemake.output[0]) + natura = gk.vector.loadVector(snakemake.input[0]) + extent = gk.Extent.from_xXyY(xXyY).castTo(3035).fit(100) + extent.rasterize(natura, pixelWidth=100, pixelHeight=100, output=snakemake.output[0]) diff --git a/scripts/build_powerplants.py b/scripts/build_powerplants.py index bf5a8a1a..8fb193c5 100644 --- a/scripts/build_powerplants.py +++ b/scripts/build_powerplants.py @@ -1,4 +1,38 @@ # coding: utf-8 +""" +Retrieves conventional powerplant capacities and locations from `powerplantmatching `_, assigns these to buses and creates a ``.csv`` file. + +Relevant Settings +----------------- + +.. code:: yaml + + enable: + powerplantmatching: + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at + :ref:`toplevel_cf` + +Inputs +------ + +- ``networks/base.nc``: confer :ref:`base`. + +Outputs +------- + +- ``resource/powerplants.csv``: A list of conventional power plants (i.e. neither wind nor solar) with fields for name, fuel type, technology, country, capacity in MW, duration, commissioning year, retrofit year, latitude, longitude, and dam information as documented in the `powerplantmatching README `_; additionally it includes information on the closest substation/bus in ``networks/base.nc``. + + .. image:: ../img/powerplantmatching.png + :scale: 30 % + + **Source:** `powerplantmatching on GitHub `_ + +Description +----------- + +""" import logging import numpy as np diff --git a/scripts/build_renewable_profiles.py b/scripts/build_renewable_profiles.py index 4368c6ab..dc1252ae 100644 --- a/scripts/build_renewable_profiles.py +++ b/scripts/build_renewable_profiles.py @@ -1,4 +1,155 @@ #!/usr/bin/env python +"""Calculates for each network node the +(i) installable capacity (based on land-use), (ii) the available generation time +series (based on weather data), and (iii) the average distance from the node for +onshore wind, AC-connected offshore wind, DC-connected offshore wind and solar +PV generators. In addition for offshore wind it calculates the fraction of the +grid connection which is under water. + +.. note:: Hydroelectric profiles are built in script :mod:`build_hydro_profiles`. + +Relevant settings +----------------- + +.. code:: yaml + + snapshots: + + atlite: + nprocesses: + + renewable: + {technology}: + cutout: + corine: + grid_codes: + distance: + natura: + max_depth: + max_shore_distance: + min_shore_distance: + capacity_per_sqkm: + correction_factor: + potential: + min_p_max_pu: + clip_p_max_pu: + resource: + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at + :ref:`snapshots_cf`, :ref:`atlite_cf`, :ref:`renewable_cf` + +Inputs +------ + +- ``data/bundle/corine/g250_clc06_V18_5.tif``: `CORINE Land Cover (CLC) `_ inventory on `44 classes `_ of land use (e.g. forests, arable land, industrial, urban areas). + + .. image:: ../img/corine.png + :scale: 33 % + +- ``data/bundle/GEBCO_2014_2D.nc``: A `bathymetric `_ data set with a global terrain model for ocean and land at 15 arc-second intervals by the `General Bathymetric Chart of the Oceans (GEBCO) `_. + + .. image:: ../img/gebco_2019_grid_image.jpg + :scale: 50 % + + **Source:** `GEBCO `_ + +- ``resources/natura.tiff``: confer :ref:`natura` +- ``resources/country_shapes.geojson``: confer :ref:`shapes` +- ``resources/offshore_shapes.geojson``: confer :ref:`shapes` +- ``resources/regions_onshore.geojson``: (if not offshore wind), confer :ref:`busregions` +- ``resources/regions_offshore.geojson``: (if offshore wind), :ref:`busregions` +- ``"cutouts/" + config["renewable"][{technology}]['cutout']``: :ref:`cutout` +- ``networks/base.nc``: :ref:`base` + +Outputs +------- + +- ``resources/profile_{technology}.nc`` with the following structure + + =================== ========== ========================================================= + Field Dimensions Description + =================== ========== ========================================================= + profile bus, time the per unit hourly availability factors for each node + ------------------- ---------- --------------------------------------------------------- + weight bus sum of the layout weighting for each node + ------------------- ---------- --------------------------------------------------------- + p_nom_max bus maximal installable capacity at the node (in MW) + ------------------- ---------- --------------------------------------------------------- + potential y, x layout of generator units at cutout grid cells inside the + Voronoi cell (maximal installable capacity at each grid + cell multiplied by capacity factor) + ------------------- ---------- --------------------------------------------------------- + average_distance bus average distance of units in the Voronoi cell to the + grid node (in km) + ------------------- ---------- --------------------------------------------------------- + underwater_fraction bus fraction of the average connection distance which is + under water (only for offshore) + =================== ========== ========================================================= + + - **profile** + + .. image:: ../img/profile_ts.png + :scale: 33 % + + - **p_nom_max** + + .. image:: ../img/p_nom_max_hist.png + :scale: 33 % + + - **potential** + + .. image:: ../img/potential_heatmap.png + :scale: 33 % + + - **average_distance** + + .. image:: ../img/distance_hist.png + :scale: 33 % + + - **underwater_fraction** + + .. image:: ../img/underwater_hist.png + :scale: 33 % + +Description +----------- + +This script functions at two main spatial resolutions: the resolution of the +network nodes and their `Voronoi cells +`_, and the resolution of the +cutout grid cells for the weather data. Typically the weather data grid is +finer than the network nodes, so we have to work out the distribution of +generators across the grid cells within each Voronoi cell. This is done by +taking account of a combination of the available land at each grid cell and the +capacity factor there. + +First the script computes how much of the technology can be installed at each +cutout grid cell and each node using the `GLAES +`_ library. This uses the CORINE land use data, +Natura2000 nature reserves and GEBCO bathymetry data. + +To compute the layout of generators in each node's Voronoi cell, the +installable potential in each grid cell is multiplied with the capacity factor +at each grid cell. This is done since we assume more generators are installed +at cells with a higher capacity factor. + +This layout is then used to compute the generation availability time series +from the weather data cutout from ``atlite``. + +Two methods are available to compute the maximal installable potential for the +node (`p_nom_max`): ``simple`` and ``conservative``: + +- ``simple`` adds up the installable potentials of the individual grid cells. + If the model comes close to this limit, then the time series may slightly + overestimate production since it is assumed the geographical distribution is + proportional to capacity factor. + +- ``conservative`` assertains the nodal limit by increasing capacities + proportional to the layout until the limit of an individual grid cell is + reached. + +""" import matplotlib.pyplot as plt @@ -33,8 +184,9 @@ def init_globals(bounds_xXyY, n_dx, n_dy, n_config, n_paths): config = n_config paths = n_paths - gebco = gk.raster.loadRaster(paths["gebco"]) - gebco.SetProjection(gk.srs.loadSRS(4326).ExportToWkt()) + if "max_depth" in config: + gebco = gk.raster.loadRaster(paths["gebco"]) + gebco.SetProjection(gk.srs.loadSRS(4326).ExportToWkt()) clc = gk.raster.loadRaster(paths["corine"]) clc.SetProjection(gk.srs.loadSRS(3035).ExportToWkt()) diff --git a/scripts/build_shapes.py b/scripts/build_shapes.py index e3bc0a18..972706cf 100644 --- a/scripts/build_shapes.py +++ b/scripts/build_shapes.py @@ -1,3 +1,68 @@ +""" +Creates GIS shape files of the countries, exclusive economic zones and `NUTS3 `_ areas. + +Relevant Settings +----------------- + +.. code:: yaml + + countries: + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at + :ref:`toplevel_cf` + +Inputs +------ + +- ``data/bundle/naturalearth/ne_10m_admin_0_countries.shp``: World country shapes + + .. image:: ../img/countries.png + :scale: 33 % + +- ``data/bundle/eez/World_EEZ_v8_2014.shp``: World `exclusive economic zones `_ (EEZ) + + .. image:: ../img/eez.png + :scale: 33 % + +- ``data/bundle/NUTS_2013_60M_SH/data/NUTS_RG_60M_2013.shp``: Europe NUTS3 regions + + .. image:: ../img/nuts3.png + :scale: 33 % + +- ``data/bundle/nama_10r_3popgdp.tsv.gz``: Average annual population by NUTS3 region (`eurostat `_) +- ``data/bundle/nama_10r_3gdp.tsv.gz``: Gross domestic product (GDP) by NUTS 3 regions (`eurostat `_) +- ``data/bundle/ch_cantons.csv``: Mapping between Swiss Cantons and NUTS3 regions +- ``data/bundle/je-e-21.03.02.xls``: Population and GDP data per Canton (`BFS - Swiss Federal Statistical Office `_ ) + +Outputs +------- + +- ``resources/country_shapes.geojson``: country shapes out of country selection + + .. image:: ../img/country_shapes.png + :scale: 33 % + +- ``resources/offshore_shapes.geojson``: EEZ shapes out of country selection + + .. image:: ../img/offshore_shapes.png + :scale: 33 % + +- ``resources/europe_shape.geojson``: Shape of Europe including countries and EEZ + + .. image:: ../img/europe_shape.png + :scale: 33 % + +- ``resources/nuts3_shapes.geojson``: NUTS3 shapes out of country selection including population and GDP data. + + .. image:: ../img/nuts3_shapes.png + :scale: 33 % + +Description +----------- + +""" + import os import numpy as np from operator import attrgetter diff --git a/scripts/cluster_network.py b/scripts/cluster_network.py index f1ffdfc4..a43db0c0 100644 --- a/scripts/cluster_network.py +++ b/scripts/cluster_network.py @@ -1,4 +1,93 @@ # coding: utf-8 +""" +Creates networks clustered to ``{cluster}`` number of zones with aggregated buses, generators and transmission corridors. + +Relevant Settings +----------------- + +.. code:: yaml + + renewable: (keys) + {technology}: + potential: + + solving: + solver: + name: + + lines: + length_factor: + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at + :ref:`renewable_cf`, :ref:`solving_cf`, :ref:`lines_cf` + +Inputs +------ + +- ``resources/regions_onshore_{network}_s{simpl}.geojson``: confer :ref:`simplify` +- ``resources/regions_offshore_{network}_s{simpl}.geojson``: confer :ref:`simplify` +- ``resources/clustermaps_{network}_s{simpl}.h5``: confer :ref:`simplify` +- ``networks/{network}_s{simpl}.nc``: confer :ref:`simplify` + +Outputs +------- + +- ``resources/regions_onshore_{network}_s{simpl}_{clusters}.geojson``: + + .. image:: ../img/regions_onshore_elec_s_X.png + :scale: 33 % + +- ``resources/regions_offshore_{network}_s{simpl}_{clusters}.geojson``: + + .. image:: ../img/regions_offshore_elec_s_X.png + :scale: 33 % + +- ``resources/clustermaps_{network}_s{simpl}_{clusters}.h5``: Mapping of buses and lines from ``networks/elec_s{simpl}.nc`` to ``networks/elec_s{simpl}_{clusters}.nc``; has keys ['/busmap', '/busmap_s', '/linemap', '/linemap_negative', '/linemap_positive'] +- ``networks/{network}_s{simpl}_{clusters}.nc``: + + .. image:: ../img/elec_s_X.png + :scale: 40 % + +Description +----------- + +.. note:: + + **Why is clustering used both in** ``simplify_network`` **and** ``cluster_network`` **?** + + Consider for example a network ``networks/elec_s100_50.nc`` in which + ``simplify_network`` clusters the network to 100 buses and in a second + step ``cluster_network``` reduces it down to 50 buses. + + In preliminary tests, it turns out, that the principal effect of + changing spatial resolution is actually only partially due to the + transmission network. It is more important to differentiate between + wind generators with higher capacity factors from those with lower + capacity factors, i.e. to have a higher spatial resolution in the + renewable generation than in the number of buses. + + The two-step clustering allows to study this effect by looking at + networks like ``networks/elec_s100_50m.nc``. Note the additional + ``m`` in the ``{cluster}`` wildcard. So in the example network + there are still up to 100 different wind generators. + + In combination these two features allow you to study the spatial + resolution of the transmission network separately from the + spatial resolution of renewable generators. + + **Is it possible to run the model without the** ``simplify_network`` **rule?** + + No, the network clustering methods in the PyPSA module + `pypsa.networkclustering `_ + do not work reliably with multiple voltage levels and transformers. + +.. tip:: + The rule :mod:`cluster_all_networks` runs + for all ``scenario`` s in the configuration file + the rule :mod:`cluster_network`. + +""" import pandas as pd idx = pd.IndexSlice @@ -62,7 +151,7 @@ def plot_weighting(n, country, country_shape=None): def distribute_clusters(n, n_clusters, solver_name=None): if solver_name is None: - solver_name = snakemake.config['solver']['solver']['name'] + solver_name = snakemake.config['solving']['solver']['name'] L = (n.loads_t.p_set.mean() .groupby(n.loads.bus).sum() @@ -243,5 +332,3 @@ if __name__ == "__main__": store.put(attr, getattr(clustering, attr), format="table", index=False) cluster_regions((clustering.busmap,)) - - diff --git a/scripts/make_summary.py b/scripts/make_summary.py index 8b97def3..150c159e 100644 --- a/scripts/make_summary.py +++ b/scripts/make_summary.py @@ -1,3 +1,54 @@ +""" +Creates summaries of aggregated energy and costs as ``.csv`` files. + +Relevant Settings +----------------- + +.. code:: yaml + + costs: + USD2013_to_EUR2013: + discountrate: + marginal_cost: + capital_cost: + + electricity: + max_hours: + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at + :ref:`costs_cf`, :ref:`electricity_cf` + +Inputs +------ + +Outputs +------- + +Description +----------- + +The following rule can be used to summarize the results in seperate .csv files: + +.. code:: + + snakemake results/summaries/elec_s_all_lall_Co2L-3H_all + clusters + line volume or cost cap + - options + - all countries + +the line volume/cost cap field can be set to one of the following: +* ``lv1.25`` for a particular line volume extension by 25% +* ``lc1.25`` for a line cost extension by 25 % +* ``lall`` for all evalutated caps +* ``lvall`` for all line volume caps +* ``lcall`` for all line cost caps + +Replacing '/summaries/' with '/plots/' creates nice colored maps of the results. + +""" + import os from six import iteritems from itertools import product diff --git a/scripts/plot_network.py b/scripts/plot_network.py index 4aa204fe..4c666f55 100644 --- a/scripts/plot_network.py +++ b/scripts/plot_network.py @@ -1,16 +1,19 @@ -if 'snakemake' not in globals(): - from vresutils.snakemake import MockSnakemake, Dict - from snakemake.rules import expand - import yaml - snakemake = Dict() - snakemake = MockSnakemake( - path='..', - wildcards=dict(network='elec', simpl='', clusters='90', lv='1.25', opts='Co2L-3H', attr='p_nom', ext="pdf"), - input=dict(network="results/networks/{network}_s{simpl}_{clusters}_lv{lv}_{opts}.nc", - tech_costs="data/costs.csv"), - output=dict(only_map="results/plots/{network}_s{simpl}_{clusters}_lv{lv}_{opts}_{attr}.{ext}", - ext="results/plots/{network}_s{simpl}_{clusters}_lv{lv}_{opts}_{attr}_ext.{ext}") - ) +""" +Plots map with pie charts and cost box bar charts. + +Relevant Settings +----------------- + +Inputs +------ + +Outputs +------- + +Description +----------- + +""" import pypsa @@ -26,7 +29,9 @@ from itertools import product, chain from six.moves import map, zip from six import itervalues, iterkeys from collections import OrderedDict as odict +import logging +import cartopy.crs as ccrs import matplotlib.pyplot as plt import matplotlib as mpl from matplotlib.patches import Circle, Ellipse @@ -59,203 +64,242 @@ def make_handler_map_to_scale_circles_as_in(ax, dont_resize_actively=False): def make_legend_circles_for(sizes, scale=1.0, **kw): return [Circle((0,0), radius=(s/scale)**0.5, **kw) for s in sizes] -plt.style.use(['classic', 'seaborn-white', - {'axes.grid': False, 'grid.linestyle': '--', 'grid.color': u'0.6', - 'hatch.color': 'white', - 'patch.linewidth': 0.5, - 'font.size': 12, - 'legend.fontsize': 'medium', - 'lines.linewidth': 1.5, - 'pdf.fonttype': 42, - # 'font.family': 'Times New Roman' - }]) +def set_plot_style(): + plt.style.use(['classic', 'seaborn-white', + {'axes.grid': False, 'grid.linestyle': '--', 'grid.color': u'0.6', + 'hatch.color': 'white', + 'patch.linewidth': 0.5, + 'font.size': 12, + 'legend.fontsize': 'medium', + 'lines.linewidth': 1.5, + 'pdf.fonttype': 42, + # 'font.family': 'Times New Roman' + }]) -opts = snakemake.config['plotting'] -map_figsize = opts['map']['figsize'] -map_boundaries = opts['map']['boundaries'] +def plot_map(n, ax=None, attribute='p_nom', opts={}): + if ax is None: + ax = plt.gca() -n = load_network(snakemake.input.network, snakemake.input.tech_costs, snakemake.config) + ## DATA + line_colors = {'cur': "purple", + 'exp': to_rgba("red", 0.7)} + tech_colors = opts['tech_colors'] -scenario_opts = snakemake.wildcards.opts.split('-') - -## DATA -line_colors = {'cur': "purple", - 'exp': to_rgba("red", 0.7)} -tech_colors = opts['tech_colors'] - -if snakemake.wildcards.attr == 'p_nom': - # bus_sizes = n.generators_t.p.sum().loc[n.generators.carrier == "load"].groupby(n.generators.bus).sum() - bus_sizes = pd.concat((n.generators.query('carrier != "load"').groupby(['bus', 'carrier']).p_nom_opt.sum(), - n.storage_units.groupby(['bus', 'carrier']).p_nom_opt.sum())) - line_widths_exp = dict(Line=n.lines.s_nom_opt, Link=n.links.p_nom_opt) - line_widths_cur = dict(Line=n.lines.s_nom_min, Link=n.links.p_nom_min) -else: - raise 'plotting of {} has not been implemented yet'.format(plot) + if attribute == 'p_nom': + # bus_sizes = n.generators_t.p.sum().loc[n.generators.carrier == "load"].groupby(n.generators.bus).sum() + bus_sizes = pd.concat((n.generators.query('carrier != "load"').groupby(['bus', 'carrier']).p_nom_opt.sum(), + n.storage_units.groupby(['bus', 'carrier']).p_nom_opt.sum())) + line_widths_exp = dict(Line=n.lines.s_nom_opt, Link=n.links.p_nom_opt) + line_widths_cur = dict(Line=n.lines.s_nom_min, Link=n.links.p_nom_min) + else: + raise 'plotting of {} has not been implemented yet'.format(plot) -line_colors_with_alpha = \ -dict(Line=(line_widths_cur['Line'] / n.lines.s_nom > 1e-3) - .map({True: line_colors['cur'], False: to_rgba(line_colors['cur'], 0.)}), - Link=(line_widths_cur['Link'] / n.links.p_nom > 1e-3) - .map({True: line_colors['cur'], False: to_rgba(line_colors['cur'], 0.)})) + line_colors_with_alpha = \ + dict(Line=(line_widths_cur['Line'] / n.lines.s_nom > 1e-3) + .map({True: line_colors['cur'], False: to_rgba(line_colors['cur'], 0.)}), + Link=(line_widths_cur['Link'] / n.links.p_nom > 1e-3) + .map({True: line_colors['cur'], False: to_rgba(line_colors['cur'], 0.)})) -## FORMAT -linewidth_factor = opts['map'][snakemake.wildcards.attr]['linewidth_factor'] -bus_size_factor = opts['map'][snakemake.wildcards.attr]['bus_size_factor'] + ## FORMAT + linewidth_factor = opts['map'][attribute]['linewidth_factor'] + bus_size_factor = opts['map'][attribute]['bus_size_factor'] -## PLOT -fig, ax = plt.subplots(figsize=map_figsize) -n.plot(line_widths=pd.concat(line_widths_exp)/linewidth_factor, - line_colors=dict(Line=line_colors['exp'], Link=line_colors['exp']), - bus_sizes=bus_sizes/bus_size_factor, - bus_colors=tech_colors, - boundaries=map_boundaries, - basemap=True, - ax=ax) -n.plot(line_widths=pd.concat(line_widths_cur)/linewidth_factor, - line_colors=pd.concat(line_colors_with_alpha), - bus_sizes=0, - bus_colors=tech_colors, - boundaries=map_boundaries, - basemap=False, - ax=ax) -ax.set_aspect('equal') -ax.axis('off') + ## PLOT + n.plot(line_widths=pd.concat(line_widths_exp)/linewidth_factor, + line_colors=dict(Line=line_colors['exp'], Link=line_colors['exp']), + bus_sizes=bus_sizes/bus_size_factor, + bus_colors=tech_colors, + boundaries=map_boundaries, + geomap=True, + ax=ax) + n.plot(line_widths=pd.concat(line_widths_cur)/linewidth_factor, + line_colors=pd.concat(line_colors_with_alpha), + bus_sizes=0, + bus_colors=tech_colors, + boundaries=map_boundaries, + geomap=True, # TODO : Turn to False, after the release of PyPSA 0.14.2 (refer to https://github.com/PyPSA/PyPSA/issues/75) + ax=ax) + ax.set_aspect('equal') + ax.axis('off') -# x1, y1, x2, y2 = map_boundaries -# ax.set_xlim(x1, x2) -# ax.set_ylim(y1, y2) + # x1, y1, x2, y2 = map_boundaries + # ax.set_xlim(x1, x2) + # ax.set_ylim(y1, y2) -# Rasterize basemap -for c in ax.collections[:2]: c.set_rasterized(True) + # Rasterize basemap + # TODO : Check if this also works with cartopy + for c in ax.collections[:2]: c.set_rasterized(True) -# LEGEND -handles = [] -labels = [] + # LEGEND + handles = [] + labels = [] -for s in (10, 1): - handles.append(plt.Line2D([0],[0],color=line_colors['exp'], - linewidth=s*1e3/linewidth_factor)) - labels.append("{} GW".format(s)) -l1 = l1_1 = ax.legend(handles, labels, - loc="upper left", bbox_to_anchor=(0.24, 1.01), - frameon=False, - labelspacing=0.8, handletextpad=1.5, - title='Transmission Exist./Exp. ') -ax.add_artist(l1_1) + for s in (10, 1): + handles.append(plt.Line2D([0],[0],color=line_colors['exp'], + linewidth=s*1e3/linewidth_factor)) + labels.append("{} GW".format(s)) + l1 = l1_1 = ax.legend(handles, labels, + loc="upper left", bbox_to_anchor=(0.24, 1.01), + frameon=False, + labelspacing=0.8, handletextpad=1.5, + title='Transmission Exist./Exp. ') + ax.add_artist(l1_1) -handles = [] -labels = [] -for s in (10, 5): - handles.append(plt.Line2D([0],[0],color=line_colors['cur'], - linewidth=s*1e3/linewidth_factor)) - labels.append("/") -l1_2 = ax.legend(handles, labels, - loc="upper left", bbox_to_anchor=(0.26, 1.01), - frameon=False, - labelspacing=0.8, handletextpad=0.5, - title=' ') -ax.add_artist(l1_2) + handles = [] + labels = [] + for s in (10, 5): + handles.append(plt.Line2D([0],[0],color=line_colors['cur'], + linewidth=s*1e3/linewidth_factor)) + labels.append("/") + l1_2 = ax.legend(handles, labels, + loc="upper left", bbox_to_anchor=(0.26, 1.01), + frameon=False, + labelspacing=0.8, handletextpad=0.5, + title=' ') + ax.add_artist(l1_2) -handles = make_legend_circles_for([10e3, 5e3, 1e3], scale=bus_size_factor, facecolor="w") -labels = ["{} GW".format(s) for s in (10, 5, 3)] -l2 = ax.legend(handles, labels, - loc="upper left", bbox_to_anchor=(0.01, 1.01), - frameon=False, labelspacing=1.0, - title='Generation', - handler_map=make_handler_map_to_scale_circles_as_in(ax)) -ax.add_artist(l2) + handles = make_legend_circles_for([10e3, 5e3, 1e3], scale=bus_size_factor, facecolor="w") + labels = ["{} GW".format(s) for s in (10, 5, 3)] + l2 = ax.legend(handles, labels, + loc="upper left", bbox_to_anchor=(0.01, 1.01), + frameon=False, labelspacing=1.0, + title='Generation', + handler_map=make_handler_map_to_scale_circles_as_in(ax)) + ax.add_artist(l2) -techs = (bus_sizes.index.levels[1]) & pd.Index(opts['vre_techs'] + opts['conv_techs'] + opts['storage_techs']) -handles = [] -labels = [] -for t in techs: - handles.append(plt.Line2D([0], [0], color=tech_colors[t], marker='o', markersize=8, linewidth=0)) - labels.append(opts['nice_names'].get(t, t)) -l3 = ax.legend(handles, labels, loc="upper center", bbox_to_anchor=(0.5, -0.), # bbox_to_anchor=(0.72, -0.05), - handletextpad=0., columnspacing=0.5, ncol=4, title='Technology') + techs = (bus_sizes.index.levels[1]) & pd.Index(opts['vre_techs'] + opts['conv_techs'] + opts['storage_techs']) + handles = [] + labels = [] + for t in techs: + handles.append(plt.Line2D([0], [0], color=tech_colors[t], marker='o', markersize=8, linewidth=0)) + labels.append(opts['nice_names'].get(t, t)) + l3 = ax.legend(handles, labels, loc="upper center", bbox_to_anchor=(0.5, -0.), # bbox_to_anchor=(0.72, -0.05), + handletextpad=0., columnspacing=0.5, ncol=4, title='Technology') - -fig.savefig(snakemake.output.only_map, dpi=150, - bbox_inches='tight', bbox_extra_artists=[l1,l2,l3]) + return fig #n = load_network(snakemake.input.network, opts, combine_hydro_ps=False) -## Add total energy p -ax1 = ax = fig.add_axes([-0.115, 0.625, 0.2, 0.2]) -ax.set_title('Energy per technology', fontdict=dict(fontsize="medium")) +def plot_total_energy_pie(n, ax=None): + """Add total energy pie plot""" + if ax is None: + ax = plt.gca() -e_primary = aggregate_p(n).drop('load', errors='ignore').loc[lambda s: s>0] + ax.set_title('Energy per technology', fontdict=dict(fontsize="medium")) -patches, texts, autotexts = ax.pie(e_primary, - startangle=90, - labels = e_primary.rename(opts['nice_names_n']).index, - autopct='%.0f%%', - shadow=False, - colors = [tech_colors[tech] for tech in e_primary.index]) -for t1, t2, i in zip(texts, autotexts, e_primary.index): - if e_primary.at[i] < 0.04 * e_primary.sum(): - t1.remove() - t2.remove() + e_primary = aggregate_p(n).drop('load', errors='ignore').loc[lambda s: s>0] -## Add average system cost bar plot -# ax2 = ax = fig.add_axes([-0.1, 0.2, 0.1, 0.33]) -# ax2 = ax = fig.add_axes([-0.1, 0.15, 0.1, 0.37]) -ax2 = ax = fig.add_axes([-0.075, 0.1, 0.1, 0.45]) -total_load = (n.snapshot_weightings * n.loads_t.p.sum(axis=1)).sum() + patches, texts, autotexts = ax.pie(e_primary, + startangle=90, + labels = e_primary.rename(opts['nice_names_n']).index, + autopct='%.0f%%', + shadow=False, + colors = [tech_colors[tech] for tech in e_primary.index]) + for t1, t2, i in zip(texts, autotexts, e_primary.index): + if e_primary.at[i] < 0.04 * e_primary.sum(): + t1.remove() + t2.remove() -def split_costs(n): - costs = aggregate_costs(n).reset_index(level=0, drop=True) - costs_ex = aggregate_costs(n, existing_only=True).reset_index(level=0, drop=True) - return (costs['capital'].add(costs['marginal'], fill_value=0.), - costs_ex['capital'], costs['capital'] - costs_ex['capital'], costs['marginal']) +def plot_total_cost_bar(n, ax=None): + """Add average system cost bar plot""" + if ax is None: + ax = plt.gca() -costs, costs_cap_ex, costs_cap_new, costs_marg = split_costs(n) + total_load = (n.snapshot_weightings * n.loads_t.p.sum(axis=1)).sum() -costs_graph = pd.DataFrame(dict(a=costs.drop('load', errors='ignore')), - index=['AC-AC', 'AC line', 'onwind', 'offwind-ac', 'offwind-dc', 'solar', 'OCGT','CCGT', 'battery', 'H2']).dropna() -bottom = np.array([0., 0.]) -texts = [] + def split_costs(n): + costs = aggregate_costs(n).reset_index(level=0, drop=True) + costs_ex = aggregate_costs(n, existing_only=True).reset_index(level=0, drop=True) + return (costs['capital'].add(costs['marginal'], fill_value=0.), + costs_ex['capital'], costs['capital'] - costs_ex['capital'], costs['marginal']) -for i,ind in enumerate(costs_graph.index): - data = np.asarray(costs_graph.loc[ind])/total_load - ax.bar([0.5], data, bottom=bottom, color=tech_colors[ind], width=0.7, zorder=-1) - bottom_sub = bottom - bottom = bottom+data + costs, costs_cap_ex, costs_cap_new, costs_marg = split_costs(n) - if ind in opts['conv_techs'] + ['AC line']: - for c in [costs_cap_ex, costs_marg]: - if ind in c: - data_sub = np.asarray([c.loc[ind]])/total_load - ax.bar([0.5], data_sub, linewidth=0, - bottom=bottom_sub, color=tech_colors[ind], - width=0.7, zorder=-1, alpha=0.8) - bottom_sub += data_sub + costs_graph = pd.DataFrame(dict(a=costs.drop('load', errors='ignore')), + index=['AC-AC', 'AC line', 'onwind', 'offwind-ac', 'offwind-dc', 'solar', 'OCGT','CCGT', 'battery', 'H2']).dropna() + bottom = np.array([0., 0.]) + texts = [] - if abs(data[-1]) < 5: - continue + for i,ind in enumerate(costs_graph.index): + data = np.asarray(costs_graph.loc[ind])/total_load + ax.bar([0.5], data, bottom=bottom, color=tech_colors[ind], width=0.7, zorder=-1) + bottom_sub = bottom + bottom = bottom+data - text = ax.text(1.1,(bottom-0.5*data)[-1]-3,opts['nice_names_n'].get(ind,ind)) - texts.append(text) + if ind in opts['conv_techs'] + ['AC line']: + for c in [costs_cap_ex, costs_marg]: + if ind in c: + data_sub = np.asarray([c.loc[ind]])/total_load + ax.bar([0.5], data_sub, linewidth=0, + bottom=bottom_sub, color=tech_colors[ind], + width=0.7, zorder=-1, alpha=0.8) + bottom_sub += data_sub -ax.set_ylabel("Average system cost [Eur/MWh]") -ax.set_ylim([0,80]) # opts['costs_max']]) -ax.set_xlim([0,1]) -#ax.set_xticks([0.5]) -ax.set_xticklabels([]) #["w/o\nEp", "w/\nEp"]) -ax.grid(True, axis="y", color='k', linestyle='dotted') + if abs(data[-1]) < 5: + continue -#fig.tight_layout() + text = ax.text(1.1,(bottom-0.5*data)[-1]-3,opts['nice_names_n'].get(ind,ind)) + texts.append(text) -ll = snakemake.wildcards.ll -ll_type = ll[0] -ll_factor = ll[1:] -lbl = dict(c='line cost', v='line volume')[ll_type] -amnt = '{ll} x today\'s'.format(ll=ll_factor) if ll_factor != 'opt' else 'optimal' -fig.suptitle('Expansion to {amount} {label} at {clusters} clusters' - .format(amount=amnt, label=lbl, clusters=snakemake.wildcards.clusters)) + ax.set_ylabel("Average system cost [Eur/MWh]") + ax.set_ylim([0,80]) # opts['costs_max']]) + ax.set_xlim([0,1]) + #ax.set_xticks([0.5]) + ax.set_xticklabels([]) #["w/o\nEp", "w/\nEp"]) + ax.grid(True, axis="y", color='k', linestyle='dotted') -fig.savefig(snakemake.output.ext, transparent=True, - bbox_inches='tight', bbox_extra_artists=[l1, l2, l3, ax1, ax2]) + +if __name__ == "__main__": + if 'snakemake' not in globals(): + from vresutils.snakemake import MockSnakemake, Dict + from snakemake.rules import expand + + snakemake = Dict() + snakemake = MockSnakemake( + path='..', + wildcards=dict(network='elec', simpl='', clusters='90', lv='1.25', opts='Co2L-3H', attr='p_nom', ext="pdf"), + input=dict(network="results/networks/{network}_s{simpl}_{clusters}_lv{lv}_{opts}.nc", + tech_costs="data/costs.csv"), + output=dict(only_map="results/plots/{network}_s{simpl}_{clusters}_lv{lv}_{opts}_{attr}.{ext}", + ext="results/plots/{network}_s{simpl}_{clusters}_lv{lv}_{opts}_{attr}_ext.{ext}") + ) + + logging.basicConfig(level=snakemake.config['logging_level']) + + set_plot_style() + + opts = snakemake.config['plotting'] + map_figsize = opts['map']['figsize'] + map_boundaries = opts['map']['boundaries'] + + n = load_network(snakemake.input.network, snakemake.input.tech_costs, snakemake.config) + + scenario_opts = snakemake.wildcards.opts.split('-') + + fig, ax = plt.subplots(figsize=map_figsize, subplot_kw={"projection": ccrs.PlateCarree()}) + plot_map(n, ax, snakemake.wildcards.attr, opts) + + fig.savefig(snakemake.output.only_map, dpi=150, + bbox_inches='tight', bbox_extra_artists=[l1,l2,l3]) + + ax1 = fig.add_axes([-0.115, 0.625, 0.2, 0.2]) + plot_total_energy_pie(n, ax1) + + ax2 = fig.add_axes([-0.075, 0.1, 0.1, 0.45]) + plot_total_cost_bar(n, ax2) + + #fig.tight_layout() + + ll = snakemake.wildcards.ll + ll_type = ll[0] + ll_factor = ll[1:] + lbl = dict(c='line cost', v='line volume')[ll_type] + amnt = '{ll} x today\'s'.format(ll=ll_factor) if ll_factor != 'opt' else 'optimal' + fig.suptitle('Expansion to {amount} {label} at {clusters} clusters' + .format(amount=amnt, label=lbl, clusters=snakemake.wildcards.clusters)) + + fig.savefig(snakemake.output.ext, transparent=True, + bbox_inches='tight', bbox_extra_artists=[l1, l2, l3, ax1, ax2]) diff --git a/scripts/plot_p_nom_max.py b/scripts/plot_p_nom_max.py index 600782aa..92f9e49a 100644 --- a/scripts/plot_p_nom_max.py +++ b/scripts/plot_p_nom_max.py @@ -1,3 +1,20 @@ +""" +Plots renewable installation potentials per capacity factor. + +Relevant Settings +----------------- + +Inputs +------ + +Outputs +------- + +Description +----------- + +""" + import pypsa import pandas as pd import matplotlib.pyplot as plt @@ -20,7 +37,7 @@ def cum_p_nom_max(net, tech, country=None): return generators -if __name__ == __main__: +if __name__ == "__main__": # Detect running outside of snakemake and mock snakemake for testing if 'snakemake' not in globals(): from vresutils.snakemake import MockSnakemake, Dict diff --git a/scripts/plot_summary.py b/scripts/plot_summary.py index 2a149a4c..9a7aa7e4 100644 --- a/scripts/plot_summary.py +++ b/scripts/plot_summary.py @@ -1,3 +1,20 @@ +""" +Plots energy and cost summaries for solved networks. + +Relevant Settings +----------------- + +Inputs +------ + +Outputs +------- + +Description +----------- + +""" + import os import pandas as pd import matplotlib.pyplot as plt diff --git a/scripts/prepare_links_p_nom.py b/scripts/prepare_links_p_nom.py index 84c1cde0..061adc35 100644 --- a/scripts/prepare_links_p_nom.py +++ b/scripts/prepare_links_p_nom.py @@ -1,25 +1,57 @@ #!/usr/bin/env python +""" +Extracts capacities of HVDC links from `Wikipedia `_. + +Relevant Settings +----------------- + +.. code:: yaml + + enable: + prepare_links_p_nom: + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at + :ref:`toplevel_cf` + +Inputs +------ + +*None* + +Outputs +------- + +- ``data/links_p_nom.csv``: A plain download of https://en.wikipedia.org/wiki/List_of_HVDC_projects#Europe plus extracted coordinates. + +Description +----------- + +*None* + +""" import pandas as pd import numpy as np -links_p_nom = pd.read_html('https://en.wikipedia.org/wiki/List_of_HVDC_projects', header=0, match="SwePol")[0] +if __name__ == "__main__": + links_p_nom = pd.read_html('https://en.wikipedia.org/wiki/List_of_HVDC_projects', header=0, match="SwePol")[0] -def extract_coordinates(s): - regex = (r"(\d{1,2})°(\d{1,2})′(\d{1,2})″(N|S) " - r"(\d{1,2})°(\d{1,2})′(\d{1,2})″(E|W)") - e = s.str.extract(regex, expand=True) - lat = (e[0].astype(float) + (e[1].astype(float) + e[2].astype(float)/60.)/60.)*e[3].map({'N': +1., 'S': -1.}) - lon = (e[4].astype(float) + (e[5].astype(float) + e[6].astype(float)/60.)/60.)*e[7].map({'E': +1., 'W': -1.}) - return lon, lat + def extract_coordinates(s): + regex = (r"(\d{1,2})°(\d{1,2})′(\d{1,2})″(N|S) " + r"(\d{1,2})°(\d{1,2})′(\d{1,2})″(E|W)") + e = s.str.extract(regex, expand=True) + lat = (e[0].astype(float) + (e[1].astype(float) + e[2].astype(float)/60.)/60.)*e[3].map({'N': +1., 'S': -1.}) + lon = (e[4].astype(float) + (e[5].astype(float) + e[6].astype(float)/60.)/60.)*e[7].map({'E': +1., 'W': -1.}) + return lon, lat -m_b = links_p_nom["Power (MW)"].str.contains('x').fillna(False) -def multiply(s): return s.str[0].astype(float) * s.str[1].astype(float) -links_p_nom.loc[m_b, "Power (MW)"] = links_p_nom.loc[m_b, "Power (MW)"].str.split('x').pipe(multiply) -links_p_nom["Power (MW)"] = links_p_nom["Power (MW)"].str.extract("[-/]?([\d.]+)", expand=False).astype(float) + m_b = links_p_nom["Power (MW)"].str.contains('x').fillna(False) + def multiply(s): return s.str[0].astype(float) * s.str[1].astype(float) -links_p_nom['x1'], links_p_nom['y1'] = extract_coordinates(links_p_nom['Converterstation 1']) -links_p_nom['x2'], links_p_nom['y2'] = extract_coordinates(links_p_nom['Converterstation 2']) + links_p_nom.loc[m_b, "Power (MW)"] = links_p_nom.loc[m_b, "Power (MW)"].str.split('x').pipe(multiply) + links_p_nom["Power (MW)"] = links_p_nom["Power (MW)"].str.extract("[-/]?([\d.]+)", expand=False).astype(float) -links_p_nom.dropna(subset=['x1', 'y1', 'x2', 'y2']).to_csv(snakemake.output[0], index=False) + links_p_nom['x1'], links_p_nom['y1'] = extract_coordinates(links_p_nom['Converterstation 1']) + links_p_nom['x2'], links_p_nom['y2'] = extract_coordinates(links_p_nom['Converterstation 2']) + links_p_nom.dropna(subset=['x1', 'y1', 'x2', 'y2']).to_csv(snakemake.output[0], index=False) diff --git a/scripts/prepare_network.py b/scripts/prepare_network.py index b44effaa..9a954eac 100644 --- a/scripts/prepare_network.py +++ b/scripts/prepare_network.py @@ -1,4 +1,54 @@ # coding: utf-8 +""" +Prepare PyPSA network for solving according to :ref:`opts` and :ref:`ll`, such as + +- adding an annual **limit** of carbon-dioxide emissions, +- adding an exogenous **price** of carbon-dioxide emissions, +- setting an **N-1 security margin** factor for transmission line capacities, +- specifying a limit on the **cost** of transmission expansion, +- specifying a limit on the **volume** of transmission expansion, and +- reducing the **temporal** resolution by averaging over multiple hours. + +Relevant Settings +----------------- + +.. code:: yaml + + costs: + emission_prices: + USD2013_to_EUR2013: + discountrate: + marginal_cost: + capital_cost: + + electricity: + co2limit: + max_hours: + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at + :ref:`costs_cf`, :ref:`electricity_cf` + +Inputs +------ + +- ``data/costs.csv``: The database of cost assumptions for all included technologies for specific years from various sources; e.g. discount rate, lifetime, investment (CAPEX), fixed operation and maintenance (FOM), variable operation and maintenance (VOM), fuel costs, efficiency, carbon-dioxide intensity. +- ``networks/{network}_s{simpl}_{clusters}.nc``: confer :ref:`cluster` + +Outputs +------- + +- ``networks/{network}_s{simpl}_{clusters}_l{ll}_{opts}.nc``: Complete PyPSA network that will be handed to the ``solve_network`` rule. + +Description +----------- + +.. tip:: + The rule :mod:`prepare_all_networks` runs + for all ``scenario`` s in the configuration file + the rule :mod:`prepare_network`. + +""" import logging logger = logging.getLogger(__name__) diff --git a/scripts/simplify_network.py b/scripts/simplify_network.py index e419989a..5ea0d282 100644 --- a/scripts/simplify_network.py +++ b/scripts/simplify_network.py @@ -1,4 +1,82 @@ # coding: utf-8 +""" +Lifts electrical transmission network to a single 380 kV voltage layer, +removes dead-ends of the network, +and reduces multi-hop HVDC connections to a single link. + +Relevant Settings +----------------- + +.. code:: yaml + + costs: + USD2013_to_EUR2013: + discountrate: + marginal_cost: + capital_cost: + + electricity: + max_hours: + + renewables: (keys) + {technology}: + potential: + + lines: + length_factor: + + links: + p_max_pu: + + solving: + solver: + name: + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at + :ref:`costs_cf`, :ref:`electricity_cf`, :ref:`renewable_cf`, + :ref:`lines_cf`, :ref:`links_cf`, :ref:`solving_cf` + +Inputs +------ + +- ``data/costs.csv``: The database of cost assumptions for all included technologies for specific years from various sources; e.g. discount rate, lifetime, investment (CAPEX), fixed operation and maintenance (FOM), variable operation and maintenance (VOM), fuel costs, efficiency, carbon-dioxide intensity. +- ``resources/regions_onshore.geojson``: confer :ref:`busregions` +- ``resources/regions_offshore.geojson``: confer :ref:`busregions` +- ``networks/{network}.nc``: confer :ref:`electricity` + +Outputs +------- + +- ``resources/regions_onshore_{network}_s{simpl}.geojson``: + + .. image:: ../img/regions_onshore_elec_s.png + :scale: 33 % + +- ``resources/regions_offshore_{network}_s{simpl}.geojson``: + + .. image:: ../img/regions_offshore_elec_s .png + :scale: 33 % + +- ``resources/clustermaps_{network}_s{simpl}.h5``: Mapping of buses from ``networks/elec.nc`` to ``networks/elec_s{simpl}.nc``; has keys ['/busmap_s'] +- ``networks/{network}_s{simpl}.nc``: + + .. image:: ../img/elec_s.png + :scale: 33 % + +Description +----------- + +The rule :mod:`simplify_network` does up to four things: + +1. Create an equivalent transmission network in which all voltage levels are mapped to the 380 kV level by the function ``simplify_network(...)``. + +2. DC only sub-networks that are connected at only two buses to the AC network are reduced to a single representative link in the function ``simplify_links(...)``. The components attached to buses in between are moved to the nearest endpoint. The grid connection cost of offshore wind generators are added to the captial costs of the generator. + +3. Stub lines and links, i.e. dead-ends of the network, are sequentially removed from the network in the function ``remove_stubs(...)``. Components are moved along. + +4. Optionally, if an integer were provided for the wildcard ``{simpl}`` (e.g. ``networks/elec_s500.nc``), the network is clustered to this number of clusters with the routines from the ``cluster_network`` rule with the function ``cluster_network.cluster(...)``. This step is usually skipped! +""" import pandas as pd idx = pd.IndexSlice diff --git a/scripts/solve_network.py b/scripts/solve_network.py index e20a8398..99b476bc 100644 --- a/scripts/solve_network.py +++ b/scripts/solve_network.py @@ -1,3 +1,82 @@ +""" +Solves linear optimal power flow for a network iteratively while updating reactances. + +Relevant Settings +----------------- + +.. code:: yaml + + (electricity:) + (BAU_mincapacities:) + (SAFE_reservemargin:) + + solving: + tmpdir: + options: + formulation: + clip_p_max_pu: + load_shedding: + noisy_costs: + nhours: + min_iterations: + max_iterations: + solver: + name: + (solveroptions): + + (plotting:) + (conv_techs:) + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at + :ref:`electricity_cf`, :ref:`solving_cf`, :ref:`plotting_cf` + +Inputs +------ + +- ``networks/{network}_s{simpl}_{clusters}_l{ll}_{opts}.nc``: confer :ref:`prepare` + +Outputs +------- + +- ``results/networks/{network}_s{simpl}_{clusters}_l{ll}_{opts}.nc``: Solved PyPSA network including optimisation results + + .. image:: ../img/results.png + :scale: 40 % + +Description +----------- + +Total annual system costs are minimised with PyPSA. The full formulation of the +linear optimal power flow (plus investment planning +is provided in the +`documentation of PyPSA `_. +Additionaly some extra constraints from :mod:`prepare_network` are added. + +Solving the network in multiple iterations is motivated through the dependence of transmission line capacities and impedances. +As lines are expanded their electrical parameters change, which renders the optimisation bilinear even if the power flow +equations are linearized. +To retain the computational advantage of continuous linear programming, a sequential linear programming technique +is used, where in between iterations the line impedances are updated. +Details (and errors made through this heuristic) are discussed in the paper + +- Fabian Neumann and Tom Brown. `Heuristics for Transmission Expansion Planning in Low-Carbon Energy System Models `_), *16th International Conference on the European Energy Market*, 2019. `arXiv:1907.10548 `_. + +.. warning:: + Capital costs of existing network components are not included in the objective function, + since for the optimisation problem they are just a constant term (no influence on optimal result). + + Therefore, these capital costs are not included in ``network.objective``! + + If you want to calculate the full total annual system costs add these to the objective value. + +.. tip:: + The rule :mod:`solve_all_networks` runs + for all ``scenario`` s in the configuration file + the rule :mod:`solve_network`. + +""" + import numpy as np import pandas as pd import logging diff --git a/scripts/solve_operations_network.py b/scripts/solve_operations_network.py index 9b454f61..595e71ee 100644 --- a/scripts/solve_operations_network.py +++ b/scripts/solve_operations_network.py @@ -1,3 +1,46 @@ +""" +Solves linear optimal dispatch in hourly resolution +using the capacities of previous capacity expansion in rule :mod:`solve_network`. + +Relevant Settings +----------------- + +.. code:: yaml + + solving: + tmpdir: + options: + formulation: + clip_p_max_pu: + load_shedding: + noisy_costs: + nhours: + min_iterations: + max_iterations: + solver: + name: + (solveroptions): + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at + :ref:`solving_cf` + +Inputs +------ + +- ``networks/{network}_s{simpl}_{clusters}.nc``: confer :ref:`cluster` +- ``results/networks/{network}_s{simpl}_{clusters}_l{ll}_{opts}.nc``: confer :ref:`solve` + +Outputs +------- + +- ``results/networks/{network}_s{simpl}_{clusters}_l{ll}_{opts}_op.nc``: Solved PyPSA network for optimal dispatch including optimisation results + +Description +----------- + +""" + import pypsa import numpy as np import re diff --git a/scripts/trace_solve_network.py b/scripts/trace_solve_network.py index da40b9f7..c62a8f52 100644 --- a/scripts/trace_solve_network.py +++ b/scripts/trace_solve_network.py @@ -1,3 +1,45 @@ +""" +Iteratively solves expansion problem like the rule :mod:`solve_network`, but additionally +records intermediate branch capacity steps and values of the objective function. + +Relevant Settings +----------------- + +.. code:: yaml + + solving: + tmpdir: + options: + formulation: + clip_p_max_pu: + load_shedding: + noisy_costs: + nhours: + min_iterations: + max_iterations: + solver: + name: + (solveroptions): + +.. seealso:: + Documentation of the configuration file ``config.yaml`` at + :ref:`solving_cf` + +Inputs +------ + +- ``networks/{network}_s{simpl}_{clusters}_l{ll}_{opts}.nc``: confer :ref:`prepare` + +Outputs +------- + +- ``results/networks/{network}_s{simpl}_{clusters}_l{ll}_{opts}_trace.nc``: Solved PyPSA network including optimisation results (with trace) + +Description +----------- + +""" + import numpy as np import pandas as pd import logging