draft validation section in config

config/config.validation.yaml

@@ -1,22 +1,21 @@
 # SPDX-FileCopyrightText: : 2017-2023 The PyPSA-Eur Authors
 #
 # SPDX-License-Identifier: CC0-1.0
-
 run:
-  name: "validation" # use this to keep track of runs with different settings
+  name: "validation"
 
 scenario:
-  ll:  # allowed transmission line volume expansion, can be any float >= 1.0 with a prefix v|c (today) or "copt"
+  ll:
   - v1.0
-  clusters:  # number of nodes in Europe, any integer between 37 (1 node per country-zone) and several hundred
+  clusters:
   - 37
-  opts: # only relevant for PyPSA-Eur
+  opts:
   - 'Ept'
 
 snapshots:
   start: "2019-01-01"
   end: "2020-01-01"
-  inclusive: 'left' # include start, not end
+  inclusive: 'left'
 
 enable:
   retrieve_cutout: false
@@ -26,11 +25,10 @@ electricity:
 
   extendable_carriers:
     Generator: []
-    StorageUnit: [] # battery, H2
+    StorageUnit: []
     Store: []
-    Link: [] # H2 pipeline
+    Link: []
 
-  # use pandas query strings here, e.g. Country not in ['Germany']
   powerplants_filter: not (DateOut < 2019)
 
   conventional_carriers: [nuclear, oil, OCGT, CCGT, coal, lignite, geothermal, biomass]
@@ -39,22 +37,17 @@ electricity:
   estimate_renewable_capacities:
     year: 2019
 
-
 atlite:
   default_cutout: europe-2019-era5
   cutouts:
-    # use 'base' to determine geographical bounds and time span from config
-    # base:
-      # module: era5
     europe-2019-era5:
-      module: era5 # in priority order
+      module: era5
       x: [-12., 35.]
       y: [33., 72]
       dx: 0.3
       dy: 0.3
       time: ['2019', '2019']
 
-
 renewable:
   onwind:
     cutout: europe-2019-era5
@@ -69,29 +62,26 @@ renewable:
     flatten_dispatch: 0.01
 
 conventional:
-  unit_commitment: true
+  unit_commitment: false
   dynamic_fuel_price: true
   nuclear:
-    p_max_pu: "data/nuclear_p_max_pu.csv" # float of file name
+    p_max_pu: "data/nuclear_p_max_pu.csv"
   biomass:
     p_max_pu: 0.65
 
-
 load:
-  power_statistics: false # only for files from <2019; set false in order to get ENTSOE transparency data
-
+  power_statistics: false
 
 lines:
   s_max_pu: 0.23
-  under_construction: 'remove' # 'zero': set capacity to zero, 'remove': remove, 'keep': with full capacity
-
+  under_construction: 'remove'
 
 links:
   include_tyndp: false
 
 costs:
   year: 2020
-  emission_prices: # in currency per tonne emission, only used with the option Ep
+  emission_prices:
     co2: 25
 
 clustering:
@@ -100,9 +90,7 @@ clustering:
   cluster_network:
     consider_efficiency_classes: true
 
-
 solving:
-  #tmpdir: "path/to/tmp"
   options:
     load_shedding: true
     rolling_horizon: false

doc/index.rst

@@ -280,6 +280,7 @@ The PyPSA-Eur workflow is continuously tested for Linux, macOS and Windows (WSL
 
    release_notes
    licenses
+   validation
    limitations
    contributing
    support

doc/validation.rst

@@ -0,0 +1,53 @@
+..
+  SPDX-FileCopyrightText: 2019-2023 The PyPSA-Eur Authors
+
+  SPDX-License-Identifier: CC-BY-4.0
+
+##########################################
+Validation
+##########################################
+
+The PyPSA-Eur model workflow provides a built-in mechanism for validation. This allows users to contrast the outcomes of network optimization against the historical performance of the European power system. The snakemake rule ``validate_elec_networks`` enables this by generating comparative figures that encapsulate key data points such as dispatch carrier, cross-border flows, and market prices per price zone.
+
+These comparisons utilize data from the 2019 ENTSO-E Transparency Platform. To enable this, an ENTSO-E API key must be inserted into the ``config.yaml`` file. Detailed steps for this process can be found in the user guide `here <https://transparency.entsoe.eu/content/static_content/Static%20content/web%20api/Guide.html>`_.
+
+Once the API key is set, the validation workflow can be triggered by running the following command:
+
+    snakemake validate_elec_networks --configfile config/config.validation.yaml -c8
+
+
+The configuration file `config/config.validation.yaml` contains the following parameters:
+
+.. literalinclude:: ../config/config.validation.yaml
+   :language: yaml
+
+The setup uses monthly varying fuel prices for gas, lignite, coal and oil as well as CO2 prices, which are created by the script ``build_monthly_prices``. Upon completion of the validation process, the resulting network and generated figures will be stored in the ``results/validation`` directory for further analysis.
+
+
+Results
+=======
+
+By the time of writing the comparison with the historical data shows partially accurate, partially improvable results. The following figures show the comparison of the dispatch of the different carriers.
+
+.. image:: ../graphics/validation_seasonal_operation_area_elec_s_37_ec_lv1.0_Ept.png
+   :width: 100%
+   :align: center
+
+.. image:: ../graphics/validation_production_bar_elec_s_37_ec_lv1.0_Ept.png
+   :width: 100%
+   :align: center
+
+
+
+Issues and possible improvements
+--------------------------------
+
+**Overestimated dispatch of wind and solar:** Renewable potentials of wind and solar are slightly overestimated in the model. This leads to a higher dispatch of these carriers than in the historical data. In particular, the solar dispatch during winter is overestimated.
+
+**Coal - Lignite fuel switch:** The model has a fuel switch from coal to lignite. This might result from non-captured subsidies for lignite and coal in the model. In order to fix the fuel switch from coal to lignite, a manual cost correction was added to the script ``build_monthly_prices``.
+
+**Planned outages of nuclear power plants:** Planned outages of nuclear power plants are not captured in the model. This leads to a underestimated dispatch of nuclear power plants in winter and a overestimated dispatch in summer. This point is hard to fix, since the planned outages are not published in the ENTSO-E Transparency Platform.
+
+**False classification of run-of-river power plants:** Some run-of-river power plants are classified as hydro power plants in the model. This leads to a general overestimation of the hydro power dispatch. In particular, Swedish hydro power plants are overestimated.
+
+**Load shedding:** Due to constraint NTC's (crossborder capacities), the model has to shed load in some regions. This leads to a high market prices in the regions which drive the average market price up. Further fine-tuning of the NTC's is needed to avoid load shedding.

matplotlibrc

@@ -5,3 +5,4 @@ font.family: sans-serif
 font.sans-serif: Ubuntu, DejaVu Sans
 image.cmap: viridis
 figure.autolayout : True
+figure.dpi: 300