Addition of unsustainable biomass potentials (#1139)

* add columns to potential df defined by difference to eurostat

* add network components

* add unsustainable bioliquids

* replaced stores by generators, still infeasible

* remove municipal waste

* remove separate treatment of waste from biomass potential calculation

* phase out unsustainble biomass potentials

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* phase-out unsustainable bioliquids

* remove waste_incineration from build_sector rule

* multiple potential calculation for different planning horizons

* raised costs of unsustainable solid biomass

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* stores instead of generators

* change snakemake inputs

* add phas-eout to config

* add techcolor for unsustainable bioliquids

* add config parameter to disable inclusion of unsustainable bioenergy potentials

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add biomass to params

* remove call of snakemake object in define_spatial

* Quick resolve of review part 1 (config parameters, if-clause-reduction, bioliquid spatial, fix bioliquid link capacity

* Quick resolve of review part 2 (config table, helper function, fixed build_eurostat, removed dir-change, forced unsustainable usage, reverted overnight distinction in Snakefile)

* Cast of planning_horizon parameter to int type after test run

* added JRC fuel costs for solid and liquid biofuels, BtL VOM

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* clean-up after master merge

* adressed review (increase threads for build_eurostat, fixed e_max_pu of Stores, changed version of technology-data); added release note

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: lisazeyen <35347358+lisazeyen@users.noreply.github.com>

config/config.default.yaml

@@ -369,6 +369,23 @@ biomass:
     - Sludge
     municipal solid waste:
     - Municipal waste
+  share_unsustainable_use_retained:
+    2020: 1
+    2025: 0.66
+    2030: 0.33
+    2035: 0
+    2040: 0
+    2045: 0
+    2050: 0
+  share_sustainable_potential_available:
+    2020: 0
+    2025: 0.33
+    2030: 0.66
+    2035: 1
+    2040: 1
+    2045: 1
+    2050: 1
+
 
 # docs in https://pypsa-eur.readthedocs.io/en/latest/configuration.html#solar-thermal
 solar_thermal:
@@ -737,7 +754,7 @@ industry:
 # docs in https://pypsa-eur.readthedocs.io/en/latest/configuration.html#costs
 costs:
   year: 2030
-  version: v0.9.0
+  version: v0.9.1
   social_discountrate: 0.02
   fill_values:
     FOM: 0
@@ -1055,6 +1072,7 @@ plotting:
     services rural biomass boiler: '#c6cf98'
     services urban decentral biomass boiler: '#dde5b5'
     biomass to liquid: '#32CD32'
+    unsustainable bioliquids: '#32CD32'
     electrobiofuels: 'red'
     BioSNG: '#123456'
     # power transmission

doc/configtables/biomass.csv

@@ -5,3 +5,5 @@ classes ,,,
 -- solid biomass,--,Array of biomass comodity,The comodity that are included as solid biomass
 -- not included,--,Array of biomass comodity,The comodity that are not included as a biomass potential
 -- biogas,--,Array of biomass comodity,The comodity that are included as biogas
+share_unsustainable_use_retained,--,Dictionary with planning horizons as keys., Share of unsustainable biomass use retained using primary production of Eurostat data as reference
+share_sustainable_potential_available,--,Dictionary with planning horizons as keys., Share determines phase-in of ENSPRESO biomass potentials

doc/release_notes.rst

@@ -10,6 +10,10 @@ Release Notes
 Upcoming Release
 ================
 
+* Added unsustainable biomass potentials for solid, gaseous, and liquid biomass. The potentials can be phased-out and/or
+  substituted by the phase-in of sustainable biomass types using the config parameters
+  ``biomass: share_unsustainable_use_retained`` and ``biomass: share_sustainable_potential_available``.
+
 * The rule ``prepare_links_p_nom`` was removed since it was outdated and not used.
 
 * Changed heat pump COP approximation for central heating to be based on `Jensen et al. (2018) <https://backend.orbit.dtu.dk/ws/portalfiles/portal/151965635/MAIN_Final.pdf>`__ and a default forward temperature of 90C. This is more realistic for district heating than the previously used approximation method.

rules/build_sector.smk

@@ -345,7 +345,8 @@ rule build_biomass_potentials:
             "https://zenodo.org/records/10356004/files/ENSPRESO_BIOMASS.xlsx",
             keep_local=True,
         ),
-        nuts2="data/bundle/nuts/NUTS_RG_10M_2013_4326_LEVL_2.geojson",  # https://gisco-services.ec.europa.eu/distribution/v2/nuts/download/#nuts21
+        eurostat="data/eurostat/Balances-April2023",
+        nuts2="data/bundle/nuts/NUTS_RG_10M_2013_4326_LEVL_2.geojson",
         regions_onshore=resources("regions_onshore_elec_s{simpl}_{clusters}.geojson"),
         nuts3_population=ancient("data/bundle/nama_10r_3popgdp.tsv.gz"),
         swiss_cantons=ancient("data/ch_cantons.csv"),
@@ -358,7 +359,7 @@ rule build_biomass_potentials:
         biomass_potentials=resources(
             "biomass_potentials_s{simpl}_{clusters}_{planning_horizons}.csv"
         ),
-    threads: 1
+    threads: 8
     resources:
         mem_mb=1000,
     log:
@@ -954,6 +955,7 @@ rule prepare_sector_network:
         countries=config_provider("countries"),
         adjustments=config_provider("adjustments", "sector"),
         emissions_scope=config_provider("energy", "emissions"),
+        biomass=config_provider("biomass"),
         RDIR=RDIR,
         heat_pump_sources=config_provider("sector", "heat_pump_sources"),
         heat_systems=config_provider("sector", "heat_systems"),

rules/retrieve.smk

@@ -249,6 +249,8 @@ if config["enable"]["retrieve"]:
             unpack_archive(params["zip"], output_folder)
             os.remove(params["zip"])
 
+
+
 if config["enable"]["retrieve"]:
 
     # Download directly from naciscdn.org which is a redirect from naturalearth.com

scripts/build_biomass_potentials.py

@@ -13,11 +13,51 @@ import geopandas as gpd
 import numpy as np
 import pandas as pd
 from _helpers import configure_logging, set_scenario_config
+from build_energy_totals import build_eurostat
 
 logger = logging.getLogger(__name__)
 AVAILABLE_BIOMASS_YEARS = [2010, 2020, 2030, 2040, 2050]
 
 
+def _calc_unsustainable_potential(df, df_unsustainable, share_unsus, resource_type):
+    """
+    Calculate the unsustainable biomass potential for a given resource type or
+    regex.
+
+    Parameters
+    ----------
+    df : pd.DataFrame
+        The dataframe with sustainable biomass potentials.
+    df_unsustainable : pd.DataFrame
+        The dataframe with unsustainable biomass potentials.
+    share_unsus : float
+        The share of unsustainable biomass potential retained.
+    resource_type : str or regex
+        The resource type to calculate the unsustainable potential for.
+
+    Returns
+    -------
+    pd.Series
+        The unsustainable biomass potential for the given resource type or regex.
+    """
+
+    if "|" in resource_type:
+        resource_potential = df_unsustainable.filter(regex=resource_type).sum(axis=1)
+    else:
+        resource_potential = df_unsustainable[resource_type]
+
+    return (
+        df.apply(
+            lambda c: c.sum()
+            / df.loc[df.index.str[:2] == c.name[:2]].sum().sum()
+            * resource_potential.loc[c.name[:2]],
+            axis=1,
+        )
+        .mul(share_unsus)
+        .clip(lower=0)
+    )
+
+
 def build_nuts_population_data(year=2013):
     pop = pd.read_csv(
         snakemake.input.nuts3_population,
@@ -211,15 +251,104 @@ def convert_nuts2_to_regions(bio_nuts2, regions):
     return bio_regions
 
 
+def add_unsustainable_potentials(df):
+    """
+    Add unsustainable biomass potentials to the given dataframe. The difference
+    between the data of JRC and Eurostat is assumed to be unsustainable
+    biomass.
+
+    Parameters
+    ----------
+    df : pd.DataFrame
+        The dataframe with sustainable biomass potentials.
+    unsustainable_biomass : str
+        Path to the file with unsustainable biomass potentials.
+
+    Returns
+    -------
+    pd.DataFrame
+        The dataframe with added unsustainable biomass potentials.
+    """
+    if "GB" in snakemake.config["countries"]:
+        latest_year = 2019
+    else:
+        latest_year = 2021
+    idees_rename = {"GR": "EL", "GB": "UK"}
+    df_unsustainable = (
+        build_eurostat(
+            countries=snakemake.config["countries"],
+            input_eurostat=snakemake.input.eurostat,
+            nprocesses=int(snakemake.threads),
+        )
+        .xs(
+            max(min(latest_year, int(snakemake.wildcards.planning_horizons)), 1990),
+            level=1,
+        )
+        .xs("Primary production", level=2)
+        .droplevel([1, 2, 3])
+    )
+
+    df_unsustainable.index = df_unsustainable.index.str.strip()
+    df_unsustainable = df_unsustainable.rename(
+        {v: k for k, v in idees_rename.items()}, axis=0
+    )
+
+    bio_carriers = [
+        "Primary solid biofuels",
+        "Biogases",
+        "Renewable municipal waste",
+        "Pure biogasoline",
+        "Blended biogasoline",
+        "Pure biodiesels",
+        "Blended biodiesels",
+        "Pure bio jet kerosene",
+        "Blended bio jet kerosene",
+        "Other liquid biofuels",
+    ]
+
+    df_unsustainable = df_unsustainable[bio_carriers]
+
+    # Phase out unsustainable biomass potentials linearly from 2020 to 2035 while phasing in sustainable potentials
+    share_unsus = params.get("share_unsustainable_use_retained").get(investment_year)
+
+    df_wo_ch = df.drop(df.filter(regex="CH\d", axis=0).index)
+
+    # Calculate unsustainable solid biomass
+    df_wo_ch["unsustainable solid biomass"] = _calc_unsustainable_potential(
+        df_wo_ch, df_unsustainable, share_unsus, "Primary solid biofuels"
+    )
+
+    # Calculate unsustainable biogas
+    df_wo_ch["unsustainable biogas"] = _calc_unsustainable_potential(
+        df_wo_ch, df_unsustainable, share_unsus, "Biogases"
+    )
+
+    # Calculate unsustainable bioliquids
+    df_wo_ch["unsustainable bioliquids"] = _calc_unsustainable_potential(
+        df_wo_ch,
+        df_unsustainable,
+        share_unsus,
+        resource_type="gasoline|diesel|kerosene|liquid",
+    )
+
+    share_sus = params.get("share_sustainable_potential_available").get(investment_year)
+    df *= share_sus
+
+    df = df.join(df_wo_ch.filter(like="unsustainable")).fillna(0)
+
+    return df
+
+
 if __name__ == "__main__":
     if "snakemake" not in globals():
+
         from _helpers import mock_snakemake
 
         snakemake = mock_snakemake(
             "build_biomass_potentials",
             simpl="",
-            clusters="5",
+            clusters="37",
-            planning_horizons=2050,
+            planning_horizons=2020,
         )
 
     configure_logging(snakemake)
@@ -269,6 +398,8 @@ if __name__ == "__main__":
     grouper = {v: k for k, vv in params["classes"].items() for v in vv}
     df = df.T.groupby(grouper).sum().T
 
+    df = add_unsustainable_potentials(df)
+
     df *= 1e6  # TWh/a to MWh/a
     df.index.name = "MWh/a"
 

scripts/build_shapes.py

@@ -73,10 +73,10 @@ from functools import reduce
 from itertools import takewhile
 from operator import attrgetter
 
+import country_converter as coco
 import geopandas as gpd
 import numpy as np
 import pandas as pd
-import country_converter as coco
 from _helpers import configure_logging, set_scenario_config
 from shapely.geometry import MultiPolygon, Polygon
 

scripts/prepare_sector_network.py

@@ -64,6 +64,7 @@ def define_spatial(nodes, options):
 
     if options.get("biomass_spatial", options["biomass_transport"]):
         spatial.biomass.nodes = nodes + " solid biomass"
+        spatial.biomass.bioliquids = nodes + " bioliquids"
         spatial.biomass.locations = nodes
         spatial.biomass.industry = nodes + " solid biomass for industry"
         spatial.biomass.industry_cc = nodes + " solid biomass for industry CC"
@@ -71,6 +72,7 @@ def define_spatial(nodes, options):
         spatial.msw.locations = nodes
     else:
         spatial.biomass.nodes = ["EU solid biomass"]
+        spatial.biomass.bioliquids = ["EU unsustainable bioliquids"]
         spatial.biomass.locations = ["EU"]
         spatial.biomass.industry = ["solid biomass for industry"]
         spatial.biomass.industry_cc = ["solid biomass for industry CC"]
@@ -2262,8 +2264,14 @@ def add_biomass(n, costs):
         biogas_potentials_spatial = biomass_potentials["biogas"].rename(
             index=lambda x: x + " biogas"
         )
+        unsustainable_biogas_potentials_spatial = biomass_potentials[
+            "unsustainable biogas"
+        ].rename(index=lambda x: x + " biogas")
     else:
         biogas_potentials_spatial = biomass_potentials["biogas"].sum()
+        unsustainable_biogas_potentials_spatial = biomass_potentials[
+            "unsustainable biogas"
+        ].sum()
 
     if options.get("biomass_spatial", options["biomass_transport"]):
         solid_biomass_potentials_spatial = biomass_potentials["solid biomass"].rename(
@@ -2272,11 +2280,27 @@ def add_biomass(n, costs):
         msw_biomass_potentials_spatial = biomass_potentials[
             "municipal solid waste"
         ].rename(index=lambda x: x + " municipal solid waste")
+        unsustainable_solid_biomass_potentials_spatial = biomass_potentials[
+            "unsustainable solid biomass"
+        ].rename(index=lambda x: x + " solid biomass")
+
     else:
         solid_biomass_potentials_spatial = biomass_potentials["solid biomass"].sum()
         msw_biomass_potentials_spatial = biomass_potentials[
             "municipal solid waste"
         ].sum()
+        unsustainable_solid_biomass_potentials_spatial = biomass_potentials[
+            "unsustainable solid biomass"
+        ].sum()
+
+    if options["regional_oil_demand"]:
+        unsustainable_liquid_biofuel_potentials_spatial = biomass_potentials[
+            "unsustainable bioliquids"
+        ].rename(index=lambda x: x + " bioliquids")
+    else:
+        unsustainable_liquid_biofuel_potentials_spatial = biomass_potentials[
+            "unsustainable bioliquids"
+        ].sum()
 
     n.add("Carrier", "biogas")
     n.add("Carrier", "solid biomass")
@@ -2401,6 +2425,81 @@ def add_biomass(n, costs):
             p_nom_extendable=True,
         )
 
+    if biomass_potentials.filter(like="unsustainable").sum().sum() > 0:
+
+        # Create timeseries to force usage of unsustainable potentials
+        e_max_pu = pd.DataFrame(1, index=n.snapshots, columns=spatial.gas.biogas)
+        e_max_pu.iloc[-1] = 0
+
+        n.madd(
+            "Store",
+            spatial.gas.biogas,
+            suffix=" unsustainable",
+            bus=spatial.gas.biogas,
+            carrier="unsustainable biogas",
+            e_nom=unsustainable_biogas_potentials_spatial,
+            marginal_cost=costs.at["biogas", "fuel"],
+            e_initial=unsustainable_biogas_potentials_spatial,
+            e_nom_extendable=False,
+            e_max_pu=e_max_pu,
+        )
+
+        e_max_pu = pd.DataFrame(1, index=n.snapshots, columns=spatial.biomass.nodes)
+        e_max_pu.iloc[-1] = 0
+
+        n.madd(
+            "Store",
+            spatial.biomass.nodes,
+            suffix=" unsustainable",
+            bus=spatial.biomass.nodes,
+            carrier="unsustainable solid biomass",
+            e_nom=unsustainable_solid_biomass_potentials_spatial,
+            marginal_cost=costs.at["fuelwood", "fuel"],
+            e_initial=unsustainable_solid_biomass_potentials_spatial,
+            e_nom_extendable=False,
+            e_max_pu=e_max_pu,
+        )
+
+        n.madd(
+            "Bus",
+            spatial.biomass.bioliquids,
+            location=spatial.biomass.locations,
+            carrier="unsustainable bioliquids",
+            unit="MWh_LHV",
+        )
+
+        e_max_pu = pd.DataFrame(
+            1, index=n.snapshots, columns=spatial.biomass.bioliquids
+        )
+        e_max_pu.iloc[-1] = 0
+
+        n.madd(
+            "Store",
+            spatial.biomass.bioliquids,
+            suffix=" unsustainable",
+            bus=spatial.biomass.bioliquids,
+            carrier="unsustainable bioliquids",
+            e_nom=unsustainable_liquid_biofuel_potentials_spatial,
+            marginal_cost=costs.at["biodiesel crops", "fuel"],
+            e_initial=unsustainable_liquid_biofuel_potentials_spatial,
+            e_nom_extendable=False,
+            e_max_pu=e_max_pu,
+        )
+
+        n.madd(
+            "Link",
+            spatial.biomass.bioliquids,
+            bus0=spatial.biomass.bioliquids,
+            bus1=spatial.oil.nodes,
+            bus2="co2 atmosphere",
+            carrier="unsustainable bioliquids",
+            efficiency=1,
+            efficiency2=-costs.at["solid biomass", "CO2 intensity"]
+            + costs.at["BtL", "CO2 stored"],
+            p_nom=unsustainable_liquid_biofuel_potentials_spatial,
+            marginal_cost=costs.at["BtL", "VOM"],
+        )
+
     n.madd(
         "Link",
         spatial.gas.biogas_to_gas,
@@ -4132,6 +4231,7 @@ def add_enhanced_geothermal(n, egs_potentials, egs_overlap, costs):
 # %%
 if __name__ == "__main__":
     if "snakemake" not in globals():
+
         from _helpers import mock_snakemake
 
         snakemake = mock_snakemake(