pypsa-eur/scripts/solve_network.py

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

727 lines
25 KiB
Python
Raw Normal View History

# -*- coding: utf-8 -*-
# SPDX-FileCopyrightText: : 2017-2023 The PyPSA-Eur Authors
#
2021-09-14 14:37:41 +00:00
# SPDX-License-Identifier: MIT
"""
2023-03-10 15:07:00 +00:00
Solves optimal operation and capacity for a network with the option to
iteratively optimize while updating line reactances.
This script is used for optimizing the electrical network as well as the
sector coupled network.
Relevant Settings
-----------------
2023-03-10 15:07:00 +00:00
.. code:: yaml
2023-03-10 15:07:00 +00:00
solving:
options:
formulation:
clip_p_max_pu:
load_shedding:
noisy_costs:
nhours:
min_iterations:
max_iterations:
skip_iterations:
track_iterations:
solver:
name:
options:
.. seealso::
Documentation of the configuration file ``config.yaml`` at
:ref:`electricity_cf`, :ref:`solving_cf`, :ref:`plotting_cf`
Inputs
------
2023-03-10 15:07:00 +00:00
- ``networks/elec_s{simpl}_{clusters}_ec_l{ll}_{opts}.nc``: confer :ref:`prepare`
2019-08-11 09:40:47 +00:00
2023-03-10 15:07:00 +00:00
Outputs
-------
2023-03-10 15:07:00 +00:00
- ``results/networks/elec_s{simpl}_{clusters}_ec_l{ll}_{opts}.nc``: Solved PyPSA network including optimisation results
2023-03-10 15:12:02 +00:00
.. image:: img/results.png
2023-03-10 15:07:00 +00:00
:scale: 40 %
Description
-----------
2023-03-10 15:07:00 +00:00
Total annual system costs are minimised with PyPSA. The full formulation of the
linear optimal power flow (plus investment planning
is provided in the
`documentation of PyPSA <https://pypsa.readthedocs.io/en/latest/optimal_power_flow.html#linear-optimal-power-flow>`_.
2023-03-10 15:07:00 +00:00
The optimization is based on the ``pyomo=False`` setting in the :func:`network.lopf` and :func:`pypsa.linopf.ilopf` function.
Additionally, some extra constraints specified in :mod:`prepare_network` are added.
2023-03-10 15:07:00 +00:00
Solving the network in multiple iterations is motivated through the dependence of transmission line capacities and impedances.
As lines are expanded their electrical parameters change, which renders the optimisation bilinear even if the power flow
equations are linearized.
To retain the computational advantage of continuous linear programming, a sequential linear programming technique
is used, where in between iterations the line impedances are updated.
Details (and errors made through this heuristic) are discussed in the paper
- Fabian Neumann and Tom Brown. `Heuristics for Transmission Expansion Planning in Low-Carbon Energy System Models <https://arxiv.org/abs/1907.10548>`_), *16th International Conference on the European Energy Market*, 2019. `arXiv:1907.10548 <https://arxiv.org/abs/1907.10548>`_.
.. warning::
2023-03-10 15:07:00 +00:00
Capital costs of existing network components are not included in the objective function,
since for the optimisation problem they are just a constant term (no influence on optimal result).
2023-03-10 15:07:00 +00:00
Therefore, these capital costs are not included in ``network.objective``!
If you want to calculate the full total annual system costs add these to the objective value.
.. tip::
2023-03-10 15:07:00 +00:00
The rule :mod:`solve_all_networks` runs
for all ``scenario`` s in the configuration file
the rule :mod:`solve_network`.
"""
import logging
import re
2019-08-11 11:17:36 +00:00
import numpy as np
import pandas as pd
import pypsa
import xarray as xr
from _helpers import (
configure_logging,
override_component_attrs,
update_config_with_sector_opts,
)
from vresutils.benchmark import memory_logger
2019-08-11 11:17:36 +00:00
logger = logging.getLogger(__name__)
pypsa.pf.logger.setLevel(logging.WARNING)
2019-08-11 09:40:47 +00:00
def add_land_use_constraint(n, config):
if "m" in snakemake.wildcards.clusters:
_add_land_use_constraint_m(n, config)
else:
_add_land_use_constraint(n, config)
2019-08-11 20:34:18 +00:00
2019-08-11 09:40:47 +00:00
2023-03-10 13:12:22 +00:00
def _add_land_use_constraint(n, config):
# warning: this will miss existing offwind which is not classed AC-DC and has carrier 'offwind'
2019-08-11 20:34:18 +00:00
for carrier in ["solar", "onwind", "offwind-ac", "offwind-dc"]:
ext_i = (n.generators.carrier == carrier) & ~n.generators.p_nom_extendable
existing = (
n.generators.loc[ext_i, "p_nom"]
.groupby(n.generators.bus.map(n.buses.location))
.sum()
)
existing.index += " " + carrier + "-" + snakemake.wildcards.planning_horizons
n.generators.loc[existing.index, "p_nom_max"] -= existing
# check if existing capacities are larger than technical potential
existing_large = n.generators[
n.generators["p_nom_min"] > n.generators["p_nom_max"]
].index
if len(existing_large):
logger.warning(
f"Existing capacities larger than technical potential for {existing_large},\
adjust technical potential to existing capacities"
)
n.generators.loc[existing_large, "p_nom_max"] = n.generators.loc[
existing_large, "p_nom_min"
]
2019-08-14 08:35:41 +00:00
n.generators.p_nom_max.clip(lower=0, inplace=True)
2019-08-11 09:40:47 +00:00
2019-08-14 09:07:52 +00:00
def _add_land_use_constraint_m(n, config):
# if generators clustering is lower than network clustering, land_use accounting is at generators clusters
2019-08-14 09:07:52 +00:00
planning_horizons = config["scenario"]["planning_horizons"]
grouping_years = config["existing_capacities"]["grouping_years"]
current_horizon = snakemake.wildcards.planning_horizons
2019-08-14 09:07:52 +00:00
for carrier in ["solar", "onwind", "offwind-ac", "offwind-dc"]:
existing = n.generators.loc[n.generators.carrier == carrier, "p_nom"]
ind = list(
set(
[
i.split(sep=" ")[0] + " " + i.split(sep=" ")[1]
for i in existing.index
]
)
)
2019-08-14 09:07:52 +00:00
previous_years = [
str(y)
for y in planning_horizons + grouping_years
if y < int(snakemake.wildcards.planning_horizons)
]
2019-08-14 09:07:52 +00:00
for p_year in previous_years:
ind2 = [
i for i in ind if i + " " + carrier + "-" + p_year in existing.index
]
sel_current = [i + " " + carrier + "-" + current_horizon for i in ind2]
sel_p_year = [i + " " + carrier + "-" + p_year for i in ind2]
n.generators.loc[sel_current, "p_nom_max"] -= existing.loc[
sel_p_year
].rename(lambda x: x[:-4] + current_horizon)
2019-08-14 09:07:52 +00:00
n.generators.p_nom_max.clip(lower=0, inplace=True)
Add logging to logfiles to all snakemake workflow scripts. (#102) * Add logging to logfiles to all snakemake workflow scripts. * Fix missing quotation marks in Snakefile. * Apply suggestions from code review Co-Authored-By: Fabian Neumann <fabian.neumann@outlook.de> * Apply suggestions from code review Co-Authored-By: Fabian Neumann <fabian.neumann@outlook.de> * doc: fix _ec_ filenames in docs * Allow logging message format to be specified in config.yaml. * Add logging for Snakemake rule 'retrieve_databundle '. * Add limited logging to STDERR only for retrieve_*.py scripts. * Import progressbar module only on demand. * Fix logging to file and enable concurrent printing to STDERR for most scripts. * Add new 'logging_format' option to Travis CI test config.yaml. * Add missing parenthesis (bug fix) and cross-os compatible paths. * Fix typos in messages. * Use correct log files for logging (bug fix). * doc: fix line references * config: logging_format in all configs * doc: add doc for logging_format * environment: update to powerplantmatching 0.4.3 * doc: update line references for tutorial.rst * Change logging configuration scheme for config.yaml. * Add helper function for doing basic logging configuration. * Add logpath for prepare_links_p_nom rule. * Outsource basic logging configuration for all scripts to _helper submodule. * Update documentation for changed config.yaml structure. Instead of 'logging_level' and 'logging_format', now 'logging' with subcategories is used. * _helpers: Change configure_logging signature.
2019-11-28 07:22:52 +00:00
def add_co2_sequestration_limit(n, limit=200):
"""
Add a global constraint on the amount of Mt CO2 that can be sequestered.
"""
n.carriers.loc["co2 stored", "co2_absorptions"] = -1
n.carriers.co2_absorptions = n.carriers.co2_absorptions.fillna(0)
2018-02-01 11:42:56 +00:00
limit = limit * 1e6
for o in opts:
if "seq" not in o:
continue
limit = float(o[o.find("seq") + 3 :]) * 1e6
break
n.add(
"GlobalConstraint",
"co2_sequestration_limit",
sense="<=",
constant=limit,
type="primary_energy",
carrier_attribute="co2_absorptions",
)
def prepare_network(n, solve_opts=None, config=None):
if "clip_p_max_pu" in solve_opts:
for df in (
n.generators_t.p_max_pu,
n.generators_t.p_min_pu, # TODO: check if this can be removed
n.storage_units_t.inflow,
):
df.where(df > solve_opts["clip_p_max_pu"], other=0.0, inplace=True)
if solve_opts.get("load_shedding"):
# intersect between macroeconomic and surveybased willingness to pay
# http://journal.frontiersin.org/article/10.3389/fenrg.2015.00055/full
# TODO: retrieve color and nice name from config
2022-03-03 22:08:29 +00:00
n.add("Carrier", "load", color="#dd2e23", nice_name="Load shedding")
buses_i = n.buses.query("carrier == 'AC'").index
if not np.isscalar(load_shedding):
# TODO: do not scale via sign attribute (use Eur/MWh instead of Eur/kWh)
load_shedding = 1e2 # Eur/kWh
n.madd(
"Generator",
buses_i,
" load",
bus=n.buses.index,
carrier="load",
sign=1e-3, # Adjust sign to measure p and p_nom in kW instead of MW
marginal_cost=load_shedding, # Eur/kWh
p_nom=1e9, # kW
)
if solve_opts.get("noisy_costs"):
for t in n.iterate_components():
# if 'capital_cost' in t.df:
# t.df['capital_cost'] += 1e1 + 2.*(np.random.random(len(t.df)) - 0.5)
if "marginal_cost" in t.df:
t.df["marginal_cost"] += 1e-2 + 2e-3 * (
np.random.random(len(t.df)) - 0.5
)
for t in n.iterate_components(["Line", "Link"]):
t.df["capital_cost"] += (
1e-1 + 2e-2 * (np.random.random(len(t.df)) - 0.5)
) * t.df["length"]
if solve_opts.get("nhours"):
nhours = solve_opts["nhours"]
n.set_snapshots(n.snapshots[:nhours])
n.snapshot_weightings[:] = 8760.0 / nhours
if config["foresight"] == "myopic":
add_land_use_constraint(n, config)
if n.stores.carrier.eq("co2 stored").any():
limit = config["sector"].get("co2_sequestration_potential", 200)
add_co2_sequestration_limit(n, limit=limit)
return n
def add_CCL_constraints(n, config):
2023-03-09 13:24:25 +00:00
"""
Add CCL (country & carrier limit) constraint to the network.
Add minimum and maximum levels of generator nominal capacity per carrier
for individual countries. Opts and path for agg_p_nom_minmax.csv must be defined
in config.yaml. Default file is available at data/agg_p_nom_minmax.csv.
Parameters
----------
n : pypsa.Network
config : dict
Example
-------
scenario:
opts: [Co2L-CCL-24H]
electricity:
agg_p_nom_limits: data/agg_p_nom_minmax.csv
"""
agg_p_nom_minmax = pd.read_csv(
config["electricity"]["agg_p_nom_limits"], index_col=[0, 1]
)
logger.info("Adding generation capacity constraints per carrier and country")
p_nom = n.model["Generator-p_nom"]
gens = n.generators.query("p_nom_extendable").rename_axis(index="Generator-ext")
grouper = [gens.bus.map(n.buses.country), gens.carrier]
grouper = xr.DataArray(pd.MultiIndex.from_arrays(grouper), dims=["Generator-ext"])
lhs = p_nom.groupby(grouper).sum().rename(bus="country")
minimum = xr.DataArray(agg_p_nom_minmax["min"].dropna()).rename(dim_0="group")
index = minimum.indexes["group"].intersection(lhs.indexes["group"])
if not index.empty:
n.model.add_constraints(
lhs.sel(group=index) >= minimum.loc[index], name="agg_p_nom_min"
)
2023-03-09 13:24:25 +00:00
maximum = xr.DataArray(agg_p_nom_minmax["max"].dropna()).rename(dim_0="group")
index = maximum.indexes["group"].intersection(lhs.indexes["group"])
if not index.empty:
n.model.add_constraints(
lhs.sel(group=index) <= maximum.loc[index], name="agg_p_nom_max"
)
def add_EQ_constraints(n, o, scaling=1e-1):
2023-03-09 13:24:25 +00:00
"""
Add equity constraints to the network.
Currently this is only implemented for the electricity sector only.
2023-03-09 13:24:25 +00:00
Opts must be specified in the config.yaml.
Parameters
----------
n : pypsa.Network
o : str
Example
-------
scenario:
opts: [Co2L-EQ0.7-24H]
Require each country or node to on average produce a minimal share
of its total electricity consumption itself. Example: EQ0.7c demands each country
2023-03-09 13:24:25 +00:00
to produce on average at least 70% of its consumption; EQ0.7 demands
each node to produce on average at least 70% of its consumption.
"""
# TODO: Generalize to cover myopic and other sectors?
float_regex = "[0-9]*\.?[0-9]+"
level = float(re.findall(float_regex, o)[0])
if o[-1] == "c":
ggrouper = n.generators.bus.map(n.buses.country).to_xarray()
lgrouper = n.loads.bus.map(n.buses.country).to_xarray()
sgrouper = n.storage_units.bus.map(n.buses.country).to_xarray()
else:
ggrouper = n.generators.bus.to_xarray()
lgrouper = n.loads.bus.to_xarray()
sgrouper = n.storage_units.bus.to_xarray()
load = (
n.snapshot_weightings.generators
@ n.loads_t.p_set.groupby(lgrouper, axis=1).sum()
)
inflow = (
n.snapshot_weightings.stores
@ n.storage_units_t.inflow.groupby(sgrouper, axis=1).sum()
)
inflow = inflow.reindex(load.index).fillna(0.0)
rhs = scaling * (level * load - inflow)
p = n.model["Generator-p"]
lhs_gen = (
(p * (n.snapshot_weightings.generators * scaling))
.groupby(ggrouper)
.sum()
2023-03-09 13:24:25 +00:00
.sum("snapshot")
)
# TODO: double check that this is really needed, why do have to subtract the spillage
if not n.storage_units_t.inflow.empty:
spillage = n.model["StorageUnit-spill"]
lhs_spill = (
(spillage * (-n.snapshot_weightings.stores * scaling))
.groupby(sgrouper)
.sum()
2023-03-09 13:24:25 +00:00
.sum("snapshot")
)
lhs = lhs_gen + lhs_spill
else:
lhs = lhs_gen
2023-03-09 13:24:25 +00:00
n.model.add_constraints(lhs >= rhs, name="equity_min")
def add_BAU_constraints(n, config):
2023-03-09 13:24:25 +00:00
"""
Add a per-carrier minimal overall capacity.
BAU_mincapacities and opts must be adjusted in the config.yaml.
Parameters
----------
n : pypsa.Network
config : dict
Example
-------
scenario:
opts: [Co2L-BAU-24H]
electricity:
BAU_mincapacities:
solar: 0
onwind: 0
OCGT: 100000
offwind-ac: 0
offwind-dc: 0
Which sets minimum expansion across all nodes e.g. in Europe to 100GW.
OCGT bus 1 + OCGT bus 2 + ... > 100000
"""
mincaps = pd.Series(config["electricity"]["BAU_mincapacities"])
p_nom = n.model["Generator-p_nom"]
2023-03-09 13:24:25 +00:00
ext_i = n.generators.query("p_nom_extendable")
ext_carrier_i = xr.DataArray(ext_i.carrier.rename_axis("Generator-ext"))
lhs = p_nom.groupby(ext_carrier_i).sum()
index = mincaps.index.intersection(lhs.indexes["carrier"])
rhs = mincaps[index].rename_axis("carrier")
2023-03-09 13:24:25 +00:00
n.model.add_constraints(lhs >= rhs, name="bau_mincaps")
# TODO: think about removing or make per country
def add_SAFE_constraints(n, config):
2023-03-09 13:24:25 +00:00
"""
Add a capacity reserve margin of a certain fraction above the peak demand.
Renewable generators and storage do not contribute. Ignores network.
Parameters
----------
n : pypsa.Network
config : dict
Example
-------
config.yaml requires to specify opts:
scenario:
opts: [Co2L-SAFE-24H]
electricity:
SAFE_reservemargin: 0.1
Which sets a reserve margin of 10% above the peak demand.
"""
peakdemand = n.loads_t.p_set.sum(axis=1).max()
margin = 1.0 + config["electricity"]["SAFE_reservemargin"]
reserve_margin = peakdemand * margin
# TODO: do not take this from the plotting config!
conv_techs = config["plotting"]["conv_techs"]
2023-03-09 13:24:25 +00:00
ext_gens_i = n.generators.query("carrier in @conv_techs & p_nom_extendable").index
p_nom = n.model["Generator-p_nom"].loc[ext_gens_i]
lhs = p_nom.sum()
exist_conv_caps = n.generators.query(
"~p_nom_extendable & carrier in @conv_techs"
).p_nom.sum()
2023-03-09 13:24:25 +00:00
rhs = reserve_margin - exist_conv_caps
n.model.add_constraints(lhs >= rhs, name="safe_mintotalcap")
def add_operational_reserve_margin(n, sns, config):
2023-03-09 13:24:25 +00:00
"""
Build reserve margin constraints based on the formulation given in
https://genxproject.github.io/GenX/dev/core/#Reserves.
2023-03-09 13:24:25 +00:00
Parameters
----------
n : pypsa.Network
sns: pd.DatetimeIndex
2023-03-09 13:24:25 +00:00
config : dict
Example:
--------
config.yaml requires to specify operational_reserve:
operational_reserve: # like https://genxproject.github.io/GenX/dev/core/#Reserves
activate: true
epsilon_load: 0.02 # percentage of load at each snapshot
epsilon_vres: 0.02 # percentage of VRES at each snapshot
contingency: 400000 # MW
"""
reserve_config = config["electricity"]["operational_reserve"]
EPSILON_LOAD = reserve_config["epsilon_load"]
EPSILON_VRES = reserve_config["epsilon_vres"]
CONTINGENCY = reserve_config["contingency"]
# Reserve Variables
2023-03-09 13:24:25 +00:00
n.model.add_variables(
0, np.inf, coords=[sns, n.generators.index], name="Generator-r"
)
reserve = n.model["Generator-r"]
lhs = reserve.sum("Generator")
# Share of extendable renewable capacities
ext_i = n.generators.query("p_nom_extendable").index
vres_i = n.generators_t.p_max_pu.columns
if not ext_i.empty and not vres_i.empty:
capacity_factor = n.generators_t.p_max_pu[vres_i.intersection(ext_i)]
p_nom_vres = (
2023-03-09 13:24:25 +00:00
n.model["Generator-p_nom"]
.loc[vres_i.intersection(ext_i)]
2023-03-09 13:24:25 +00:00
.rename({"Generator-ext": "Generator"})
)
lhs = lhs + (p_nom_vres * (-EPSILON_VRES * capacity_factor)).sum()
2023-03-09 13:24:25 +00:00
# Total demand per t
demand = n.loads_t.p_set.sum(axis=1)
# VRES potential of non extendable generators
capacity_factor = n.generators_t.p_max_pu[vres_i.difference(ext_i)]
renewable_capacity = n.generators.p_nom[vres_i.difference(ext_i)]
potential = (capacity_factor * renewable_capacity).sum(axis=1)
# Right-hand-side
rhs = EPSILON_LOAD * demand + EPSILON_VRES * potential + CONTINGENCY
2023-03-09 13:24:25 +00:00
n.model.add_constraints(lhs >= rhs, name="reserve_margin")
reserve = n.model["Generator-r"]
lhs = n.model.constraints["Generator-fix-p-upper"].lhs
lhs = lhs + reserve.loc[:, lhs.coords["Generator-fix"]].drop("Generator")
rhs = n.model.constraints["Generator-fix-p-upper"].rhs
n.model.add_constraints(lhs <= rhs, name="Generator-fix-p-upper-reserve")
2023-03-09 13:24:25 +00:00
lhs = n.model.constraints["Generator-ext-p-upper"].lhs
lhs = lhs + reserve.loc[:, lhs.coords["Generator-ext"]].drop("Generator")
rhs = n.model.constraints["Generator-ext-p-upper"].rhs
n.model.add_constraints(lhs >= rhs, name="Generator-ext-p-upper-reserve")
def add_battery_constraints(n):
2023-03-09 13:24:25 +00:00
"""
Add constraint ensuring that charger = discharger, i.e.
1 * charger_size - efficiency * discharger_size = 0
"""
if not n.links.p_nom_extendable.any():
return
discharger_bool = n.links.index.str.contains("battery discharger")
charger_bool = n.links.index.str.contains("battery charger")
dischargers_ext = n.links[discharger_bool].query("p_nom_extendable").index
chargers_ext = n.links[charger_bool].query("p_nom_extendable").index
eff = n.links.efficiency[dischargers_ext].values
lhs = (
n.model["Link-p_nom"].loc[chargers_ext]
- n.model["Link-p_nom"].loc[dischargers_ext] * eff
2023-03-09 13:24:25 +00:00
)
n.model.add_constraints(lhs == 0, name="Link-charger_ratio")
def add_chp_constraints(n):
electric = (
n.links.index.str.contains("urban central")
& n.links.index.str.contains("CHP")
& n.links.index.str.contains("electric")
)
heat = (
n.links.index.str.contains("urban central")
& n.links.index.str.contains("CHP")
& n.links.index.str.contains("heat")
2023-03-09 13:24:25 +00:00
)
electric_ext = n.links[electric].query("p_nom_extendable").index
heat_ext = n.links[heat].query("p_nom_extendable").index
electric_fix = n.links[electric].query("~p_nom_extendable").index
heat_fix = n.links[heat].query("~p_nom_extendable").index
p = n.model["Link-p"] # dimension: [time, link]
# output ratio between heat and electricity and top_iso_fuel_line for extendable
if not electric_ext.empty:
p_nom = n.model["Link-p_nom"]
lhs = (
p_nom.loc[electric_ext]
* (n.links.p_nom_ratio * n.links.efficiency)[electric_ext].values
- p_nom.loc[heat_ext] * n.links.efficiency[heat_ext].values
)
n.model.add_constraints(lhs == 0, name="chplink-fix_p_nom_ratio")
2023-03-09 13:24:25 +00:00
rename = {"Link-ext": "Link"}
lhs = (
p.loc[:, electric_ext]
+ p.loc[:, heat_ext]
- p_nom.rename(rename).loc[electric_ext]
)
n.model.add_constraints(lhs <= 0, name="chplink-top_iso_fuel_line_ext")
# top_iso_fuel_line for fixed
if not electric_fix.empty:
lhs = p.loc[:, electric_fix] + p.loc[:, heat_fix]
rhs = n.links.p_nom[electric_fix]
n.model.add_constraints(lhs <= rhs, name="chplink-top_iso_fuel_line_fix")
# back-pressure
if not electric.empty:
lhs = (
p.loc[:, heat] * (n.links.efficiency[heat] * n.links.c_b[electric].values)
- p.loc[:, electric] * n.links.efficiency[electric]
)
n.model.add_constraints(lhs <= rhs, name="chplink-backpressure")
def add_pipe_retrofit_constraint(n):
2023-03-09 13:24:25 +00:00
"""
Add constraint for retrofitting existing CH4 pipelines to H2 pipelines.
"""
gas_pipes_i = n.links.query("carrier == 'gas pipeline' and p_nom_extendable").index
h2_retrofitted_i = n.links.query(
"carrier == 'H2 pipeline retrofitted' and p_nom_extendable"
).index
if h2_retrofitted_i.empty or gas_pipes_i.empty:
return
p_nom = n.model["Link-p_nom"]
CH4_per_H2 = 1 / n.config["sector"]["H2_retrofit_capacity_per_CH4"]
lhs = p_nom.loc[gas_pipes_i] + CH4_per_H2 * p_nom.loc[h2_retrofitted_i]
rhs = n.links.p_nom[gas_pipes_i].rename_axis("Link-ext")
n.model.add_constraints(lhs == rhs, name="Link-pipe_retrofit")
def extra_functionality(n, snapshots):
2023-03-10 14:58:53 +00:00
"""
Collects supplementary constraints which will be passed to
``pypsa.optimization.optimize``.
If you want to enforce additional custom constraints, this is a good
location to add them. The arguments ``opts`` and
``snakemake.config`` are expected to be attached to the network.
"""
opts = n.opts
config = n.config
if "BAU" in opts and n.generators.p_nom_extendable.any():
add_BAU_constraints(n, config)
if "SAFE" in opts and n.generators.p_nom_extendable.any():
add_SAFE_constraints(n, config)
if "CCL" in opts and n.generators.p_nom_extendable.any():
add_CCL_constraints(n, config)
reserve = config["electricity"].get("operational_reserve", {})
if reserve.get("activate"):
add_operational_reserve_margin(n, snapshots, config)
for o in opts:
if "EQ" in o:
add_EQ_constraints(n, o)
add_battery_constraints(n)
add_pipe_retrofit_constraint(n)
def solve_network(n, config, opts="", **kwargs):
set_of_options = config["solving"]["solver"]["options"]
solver_options = (
config["solving"]["solver_options"][set_of_options] if set_of_options else {}
)
solver_name = config["solving"]["solver"]["name"]
cf_solving = config["solving"]["options"]
track_iterations = cf_solving.get("track_iterations", False)
min_iterations = cf_solving.get("min_iterations", 4)
max_iterations = cf_solving.get("max_iterations", 6)
2020-02-10 15:47:11 +00:00
# add to network for extra_functionality
n.config = config
n.opts = opts
2020-02-10 15:47:11 +00:00
skip_iterations = cf_solving.get("skip_iterations", False)
if not n.lines.s_nom_extendable.any():
skip_iterations = True
logger.info("No expandable lines found. Skipping iterative solving.")
if skip_iterations:
status, condition = n.optimize(
2023-03-09 13:24:25 +00:00
solver_name=solver_name,
extra_functionality=extra_functionality,
**solver_options,
2023-03-09 13:24:25 +00:00
**kwargs,
)
else:
status, condition = n.optimize.optimize_transmission_expansion_iteratively(
solver_name=solver_name,
track_iterations=track_iterations,
min_iterations=min_iterations,
max_iterations=max_iterations,
2023-03-09 13:24:25 +00:00
extra_functionality=extra_functionality,
**solver_options,
2023-03-09 13:24:25 +00:00
**kwargs,
)
2023-03-09 13:24:25 +00:00
if status != "ok":
logger.warning(
f"Solving status '{status}' with termination condition '{condition}'"
)
if "infeasible" in condition:
raise RuntimeError("Solving status 'infeasible'")
return n
if __name__ == "__main__":
2018-01-30 22:09:06 +00:00
if "snakemake" not in globals():
Introduce mocksnakemake which acutally parses Snakefile (#107) * rewrite mocksnakemake for parsing real Snakefile * continue add function to scripts * going through all scripts, setting new mocksnakemake * fix plotting scripts * fix build_country_flh * fix build_country_flh II * adjust config files * fix make_summary for tutorial network * create dir also for output * incorporate suggestions * consistent import of mocksnakemake * consistent import of mocksnakemake II * Update scripts/_helpers.py Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com> * Update scripts/_helpers.py Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com> * Update scripts/_helpers.py Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com> * Update scripts/_helpers.py Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com> * Update scripts/plot_network.py Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com> * Update scripts/plot_network.py Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com> * Update scripts/retrieve_databundle.py Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com> * use pathlib for mocksnakemake * rename mocksnakemake into mock_snakemake * revert change in data * Update scripts/_helpers.py Co-Authored-By: euronion <42553970+euronion@users.noreply.github.com> * remove setting logfile in mock_snakemake, use Path in configure_logging * fix fallback path and base_dir fix return type of make_io_accessable * reformulate mock_snakemake * incorporate suggestion, fix typos * mock_snakemake: apply absolute paths again, add assertion error *.py: make hard coded io path accessable for mock_snakemake * retrieve_natura_raster: use snakemake.output for fn_out * include suggestion * Apply suggestions from code review Co-Authored-By: Jonas Hörsch <jonas.hoersch@posteo.de> * linting, add return ad end of file * Update scripts/plot_p_nom_max.py Co-Authored-By: Jonas Hörsch <jonas.hoersch@posteo.de> * Update scripts/plot_p_nom_max.py fixes #112 Co-Authored-By: Jonas Hörsch <jonas.hoersch@posteo.de> * plot_p_nom_max: small correction * config.tutorial.yaml fix snapshots end * use techs instead of technology * revert try out from previous commit, complete replacing * change clusters -> clusts in plot_p_nom_max due to wildcard constraints of clusters * change clusters -> clusts in plot_p_nom_max due to wildcard constraints of clusters II
2019-12-09 20:29:15 +00:00
from _helpers import mock_snakemake
2022-06-30 06:39:03 +00:00
snakemake = mock_snakemake(
"solve_sector_network",
configfiles="test/config.overnight.yaml",
2023-03-09 13:24:25 +00:00
simpl="",
opts="",
2023-03-09 13:24:25 +00:00
clusters="5",
ll="v1.5",
sector_opts="CO2L0-24H-T-H-B-I-A-solar+p3-dist1",
planning_horizons="2030",
)
Add logging to logfiles to all snakemake workflow scripts. (#102) * Add logging to logfiles to all snakemake workflow scripts. * Fix missing quotation marks in Snakefile. * Apply suggestions from code review Co-Authored-By: Fabian Neumann <fabian.neumann@outlook.de> * Apply suggestions from code review Co-Authored-By: Fabian Neumann <fabian.neumann@outlook.de> * doc: fix _ec_ filenames in docs * Allow logging message format to be specified in config.yaml. * Add logging for Snakemake rule 'retrieve_databundle '. * Add limited logging to STDERR only for retrieve_*.py scripts. * Import progressbar module only on demand. * Fix logging to file and enable concurrent printing to STDERR for most scripts. * Add new 'logging_format' option to Travis CI test config.yaml. * Add missing parenthesis (bug fix) and cross-os compatible paths. * Fix typos in messages. * Use correct log files for logging (bug fix). * doc: fix line references * config: logging_format in all configs * doc: add doc for logging_format * environment: update to powerplantmatching 0.4.3 * doc: update line references for tutorial.rst * Change logging configuration scheme for config.yaml. * Add helper function for doing basic logging configuration. * Add logpath for prepare_links_p_nom rule. * Outsource basic logging configuration for all scripts to _helper submodule. * Update documentation for changed config.yaml structure. Instead of 'logging_level' and 'logging_format', now 'logging' with subcategories is used. * _helpers: Change configure_logging signature.
2019-11-28 07:22:52 +00:00
configure_logging(snakemake)
update_config_with_sector_opts(snakemake.config, snakemake.wildcards.sector_opts)
Add logging to logfiles to all snakemake workflow scripts. (#102) * Add logging to logfiles to all snakemake workflow scripts. * Fix missing quotation marks in Snakefile. * Apply suggestions from code review Co-Authored-By: Fabian Neumann <fabian.neumann@outlook.de> * Apply suggestions from code review Co-Authored-By: Fabian Neumann <fabian.neumann@outlook.de> * doc: fix _ec_ filenames in docs * Allow logging message format to be specified in config.yaml. * Add logging for Snakemake rule 'retrieve_databundle '. * Add limited logging to STDERR only for retrieve_*.py scripts. * Import progressbar module only on demand. * Fix logging to file and enable concurrent printing to STDERR for most scripts. * Add new 'logging_format' option to Travis CI test config.yaml. * Add missing parenthesis (bug fix) and cross-os compatible paths. * Fix typos in messages. * Use correct log files for logging (bug fix). * doc: fix line references * config: logging_format in all configs * doc: add doc for logging_format * environment: update to powerplantmatching 0.4.3 * doc: update line references for tutorial.rst * Change logging configuration scheme for config.yaml. * Add helper function for doing basic logging configuration. * Add logpath for prepare_links_p_nom rule. * Outsource basic logging configuration for all scripts to _helper submodule. * Update documentation for changed config.yaml structure. Instead of 'logging_level' and 'logging_format', now 'logging' with subcategories is used. * _helpers: Change configure_logging signature.
2019-11-28 07:22:52 +00:00
opts = (snakemake.wildcards.opts + "-" + snakemake.wildcards.sector_opts).split("-")
opts = [o for o in opts if o != ""]
solve_opts = snakemake.config["solving"]["options"]
2018-02-01 11:42:56 +00:00
np.random.seed(solve_opts.get("seed", 123))
fn = getattr(snakemake.log, "memory", None)
with memory_logger(filename=fn, interval=30.0) as mem:
if "overrides" in snakemake.input.keys():
overrides = override_component_attrs(snakemake.input.overrides)
n = pypsa.Network(
snakemake.input.network, override_component_attrs=overrides
)
else:
n = pypsa.Network(snakemake.input.network)
n = prepare_network(n, solve_opts, config=snakemake.config)
n = solve_network(
n, config=snakemake.config, opts=opts, log_fn=snakemake.log.solver
)
2022-06-30 06:39:03 +00:00
n.meta = dict(snakemake.config, **dict(wildcards=dict(snakemake.wildcards)))
n.export_to_netcdf(snakemake.output[0])
logger.info("Maximum memory usage: {}".format(mem.mem_usage))