Commit Graph

70 Commits

Author SHA1 Message Date
Fabian Neumann
1fc1d2a17d
Revision complete (#139)
* ammonia_production: minor cleaning and move into __main__ (#106)

* biomass_potentials: code cleaning and automatic country index inferral (#107)

* Revision: build energy totals (#111)

* blacken

* energy_totals: preliminaries

* energy_totals: update build_swiss

* energy_totals: update build_eurostat

* energy_totals: update build_idees

* energy_totals: update build_energy_totals

* energy_totals: update build_eea_co2

* energy_totals: update build_eurostat_co2

* energy_totals: update build_co2_totals

* energy_totals: update build_transport_data

* energy_totals: add tqdm progressbar to idees

* energy_totals: adjust __main__ section

* energy_totals: handle inputs via Snakefile and config

* energy_totals: handle data and emissions year via config

* energy_totals: fix reading in eurostat for different years

* energy_totals: fix erroneous drop duplicates
This caused problems for waste management in HU and SI

* energy_totals: make scope selection of CO2 or GHG a config option

* Revision: build industrial production per country (#114)

* industry-ppc: format

* industry-ppc: rewrite for performance

* industry-ppc: move reference year to config

* industry-ppct: tidy up and format (#115)

* remove stale industry demand rules (#116)

* industry-epc: rewrite for performance (#117)

* Revision: industrial distribution key (#118)

* industry-distribution: first tidying

* industry-distribution: first tidying

* industry-distribution: fix syntax

* Revision: industrial energy demand per node today (#119)

* industry-epn: minor code cleaning

* industry-epn: remove accidental artifact

* industry-epn: remove accidental artifact II

* industry-ppn: code cleaning (#120)

* minor code cleaning (#121)

* Revision: industry sector ratios (#122)

* sector-ratios: basic reformatting

* sector-ratios: add new read_excel function that filters year already

* sector-ratios: rename jrc to idees

* sector-ratios: rename conv_factor to toe_to_MWh

* sector-ratios: modularise into functions

* Move overriding of component attributes to function and into data (#123)

* move overriding of component attributes to central function and store in separate folder

* fix return of helper.override_component_attrs

* prepare: fix accidental syntax error

* override_component_attrs: bugfix that aligns with pypsa components

* Revision: build population layout (#108)

* population_layouts: move inside __main__ and blacken

* population_layouts: misc code cleaning and multiprocessing

* population_layouts: fix fill_values assignment of urban fractions

* population_layouts: bugfig for UK-GB naming ambiguity

* population_layouts: sort countries alphabetically for better overview

* config: change path to atlite cutout

* Revision: build clustered population layouts (#112)

* population_layouts: move inside __main__ and blacken

* population_layouts: misc code cleaning and multiprocessing

* population_layouts: fix fill_values assignment of urban fractions

* population_layouts: bugfig for UK-GB naming ambiguity

* population_layouts: sort countries alphabetically for better overview

* cl_pop_layout: blacken

* cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning

* cl_pop_layout: add fraction column which is repeatedly calculated downstream

* Revision: build various heating-related time series (#113)

* population_layouts: move inside __main__ and blacken

* population_layouts: misc code cleaning and multiprocessing

* population_layouts: fix fill_values assignment of urban fractions

* population_layouts: bugfig for UK-GB naming ambiguity

* population_layouts: sort countries alphabetically for better overview

* cl_pop_layout: blacken

* cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning

* gitignore: add .vscode

* heating_profiles: update to new atlite and move into __main__

* heating_profiles: remove extra cutout

* heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries

* heating_profiles: load regions with .buffer(0) before squeeze()

* heating_profiles: account for transpose of dataarray

* heating_profiles: account for transpose of dataarray in add_exiting_baseyear

* Reduce verbosity of Snakefile (2) (#128)

* tidy Snakefile light

* Snakefile: fix indents

* Snakefile: add missing RDIR

* tidy config by removing quotes and expanding lists (#109)

* bugfix: reorder squeeze() and buffer()

* plot/summary: cosmetic changes including: (#131)

- matplotlibrc for default style and backend
- remove unused config options
- option to configure geomap colors
- option to configure geomap bounds

* solve: align with pypsa-eur using ilopf (#129)

* tidy myopic code scripts (#132)

* use mock_snakemake from pypsa-eur (#133)

* Snakefile: add benchmark files to each rule

* Snakefile: only run build_retro_cost if endogenously optimised

* Snakefile: remove old {network} wildcard constraints

* WIP: Revision: prepare_sector_network (#124)

* population_layouts: move inside __main__ and blacken

* population_layouts: misc code cleaning and multiprocessing

* population_layouts: fix fill_values assignment of urban fractions

* population_layouts: bugfig for UK-GB naming ambiguity

* population_layouts: sort countries alphabetically for better overview

* cl_pop_layout: blacken

* cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning

* move overriding of component attributes to central function and store in separate folder

* prepare: sort imports and remove six dependency

* prepare: remove add_emission_prices

* prepare: remove unused set_line_s_max_pu
This is a function from prepare_network

* prepare: remove unused set_line_volume_limit
This is a PyPSA-Eur function from prepare_network

* prepare: tidy add_co2limit

* remove six dependency

* prepare: tidy code first batch

* prepare: extend override_component_attrs to avoid hacky madd

* prepare: remove hacky madd() for individual components

* prepare: tidy shift function

* prepare: nodes and countries from n.buses not pop_layout

* prepare: tidy loading of pop_layout

* prepare: fix prepare_costs function

* prepare: optimise loading of traffic data

* prepare: move localizer into generate_periodic profiles

* prepare: some formatting of transport data

* prepare: eliminate some code duplication

* prepare: fix remove_h2_network
- only try to remove EU H2 store if it exists
- remove readding nodal Stores because they are never removed

* prepare: move cost adjustment to own function

* prepare: fix a syntax error

* prepare: add investment_year to get() assuming global variable

* prepare: move co2_totals out of prepare_data()

* Snakefile: remove unused prepare_sector_network inputs

* prepare: move limit p/s_nom of lines/links into function

* prepare: tidy add_co2limit file handling

* Snakefile: fix tabs

* override_component_attrs: add n/a defaults

* README: Add network picture to make scope clear

* README: Fix date of preprint (was too optimistic...)

* prepare: move some more config options to config.yaml

* prepare: runtime bugfixes

* fix benchmark path

* adjust plot ylims

* add unit attribute to bus, correct cement capture efficiency

* bugfix: land usage constrained missed inplace operation

Co-authored-by: Tom Brown <tom@nworbmot.org>

* add release notes

* remove old fix_branches() function

* deps: make geopy optional, remove unused imports

* increase default BarConvTol

* get ready for upcoming PyPSA release

* re-remove ** bug

* amend release notes

Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 20:09:04 +02:00
Fabian Neumann
555002a35b fixup: remove remaining instances of {network} wildcard from recent PR 2021-05-19 13:57:44 +02:00
Tom Brown
8bfb1490c1
Merge branch 'master' into retro-updated 2021-04-30 18:12:20 +02:00
Fabian Neumann
a156d44f28
remove unused {network} wildcard (#104) 2021-04-29 16:52:35 +02:00
lisazeyen
36685bd016 update retro script to include ventilation, thermal bridges and heat gains. Add necessary input files in the Snakefile, remove no longer needed file 2021-02-24 16:51:58 +01:00
martavp
0eb69365eb Update Snakefile
Allow white space in sector_opts, e.g. to investigate sensitivity to cost of 'H2 Electrolysis'
2021-01-14 10:21:11 +01:00
martavp
0d64cf5b35 Allow decimals in sector_opts 2021-01-12 10:58:51 +01:00
martavp
4a681749d7 Add function build_carbon_budget()
For the myopic method, based on the carbon budget indicated in the config file (sector_opts), a CO2 limit is calculated for every planning_horizon following an exponential or beta decay.  A file with CO2 limit in every planning_horizon and a plot showing historical and planned CO2 emissions
are saved in the results
2021-01-12 10:57:29 +01:00
Tom Brown
1ea31f0e57 Redo investment-year-dependent exogenous retrofitting parameter
Specify as dictionary, use get_parameter to get correct value.

Also remove old parameter "space_heating_fraction" since this is
superceded by the new exogenous retro code.
2020-12-01 14:41:11 +01:00
Tom Brown
974155bd46 Merge branch 'master' into retro 2020-11-30 17:07:43 +01:00
Tom Brown
abbaa0d098 Move CO2 budgets from data/co2_budget.csv to dict in config.yaml
Strategy is too keep as much of configuration in config.yaml as
possible.

We also aim to allow exogenous investment-year-dependent
configurations to be done in a similar manner (e.g. share of district
heating or FCEV transport).
2020-11-30 13:21:38 +01:00
lisazeyen
d1282ba315 add option for endogenously retrofitting 2020-10-21 15:21:26 +02:00
lisazeyen
75ff2d19ee move transport data input path to Snakefile 2020-10-21 13:45:49 +02:00
Tom Brown
f0ef0e1136
Merge pull request #70 from eb5194/master
compatibility with pypsa-eur PR: clustermaps to csv
2020-10-21 13:01:17 +02:00
Tom Brown
80cbe98630 industry: add current nodal electricity demand to subtract later 2020-10-12 13:26:21 +02:00
Tom Brown
f2b347334d industry: build nodal energy demand from nodal production 2020-10-12 12:20:04 +02:00
Tom Brown
f0937e203b industry: separate code for distribution key from nodal production
This allows us to reuse the key for today's nodal energy demand from
industry.
2020-10-12 12:07:49 +02:00
Tom Brown
90f5349b7d Use hotmaps industrial database for distribution in each country
I.e. per sector geographical distribution of industrial facilities
within each country.

Drop facilities outside Europe and with no geocoordinates.

Use ETS emissions as a distribution key; where emissions data is
missing, substitute with an average for that sector and that country
(strong assumption).
2020-10-05 20:04:04 +02:00
eb5194
30adabbcaa Snakefile: input files in pypsa-eur became .csv 2020-10-02 12:22:25 +02:00
Tom Brown
de52711700 Fix calculation of solar rooftop potentials for "m" clusters
I.e. when the generators are clustered to the "simplified" network
resolution, but the grid is clustered further, e.g. by using the
clusters = 37m "m" option.
2020-09-25 15:25:41 +02:00
Tom Brown
6b964865e7
Merge pull request #69 from PyPSA/salt_cavern_potential
Salt cavern potential
2020-09-22 20:19:29 +02:00
lisazeyen
27e2584ae8 split hydrogen storage into underground storage in salt cavern for countries with potential, others overground hydrogen storage, switch costs from gas storage to corresponding hydrogen storage 2020-09-22 09:52:53 +02:00
Tom Brown
791a58fc6b Make selection of biomass classes (solid, biogas) more explicit
List classes in config.yaml, rather than integer selection in
build_biomass_potentials.py.

Also output potentials for all years and scenarios for analysis.
2020-09-21 18:35:45 +02:00
Tom Brown
1652cdc09e Move all intermediately-generated files from data to resources
Now data only contains external input data.

Now resources only contains intermediary files.
2020-09-21 13:06:58 +02:00
Tom Brown
c58f18667d Add non-EU28 energy demand per country and sector for today
Use the industrial production per sector and multiply with EU28
averages for energy per sector.
2020-09-07 19:12:47 +02:00
Tom Brown
783e026ebf Separate ammonia from basic chemicals in today's industry energy 2020-09-07 18:41:07 +02:00
Tom Brown
916aa5e0d9 Generate today's industry demand per ct and per sector
Only covers EU28 at the moment.

Uses the energy balances sheets from JRC-IDEES.
2020-09-07 16:48:06 +02:00
Tom Brown
f45b9a37ae Separate ammonia from other "Basic chemicals"
This allows us to control the substitution of natural gas for hydrogen
in NH3 production.

Remaining basic chemicals are olefins, BTX and chlorine.

For 2015 NH3 production, we use the USGS data source.
2020-08-28 19:13:18 +02:00
Tom Brown
b761281b3d Move changes to primary/secondary routes for Al/St to new script
This was handled before in industry_sector_ratios.csv which was
confusing.

Now industry_sector_ratios.csv represents the genuine energy
consumption per tonne of material for each industrial route
(MWh/tMaterial).

An new file is created with ktMaterial/a in
industrial_production_per_country_tomorrow.csv which contains changes
to the fraction of primary/secondary routes compared to today's
production in industrial_production_per_country.csv.

This is less confusing I think.
2020-08-26 16:13:01 +02:00
Tom Brown
851142fe0f Separate script for industrial production per ct from energy demand 2020-08-26 13:12:16 +02:00
Tom Brown
37f36047ca Industry demand: Also record material production per industry 2020-08-26 12:06:01 +02:00
Tom Brown
beefb57c25 Fix plotting of maps and balances; move log/benchmark to results/ 2020-08-21 12:15:01 +02:00
Tom Brown
d4368a966a Replace _ with - in technology_data and prenetworks_brownfield
Preferred by kubernetes.

Also updated the documentation accordingly.
2020-08-20 18:00:27 +02:00
Tom Brown
95e676828a Remove old data/costs.csv and dependencies on it 2020-08-20 14:26:39 +02:00
Tom Brown
6444f7fde9 Weight offshore connection costs with aggregation weight
Rather than taking a mean of the clustered connection costs.

Apply cost update also for overnight scenarios based on planning year.

Add land costs for onshore wind.
2020-08-19 12:41:17 +02:00
Tom Brown
8c196a7a21 Take care of CHPs when doing p_nom < threshold & extra_functionality
- add_brownfield.py: Have to make sure that for each CHP there is both
  a heat and electric link, but they have different p_nom for each
  CHP, so have to make sure we don't remove one without the other.

- solve_network.py: Make sure extra_functionality constraints for CHP
  power-heat feasibility graph also work for non-extendable CHPs.
2020-08-14 09:11:19 +02:00
martavp
18ff256811 Update cost for wind and solar, including the AC and DC connection costs for offshore.
In order to calculate connection costs, average values for underground_fraction and average_distance are calculated for all the buses in the initial network mapped to the clustered network.
2020-08-12 11:08:09 +02:00
Tom Brown
9012125585 Snakefile: Force configs to be copied before solving
This reinstates old behaviour for both myopic and overnight foresight
settings.

Also account for the fact that the planning_horizon is now integer in
config.yaml rather than a string.
2020-08-11 11:41:53 +02:00
Tom Brown
16b05a570d Distribute heating technologies within each country by population 2020-08-11 11:09:39 +02:00
Tom Brown
658f8ad00c Distribute existing power plants to the correct nodes
Previously they were distributed only by country to the first node in
the country.

Now conventional power plants are assigned to the correct node using
the bus map from PyPSA-Eur.

Wind and solar are distributed in each country by capacity factor.

The code has been refactored and a bug was fixed whereby total
capacities of wind and solar in each country were not correct.

Now the years in the config.yaml for myopic are integers not strings.
2020-08-10 20:30:29 +02:00
Tom Brown
b57a6c18f8 Make directory for costs files configurable
Default to parallel directory for technology_data repository.
2020-07-30 08:27:33 +02:00
martavp
d66105a6dd Load 2 level multindex costs file in prepares_sector_network rule 2020-07-14 20:03:08 +02:00
martavp
b23b7a143c Add carbon_budget_name and planning_horizons to the wildcards also for the overnight option (they are not used in this case). Also, rename prenetworks_bf to prenetworks_brownfield 2020-07-08 16:34:15 +02:00
martavp
e8e83de5d2 clean code, delete commented lines 2020-07-08 09:06:58 +02:00
martavp
25bb11cdfe deletes solve_baseyear rule, since it is not needed, base year is also solved with solve_myopic rule 2020-07-08 09:04:20 +02:00
martavp
d3b99a6a44 Modify snakefile to include myopic option. 2020-07-07 18:30:37 +02:00
Tom Brown
2bee13fe40 plot_summary: Plot energy / CO2 balances for each carrier
Take data from supply_energy.csv and plot the balances for each
carrier.

One summary for energy excludes CO2 flows (unlike graphs/energy.pdf).
2020-05-13 13:40:00 +02:00
Tom Brown
f9952b9aad Fixed for PyPSA-Eur 0.1.0, PyPSA 0.17.0 and plotting
This commit now work with PyPSA-Eur 0.1.0 (tested with commit
bb3477 from 14th April 2020).

Changes to line/link_widths/colors for plotting networks in PyPSA
0.17.0.

Other corrections to plotting code so it works with this version.

Include oil boilers in colors in config.yaml.
2020-04-29 18:23:18 +02:00
Tom Brown
62887b7870 Move solving memory from Snakefile to config.yaml 2020-03-25 14:51:37 +01:00
Tom Brown
5dc572c35e solve_network.py: Remove hacked installation of conda libraries 2020-03-04 18:10:57 +01:00