Commit Graph

107 Commits

Author SHA1 Message Date
toniseibold
3298572ced new energy totals from eurostat 2023 report scaling JRC IDEES data from 2015 2024-02-23 13:53:28 +01:00
Fabian Neumann
719d4dad23 Merge branch 'master' into scenario-management 2024-02-20 12:08:54 +01:00
Fabian Neumann
16e2286188 update copyright years to 2024 2024-02-19 16:21:48 +01:00
pre-commit-ci[bot]
e0b6ebd174 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-02-12 10:54:15 +00:00
Fabian Neumann
40b27b4107 use set_scenario_config everywhere 2024-02-12 11:53:20 +01:00
Fabian Neumann
9865a97089 apply automated formatting 2024-01-22 09:29:32 +01:00
Fabian Neumann
2183e742b2 add release notes, minor code improvements 2024-01-22 09:18:26 +01:00
Tom Brown
1a477d6b32 move calculation of district heating share to its own script
Now the script build_district_heat_share.py does what the old function
create_nodes_for_heating() in prepare_sector_networks.py did.

There is no need to build nodes lists for each heating sector, since
all nodes have district heating now.
2024-01-22 08:01:31 +01:00
Tom Brown
bd8a5ecf2b build_energy_totals: output district heat share to separate file 2024-01-22 08:01:31 +01:00
Fabian
b1d21813af fix import order 2024-01-19 12:17:45 +01:00
martacki
55eddb6e1d Norway currently ignored when building energy totals 2024-01-17 10:01:10 +01:00
Fabian Neumann
c3bb41c900 Merge branch 'master' into multiyear 2024-01-03 15:14:27 +01:00
Fabian Neumann
30c1a1c857 address deprecation warnings 2024-01-02 16:31:48 +01:00
Sourcery AI
3c36c60bdd 'Refactored by Sourcery' 2023-10-08 09:20:36 +00:00
Fabian Neumann
ddd1840981 Merge remote-tracking branch 'origin/master' into multiyear 2023-07-26 10:23:32 +02:00
Fabian
1d10073514 review params implementation; reproduce objective values in CI 2023-06-15 18:52:25 +02:00
virio-andreyana
8af1fe5649
replace the word "config" to "params" in functions 2023-05-17 19:25:45 +02:00
virio-andreyana
8208ac033b
convert param back to config for "logging" and "run", made "enable" more specific 2023-05-17 18:43:30 +02:00
virio-andreyana
1fc48d8753
using python algorithm to edit rules and script 2023-05-15 10:33:17 +02:00
pre-commit-ci[bot]
add135fe05 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-04-30 08:53:00 +00:00
Fabian Neumann
6acd5da4d4 Merge branch 'multiyear-sec' into multiyear 2023-04-29 18:49:49 +02:00
Fabian
5c8e0080c8 pre-commit: update docformatter version 2023-04-21 10:11:44 +02:00
pre-commit-ci[bot]
460bbd080f [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-04-10 22:02:35 +00:00
Fabian Neumann
abe4df543e add minimal description of all scripts 2023-03-09 12:45:43 +01:00
pre-commit-ci[bot]
3402908834 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-03-09 10:39:59 +00:00
Fabian Neumann
22846d7b1b sector-coupled: run on subset of selected countries 2023-03-09 11:04:41 +01:00
Fabian
104fc68fb0 allow to disable progressbar 2023-03-07 20:37:47 +01:00
Fabian
ba6e0c2e4a move snakemake object out of imap process functions 2023-03-07 08:59:01 +01:00
Fabian Neumann
541b83b049 merge helper.py into _helpers.py 2023-03-06 19:16:37 +01:00
Fabian Neumann
3bdf3b6f5e use mock_snakemake from pypsa-eur 2023-03-06 19:09:45 +01:00
Fabian Neumann
93d9e13f52 add spdx identifiers to scripts 2023-03-06 18:49:23 +01:00
Fabian Neumann
ee5a0c4e73 conform to pypsa-eur logging level definition 2023-03-06 14:27:15 +01:00
pre-commit-ci[bot]
13769f90af [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2023-03-06 08:27:46 +00:00
virio-andreyana
d237e60b25 Replacing print statement with logging 2023-02-23 10:30:32 +01:00
Fabian
19c35f8a6a use muting of excel read with context manager 2023-02-21 23:20:02 +01:00
virio-andreyana
f276b130bd Hide uninformative text caused by subprocess outputs 2023-02-18 13:47:34 +01:00
lisazeyen
f10628930e move snakemake dependencies out for co2limit cb option to work 2022-08-01 15:21:11 +02:00
Fabian Neumann
0bad2097b0 interpolate heat demands for years outside 2007-2015 2022-07-25 14:35:54 +02:00
Fabian Neumann
614909201a energy_totals: transport sector requires specific year 2022-07-23 17:06:14 +02:00
Fabian Neumann
f70f006a47 energy_totals: co2 calculation with eurostat 2021 data 2022-07-23 17:05:56 +02:00
Fabian Neumann
ba8b3f9ce7 energy_totals: combined energy totals for all years 2022-07-23 17:05:07 +02:00
Fabian Neumann
eee6651d12 energy_totals: fetch all years 2000-2015 from IDEES 2022-07-23 11:19:37 +02:00
Fabian Neumann
d47a9aac72 energy_totals: update eurostat to 2021 2022-07-23 10:37:32 +02:00
lisazeyen
017c188518 fix typo for district heating with capital D 2021-10-04 09:23:23 +02:00
Fabian Neumann
2e6e9c6802
Merge branch 'master' into agriculture-energy-co2 2021-10-02 10:51:28 +02:00
Fabian Neumann
c0adb51454 decapitalize columns in build_energy_totals 2021-10-02 10:03:40 +02:00
lisazeyen
8322350fee move calculation of district heating share to build_energy_totals 2021-09-29 14:36:56 +02:00
Fabian Neumann
a9196f001b fix 2 typos 2021-08-18 14:17:21 +02:00
Fabian Neumann
dd177df19f fix NL hardcode typo 2021-08-16 16:26:20 +02:00
Fabian Neumann
67ee6cb86e first step towards agriculture, forestry and fishing 2021-07-06 18:32:35 +02:00
Fabian Neumann
1fc1d2a17d
Revision complete (#139)
* ammonia_production: minor cleaning and move into __main__ (#106)

* biomass_potentials: code cleaning and automatic country index inferral (#107)

* Revision: build energy totals (#111)

* blacken

* energy_totals: preliminaries

* energy_totals: update build_swiss

* energy_totals: update build_eurostat

* energy_totals: update build_idees

* energy_totals: update build_energy_totals

* energy_totals: update build_eea_co2

* energy_totals: update build_eurostat_co2

* energy_totals: update build_co2_totals

* energy_totals: update build_transport_data

* energy_totals: add tqdm progressbar to idees

* energy_totals: adjust __main__ section

* energy_totals: handle inputs via Snakefile and config

* energy_totals: handle data and emissions year via config

* energy_totals: fix reading in eurostat for different years

* energy_totals: fix erroneous drop duplicates
This caused problems for waste management in HU and SI

* energy_totals: make scope selection of CO2 or GHG a config option

* Revision: build industrial production per country (#114)

* industry-ppc: format

* industry-ppc: rewrite for performance

* industry-ppc: move reference year to config

* industry-ppct: tidy up and format (#115)

* remove stale industry demand rules (#116)

* industry-epc: rewrite for performance (#117)

* Revision: industrial distribution key (#118)

* industry-distribution: first tidying

* industry-distribution: first tidying

* industry-distribution: fix syntax

* Revision: industrial energy demand per node today (#119)

* industry-epn: minor code cleaning

* industry-epn: remove accidental artifact

* industry-epn: remove accidental artifact II

* industry-ppn: code cleaning (#120)

* minor code cleaning (#121)

* Revision: industry sector ratios (#122)

* sector-ratios: basic reformatting

* sector-ratios: add new read_excel function that filters year already

* sector-ratios: rename jrc to idees

* sector-ratios: rename conv_factor to toe_to_MWh

* sector-ratios: modularise into functions

* Move overriding of component attributes to function and into data (#123)

* move overriding of component attributes to central function and store in separate folder

* fix return of helper.override_component_attrs

* prepare: fix accidental syntax error

* override_component_attrs: bugfix that aligns with pypsa components

* Revision: build population layout (#108)

* population_layouts: move inside __main__ and blacken

* population_layouts: misc code cleaning and multiprocessing

* population_layouts: fix fill_values assignment of urban fractions

* population_layouts: bugfig for UK-GB naming ambiguity

* population_layouts: sort countries alphabetically for better overview

* config: change path to atlite cutout

* Revision: build clustered population layouts (#112)

* population_layouts: move inside __main__ and blacken

* population_layouts: misc code cleaning and multiprocessing

* population_layouts: fix fill_values assignment of urban fractions

* population_layouts: bugfig for UK-GB naming ambiguity

* population_layouts: sort countries alphabetically for better overview

* cl_pop_layout: blacken

* cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning

* cl_pop_layout: add fraction column which is repeatedly calculated downstream

* Revision: build various heating-related time series (#113)

* population_layouts: move inside __main__ and blacken

* population_layouts: misc code cleaning and multiprocessing

* population_layouts: fix fill_values assignment of urban fractions

* population_layouts: bugfig for UK-GB naming ambiguity

* population_layouts: sort countries alphabetically for better overview

* cl_pop_layout: blacken

* cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning

* gitignore: add .vscode

* heating_profiles: update to new atlite and move into __main__

* heating_profiles: remove extra cutout

* heating_profiles: load regions with .buffer(0) and remove clean_invalid_geometries

* heating_profiles: load regions with .buffer(0) before squeeze()

* heating_profiles: account for transpose of dataarray

* heating_profiles: account for transpose of dataarray in add_exiting_baseyear

* Reduce verbosity of Snakefile (2) (#128)

* tidy Snakefile light

* Snakefile: fix indents

* Snakefile: add missing RDIR

* tidy config by removing quotes and expanding lists (#109)

* bugfix: reorder squeeze() and buffer()

* plot/summary: cosmetic changes including: (#131)

- matplotlibrc for default style and backend
- remove unused config options
- option to configure geomap colors
- option to configure geomap bounds

* solve: align with pypsa-eur using ilopf (#129)

* tidy myopic code scripts (#132)

* use mock_snakemake from pypsa-eur (#133)

* Snakefile: add benchmark files to each rule

* Snakefile: only run build_retro_cost if endogenously optimised

* Snakefile: remove old {network} wildcard constraints

* WIP: Revision: prepare_sector_network (#124)

* population_layouts: move inside __main__ and blacken

* population_layouts: misc code cleaning and multiprocessing

* population_layouts: fix fill_values assignment of urban fractions

* population_layouts: bugfig for UK-GB naming ambiguity

* population_layouts: sort countries alphabetically for better overview

* cl_pop_layout: blacken

* cl_pop_layout: turn GeoDataFrame into GeoSeries + code cleaning

* move overriding of component attributes to central function and store in separate folder

* prepare: sort imports and remove six dependency

* prepare: remove add_emission_prices

* prepare: remove unused set_line_s_max_pu
This is a function from prepare_network

* prepare: remove unused set_line_volume_limit
This is a PyPSA-Eur function from prepare_network

* prepare: tidy add_co2limit

* remove six dependency

* prepare: tidy code first batch

* prepare: extend override_component_attrs to avoid hacky madd

* prepare: remove hacky madd() for individual components

* prepare: tidy shift function

* prepare: nodes and countries from n.buses not pop_layout

* prepare: tidy loading of pop_layout

* prepare: fix prepare_costs function

* prepare: optimise loading of traffic data

* prepare: move localizer into generate_periodic profiles

* prepare: some formatting of transport data

* prepare: eliminate some code duplication

* prepare: fix remove_h2_network
- only try to remove EU H2 store if it exists
- remove readding nodal Stores because they are never removed

* prepare: move cost adjustment to own function

* prepare: fix a syntax error

* prepare: add investment_year to get() assuming global variable

* prepare: move co2_totals out of prepare_data()

* Snakefile: remove unused prepare_sector_network inputs

* prepare: move limit p/s_nom of lines/links into function

* prepare: tidy add_co2limit file handling

* Snakefile: fix tabs

* override_component_attrs: add n/a defaults

* README: Add network picture to make scope clear

* README: Fix date of preprint (was too optimistic...)

* prepare: move some more config options to config.yaml

* prepare: runtime bugfixes

* fix benchmark path

* adjust plot ylims

* add unit attribute to bus, correct cement capture efficiency

* bugfix: land usage constrained missed inplace operation

Co-authored-by: Tom Brown <tom@nworbmot.org>

* add release notes

* remove old fix_branches() function

* deps: make geopy optional, remove unused imports

* increase default BarConvTol

* get ready for upcoming PyPSA release

* re-remove ** bug

* amend release notes

Co-authored-by: Tom Brown <tom@nworbmot.org>
2021-07-01 20:09:04 +02:00
Leon
55eb722ead
Make code more readable; remove misleading function arguments and add necessary ones (#94)
* Make code more readable; remove misleading function arguments and add necessary ones

* Update scripts/build_energy_totals.py

Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
2021-04-23 11:26:38 +02:00
martavp
ceba265c0a build_eea_co2() now reads version23 of UNFCCC inventory.
**This requires updating to UNFCCC_v23.csv in the data bundle**

This enables that the same function is used to read emissions in 2018, which are assumed to remain constant in 2019 and 2020 and subtracted from carbon budget (estimated from 2018 on).

I checked and 1990 emissions calculated with UNFCCC_v23 are very similar to those calculated with UNFCCC_v21 (<1% differences in all values at EU level).
Some countries show higher deviations, mainly in domestic aviation and navigation. I guess because those values started to be reported later and v23 should include more accurate values.
2021-01-12 10:57:09 +01:00
Fabian Neumann
a143ab7122 energy_totals: only fix 'BA' if in list of countries 2020-11-24 13:44:02 +01:00
Tom Brown
990ab035c3 Include light-rail and high-speed-rail in "electricity rail"
This then conforms with JRC IDEES totals
2019-07-19 10:19:21 +02:00
Tom Brown
b132193f7f Read in more sector detail for road, rail, aviation and navigation 2019-07-18 15:38:37 +02:00
Tom Brown
1b26c3c737 Copy data preparation scripts from pypsa-eur sector branch 2019-04-16 16:03:51 +02:00