bjoern/pypsa-eur
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
439 Commits 1 Branch 0 Tags 133 MiB
f5175713af
Commit Graph

73 Commits

Author SHA1 Message Date
Fabian Neumann
f5175713af Snakefile: load default config if config.yaml does not exist 2019-11-28 16:33:33 +01:00
euronion
85c356297a Add logging to logfiles to all snakemake workflow scripts. (#102) 
* Add logging to logfiles to all snakemake workflow scripts.

* Fix missing quotation marks in Snakefile.

* Apply suggestions from code review

Co-Authored-By: Fabian Neumann <fabian.neumann@outlook.de>

* Apply suggestions from code review

Co-Authored-By: Fabian Neumann <fabian.neumann@outlook.de>

* doc: fix _ec_ filenames in docs

* Allow logging message format to be specified in config.yaml.

* Add logging for Snakemake rule 'retrieve_databundle '.

* Add limited logging to STDERR only for retrieve_*.py scripts.

* Import progressbar module only on demand.

* Fix logging to file and enable concurrent printing to STDERR for most scripts.

* Add new 'logging_format' option to Travis CI test config.yaml.

* Add missing parenthesis (bug fix) and cross-os compatible paths.

* Fix typos in messages.

* Use correct log files for logging (bug fix).

* doc: fix line references

* config: logging_format in all configs

* doc: add doc for logging_format

* environment: update to powerplantmatching 0.4.3

* doc: update line references for tutorial.rst

* Change logging configuration scheme for config.yaml.

* Add helper function for doing basic logging configuration.

* Add logpath for prepare_links_p_nom rule.

* Outsource basic logging configuration for all scripts to _helper submodule.

* Update documentation for changed config.yaml structure.

Instead of 'logging_level' and 'logging_format', now 'logging' with subcategories is used.

* _helpers: Change configure_logging signature.
 2019-11-28 08:22:52 +01:00
Fabian Neumann
efb5f064b7 Snakefile: use normpath for solver logs 
* mailing list: pypsa-eur_questions related: generated data and solver conf issues, 2019/11/19, 19:13
 2019-11-22 15:13:46 +01:00
Fabian Neumann
81d9d2ab4d
battery+H2: option for stores and links instead of storage_units (#97) 
* add_electricity: stores and links instead of storageunits

* move code for attaching in add_extra_stores.py

* Update Snakefile

Co-Authored-By: Fabian Neumann <fabian.neumann@outlook.de>

* Update Snakefile

Co-Authored-By: Fabian Neumann <fabian.neumann@outlook.de>

* Update Snakefile

Co-Authored-By: Fabian Neumann <fabian.neumann@outlook.de>

* rename to add_extra_components.py

* incorporate suggestions

* Snakefile: update script name

* extra_components: import _add_missing_carriers_from_costs

* Add '_ec" infix in Snakefile

* Snakefile add infix at missing spot

* remove unintendetely inserted data.bundle, correct import in add_extra_components

* environment: fix pyyaml version

* set powerplantmatching as conda-dependency

* environment fix minimal ppm version

* environment reinsert pyyaml

* environment: carry over changes to environment.docs.yaml

* extra_components: add carriers

* update docstring

* Update scripts/add_extra_components.py

* Update scripts/add_extra_components.py

* tutorial: fix hyperlink [skip travis]

* update release notes

* update documentation

* add_electricity: merge removal of suptech
 2019-11-19 19:36:28 +01:00
JanFrederickUnnewehr
6cde3e902b own databundle (#96) 
* config.default.yaml

* retrieve_databundle: docs, config and defaults

* fix capitalisation of true/false
 2019-11-19 15:56:44 +01:00
Fabian Neumann
c857f4ce95 retrieve_cutout: execute retrieve cutout only once 2019-11-11 16:51:25 +01:00
Fabian Neumann
a35dbc067c build_natura: add cutouts as input (closes #95) 2019-11-11 16:29:27 +01:00
Fabian Neumann
fd4e43c085 Snakefile: fix inputs for retrieve_databundle 2019-11-06 17:50:57 +01:00
Fabian Neumann
99cb4a4354
Merge pull request #86 from PyPSA/workflow 
Integrate data retrieving into snakemake workflow
 2019-11-06 16:06:11 +01:00
Fabian Neumann
0fb57601bf doc: document powerplantmatching changes from #84 2019-11-06 15:59:34 +01:00
Fabian Neumann
82392c6c07 Merge branch 'co2limit' of https://github.com/fneum/pypsa-eur into fneum-co2limit 2019-11-06 11:19:41 +01:00
Fabian
3597f931cf Snakefile remove space 2019-11-05 14:57:59 +01:00
Fabian
594489ef22 snakefile: remove commented files 2019-11-05 14:55:51 +01:00
Fabian
b1a38229f4 ignore sort warning 2019-11-05 14:29:15 +01:00
Fabian
f6e44d2805 include data retrieving in workflow 2019-11-05 12:53:21 +01:00
FabianHofmann
5cd7a46339 fully enable powerplantmatching 2019-10-31 16:45:13 +01:00
FabianHofmann
0c777fc431
Merge pull request #71 from PyPSA/update_link_length 
[WIP] capital cost and underwater fraction update in clustering
 2019-10-31 11:27:58 +01:00
Fabian Neumann
5ea60a1ee0 modify to run without GEBCO and without hydro 2019-10-03 12:32:49 +02:00
Fabian Hofmann
16ff2b51bd fix small issues 2019-09-23 16:44:48 +02:00
Fabian Neumann
c99480e0f3 add co2 emissions reduction factor to wildcard 2019-06-18 11:50:54 +02:00
Jonas Hoersch
31aefae93c make_summary: Allow mixing volume and cost limit 2019-02-22 17:08:43 +01:00
Jonas Hoersch
50abd6c0c9 build_country_flh: Add rule for per-country full load hours tables and plots 2019-02-15 18:27:04 +01:00
Jonas Hoersch
15e986a0ec make_summary: Update summary creation for new line volume/cost constraints 
Fixes #4.
 2019-02-10 16:08:39 +01:00
Jonas Hoersch
c5b093380b solve_network: Slightly increase requested memory 
Encountered out-of-memory kills for the 512_lc networks.
 2019-02-06 11:49:06 +01:00
Jonas Hoersch
5dfeb6efbc trace_solve_network: Save intermediate line capacities 2019-02-04 18:32:56 +01:00
Jonas Hoersch
60dda9f909 plot_p_nom_max: Add plots for the cumulative renewable expansion potential 
The wildcards `clusters` and `technology` accept lists of values separated by ','
 2019-02-03 14:22:12 +01:00
Jonas Hoersch
dc134e88ab Add a separate rule for preparing all networks 2019-02-03 14:00:45 +01:00
Jonas Hoersch
dde0d91168 Include an optional total line cost constraint 
elec_s_37_lc1.25_... adds a constraint on the total line cost for an extension
by a 25%, compare with elec_s_37_lv1.25_... for the line volume limit.

`ll` is an acronym for line limit.
 2019-02-03 13:50:05 +01:00
Jonas Hoersch
f70e0304e5 Fix plotting settings 2019-01-16 12:08:46 +01:00
Jonas Hoersch
448ba651a0 build_renewable_profiles: Compute underwater_fraction 2018-12-11 17:30:25 +01:00
Jonas Hoersch
a035bee9c6 Merge branch 'windcosts' 2018-12-11 16:09:24 +01:00
Jonas Hoersch
802261eedd Use GLAES library by FZJ-IEK-3 for computing GIS potentials 2018-12-11 15:39:42 +01:00
Jonas Hörsch
761b1b998f make_summary: Add summary about capacities and allow single country summaries 2018-11-12 22:00:50 +01:00
Jonas Hörsch
c5e8c9ad25 add_electricity: Extend the assignment of max_hours 2018-11-12 22:00:46 +01:00
Jonas Hörsch
9c64034950 simplify_network: Add simplification cost updates 
Depends on pypsa commit https://github.com/PyPSA/PyPSA/commit/9ec406252add1b6
 2018-10-27 03:41:38 +02:00
Jonas Hörsch
b30f4a2fff Update technology costs for wind 
Calculate grid extension costs for offshore based on weighted average distance
from weather cell to substation.
 2018-10-26 21:32:04 +02:00
Jonas Hörsch
ee6085ed7e Snakefile: Comment out group settings 
They are still experimental and lead to difficult to understand bugs.
For now we are advising to use submit the whole model generation up until the
prepare_network step on one compute node, i.e. submit a job running

snakemake --until prepare_network --resources mem=60000 -j4

for a 64GB compute node.

And only run the solve_network rule in parallel on the cluster.
 2018-10-26 18:39:09 +02:00
Jonas Hörsch
5b201de7d4 {make,plot}_summary: Add rules to flexibly create summaries and plot them 2018-10-26 10:28:54 +02:00
Jonas Hörsch
0070b1c87c plot_network: Add functionality to produce a network plot of each solution 2018-10-25 16:43:24 +02:00
Jonas Hörsch
0c114c403c config: Update network_plot settings 2018-10-22 23:39:14 +02:00
Jonas Hörsch
d3d52cc60f Snakefile: Update memory requirements of solve_network 2018-10-22 23:28:56 +02:00
Jonas Hörsch
2b68ebadd5 Update Snakefile and cluster.yaml 2018-10-22 23:27:18 +02:00
Jonas Hörsch
1df85468aa add_electricity, prepare_network: Enable lv='inf' and remove converter link costs 2018-10-22 23:23:51 +02:00
Jonas Hörsch
52e214a3b2 base_network: Add links from TYNDP 
Manually collected from https://tyndp.entsoe.eu/. Deactivated by default.
 2018-09-24 20:19:47 +02:00
Jonas Hörsch
1678f319d8 prepare_network: Allow lv=inf to designate a fully cost-optimal expansion 2018-09-24 18:43:24 +02:00
Jonas Hörsch
748f4c9f36 Fix memory requirement for build_renewable_potentials 
if distance is set one needs up to 24GB.
 2018-09-24 18:43:24 +02:00
Jonas Hörsch
1069a52801 Reinstate cluster.yaml to define walltime, feature and log 
resources in rules have to be integer-valued and parameters are not aggregated
to groups. making cluster.yaml the only mechanism compatible with rules and
group.
 2018-09-20 14:26:38 +02:00
Jonas Hörsch
83e648b4c2 Update Snakefile for moab/torque profile 
Remove cluster.yaml and snakemake_cluster, which are superseeded by profiles.
 2018-09-19 15:21:11 +02:00
Jonas Hörsch
2372bb156d Prepare Snakefile and scripts for other solvers 2018-09-14 11:22:13 +02:00
Jonas Hörsch
fac30eb5a7 Snakefile: Set groups to minimize number of HPC jobs 2018-09-04 00:23:43 +02:00