Model pipeline

The main functionality of oemof-B3 is a data processing pipeline which is managed using snakemake. The pipeline preprocesses raw data into scalar and time series resources that are used to build tabular datapackages representing single energy system optimization problems. These can be understood by oemof.tabular and optimized by oemof.solph. In the next steps, the results of the optimization are postprocessed and plotted. The individual steps are documented in detail in the following sections.

The subdirectories of oemof-B3 shown below contain configurations and data of the model. Examples contains pre-fabricated datapackages that can readily be optimized. The directory scenarios contains .yml-files defining individual scenarios. Raw data is not part of the repo and has to be provided in raw. You can use the raw data we provide. For instructions on how to automatically load the raw data into oemof-B3, see section section How to run the model.

Intermediate and final results as well as the corresponding logfiles are saved in results.

.
├── examples
├── oemof_b3
│     ├── config
│     ├── model
│     ├── schema
├── raw
├── results
├── scenarios

Raw data

Raw data from external source comes in different formats. It is not part of the model on GitHub, but has to be downloaded from Zenodo at the beginning. As a first step, preprocessing scripts in the model pipeline (see Prepare resources) convert it into the oemof-B3-resources-format, explained in the next section (see Schema). Raw data that represents model-own assumptions is provided in that format already. When providing raw data you have to follow some conventions, which are summarized in the section Conventions.

Schema

Raw data with model-own assumptions and oemof-B3-resources-format follow a common data schema defined in oemof_b3/schema/. There is a separate schema for scalars and for timeseries:

Scalars

id_scal	scenario_key	name	var_name	carrier	region	tech	type	var_value	var_unit	source	comment
int	string	string	string	string	string	string	string	string	string	string	string
Numerical index	Key of scenario that allows to pass values associated with specific scenario	Unique name of the oemof component	Variable name	Energy carrier	Region (‘ALL’ if applies to all regions - ‘TOTAL’ if applies to sum of regions)	Description	oemof.tabular type	Value (type is string because this field can contain numeric/bool/dictionary/etc)	Unit of the variable	Source of the data	Additional information or notes such as the license or scope of validity

Time series

id_ts

scenario_key

name

var_name

carrier

region

tech

type

timeindex_start

timeindex_stop

timeindex_resolution

series

var_unit

source

comment

int

string

string

string

string

string

string

string

string

string

string

string

string

string

string

Numerical index

Key

Unique name of the oemof component

Variable name

Energy carrier

Region

Technology description

oemof.tabular facade type

Start of the timeindex following ISO 8601 date and time format

Stop of the timeindex following ISO 8601 date and time format

Time increment

Series

Unit of the variable

Source of the data

Additional information or notes such as the license or scope of validity

Conventions

A few more conventions are important to know:

• Missing data is left empty.

• There is no unit transformation within the model, i.e. the user needs to ensure the consistency of units. In the plotting functions MW, MWh, EUR/MWh etc. are used as units. Therefore, please provide your data in just these units if you want to use the plotting functions. In the future we would like to drop this restriction.

• The parameters id_scal and id_ts are optional and will be added automatically if you do not specify them.

• The parameter name must be specified in a certain fixed concatenation of parameters: region-carrier-tech (example: B-biomass-gt).

• If region is set to ALL in the model-own assumptions, name is to be left blank. The name will be automatically added per region modelling the energy system.

• Different attributes can be set for var_name. oemof-B3 can process

  • attributes of the components in oemoflex (which component is assigned to which attributes can be found in chapter Overview of the oemoflex documentation) and

  • attributes needed to calculate annuized costs. For this, capacity_cost_overnight, storage_capacity_cost_overnight, wacc, lifetime and fixom_cost must be passed with var_name.

• Components can receive keywords for the electricity-gas-relation-constraint via the attribute output_parameters.

  • Keywords of components powered by gas start with config.settings.optimize.gas_key and

  • such powered with electricity with config.settings.optimize.el_key followed by carrier and region (example: {"electricity-heat_decentral-B": 1}).

  • Do not provide output_parameters or leave their var_value empty to neglect a component in the constraint.

• The capacity of specific components of a scenario can be expanded during an optimization, if the facade attribute expandable is set on True.

  • To adjust this setting, the attribute needs to be specified in the file costs_efficiencies.csv.

  • An overview of all facades, which includes expandable, can be found in the facade attributes overview provided by oemof.tabular.

• There is a distinction between the maximum capacity and the already installed capacity of specific components. Both of the two attributes can be set individually:

  • The attribute capacity in var_name relates to the already installed capacity. The value can be set to 0 for a Greenfield assumption or to a positive numeric value for a Brownfield assumption.

  • The attribute capacity_potential in var_name relates to the maximum capacity, which can be set to a positive numeric value or left empty. In the latter case, an infinite capacity potential is assumed.

Prepare resources

Rules

• create_empty_scalars
• create_empty_ts
• download_raw
• prepare_conv_pp
• prepare_cop_timeseries
• prepare_electricity_demand
• prepare_feedin
• prepare_heat_demand
• prepare_re_potential
• prepare_scalars
• prepare_vehicle_charging_demand
• process_re_potential

Outputs

Output files are saved in results/_resources.

The resources are preprocessed data that serve as material for building scenarios. They are a first intermediate result in oemof-B3 and follow the schema presented in section Schema.

Build datapackages

Rules

• build_datapackage

Outputs

Output files are saved in results/scenario/preprocessed.

Next intermediate results are preprocessed datapackage, which is built using resources, scenario information and the information about the model structure. A preprocessed datapackage represents an instance of an oemof.solph.EnergySystem.

A datapackage is a collection of data in form of csv-files and metadata in form of a json. The data consists of one file for all busses and one for each component, stored in results/<scenario>/preprocessed/data/elements (scalar data) and results/<scenario>/preprocessed/data/sequences (time series for e.g. renewable feed-in or demand profiles), stored in separate folders.

The examples in oemof-B3 are readily preprocessed datapackages (e.g. https://github.com/rl-institut/oemof-B3/tree/dev/examples/example_base/preprocessed). Below is an example of the element file for the gas turbine of the base examples scenario, which can be found in examples/base/preprocessed/base/data/elements/ch4-gt.csv.

region

name

type

carrier

tech

from_bus

to_bus

capacity

capacity_cost

efficiency

carrier_cost

marginal_cost

expandable

output_parameters

BE

BE-ch4-gt

conversion

ch4

gt

BE-ch4

BE-electricity

1500000

0.619

0.021

0.0045

False

{}

BB

BB-ch4-gt

conversion

ch4

gt

BB-ch4

BB-electricity

600000

0.619

0.021

0.0045

False

{}

A separate file, additional_scalars.csv, can contain additional information on constraints. This file is not described in the metadata yet, but will become an official part of the datapackage in the future.

Other than the examples, the datapackages representing actual scenarios are built automatically from the resources, the scenario information scenarios/<scenario>.yml and the model structure.

Components and their attributes are defined in oemoflex. Components and properties can also be added or updated in oemof-B3 using the files in oemof_b3/model/.

Optimization

Rules

• optimize

Outputs

Output files are saved in results/scenario/optimized.

The results are optimized energy systems.

Postprocessing

Rules

• join_scenarios
• postprocess

Outputs

Output files are saved in results/scenario/postprocessed.

Data postprocessing makes use of oemoflex’s functionality, thus postprocessed data follows its data format. See oemoflex’ documention on postprocessed results for further information.

Visualization

Rules

• plot_conv_pp_scalars
• plot_dispatch
• plot_scalar_results
• plot_storage_levels
• table_costs_efficiencies
• table_results

Outputs

Output files are saved in results/scenario/plotted.

OEP upload/download

Rules

• download_resources
• upload_b3_data_to_oep