upload_b3_data_to_oep

Inputs

in_path: str

results/{scenario}/b3_results/data: path to scalar results in oemof-B3-format

out_path: str

results/{scenario}/b3_results/metadata: path to metadata directory

params:

results/{scenario}/{scenario}.log: path to logfile

Outputs

results/{scenario}/b3_results/metadata: directory with automatically created metadata.

Description

This script performs the following steps:

  • Setup a database connection with your credentials (can be saved in oemof_b3/config/.secrets.yaml

  • Find all csv files in input directory.

  • Create metadata for the data (in oemetadata format, using the template in oemof-B3/schema/oemetadata.json) for given model results (in oemof-B3-format) and saves them locally as json files.

  • Create tables on the OpenEnergyPlatform (OEP) based on the metadata.

  • Upload data to the OEP tables.

  • Validate metadata.

  • Upload metadata to the OEP tables.

The oemetadata format is a standardised json file format and is required for all data uploaded to the OEP. It includes the data model, the used data types, and general information about the data context. Tables in sqlalchemy are created based on the information in the oemetadata.