Review: tyndp_2020_scenario_datafile_all

[MartinHertwig]

Issue description

I'm submitting an open dataset with a corresponding OEMetadata string for review.
Please see review_process.md for technical detail.

Table with max and min supply potential of natural gas for each grid node for all scenarios of TYNDP 2020 (Ten-Year Network Development Plans).

Workflow checklist

1. GitHub

• I have submitted this issue to have metadata and data review documented (Issue #NR)
• Create a new review-branch and push OEMetadata to new branch (review/project_nameofdata#NR). (Can't, because I am not a member of the OEP-git)
• If this step is too difficult, attach a file with the metadata as a comment in this issue and let the reviewer know.
  (Metadata is added as a comment)

2. OEP

• Upload data to the OEP in schema model_draft (see upload tutorial)
• Link URL of data in this issue (model_draft.project_nameofdata)
  https://openenergy-platform.org/dataedit/view/model_draft/tyndp_2020_scenario_datafile_all

3. Start a Review

• Start a pull request (PR) from review-branch to master
• Assign a reviewer and get in contact

4. Reviewer section

• A reviewer starts working on the issue
• Review data license
• A reviewer finished working on this issue (and awarded a badge)
• Update metadata on table
• Data moved to its final schema
• Add OEP tags to table
• Merge PR and delete review-branch
• Document final links of metadata and data in issue description
• Close issue

Metadata and data for review

Here are the links to my data and metadata. Naming follows the pattern model_draft.project_nameofdata:
Metadata is added to a comment in this issue. (I am not a member of the OEP, therefore i am not allowed to push data to the git)

Data: https://openenergy-platform.org/dataedit/view/model_draft/tyndp_2020_scenario_datafile_all

Reviewed and published metadata and data

Final naming and location of the data and metadata after the review are as follows:
schema.tablename

• Metadata
• Data

[MartinHertwig]

{
"name": "TYNDP_2020_all_scenarios_supply_potential_natural_gas",
"title": "Supply potential of natural gas for all scenarios of TYNDP 2020",
"id": "https://openenergy-platform.org/api/v0/schema/model_draft/tables/tyndp_2020_scenario_datafile_all/",
"description": "Table with max and min supply potential of natural gas for each grid node for all scenarios of TYNDP 2020",
"language": [
"en-GB"
],
"keywords": [
"scenario",
"scenarios",
"tyndp",
"energy",
"natural gas",
"supply potential",
"Fraunhofer IEE",
"TYNDP",
"2020",
"ENSTO-E",
"ENTSOG"
],
"publicationDate": "2020-06-01",
"context": {
"homepage": "https://tyndp.entsoe.eu/",
"documentation": "https://eepublicdownloads.blob.core.windows.net/public-cdn-container/tyndp-documents/TYNDP2020/FINAL/entso-e_TYNDP2020_Main_Report_2108.pdf",
"sourceCode": null,
"contact": null,
"grantNo": null,
"fundingAgency": null,
"fundingAgencyLogo": null,
"publisherLogo": null
},
"spatial": {
"location": null,
"extent": "Europe",
"resolution": null
},
"temporal": {
"referenceDate": "2020-01-01",
"timeseries": {
"start": "2020-01-01T00:00+01",
"end": "2040-12-31T23:00+01",
"resolution": "5 a",
"alignment": null,
"aggregationType": "current"
}
},
"sources": [
{
"title": "TYNDP 2020 – Scenario Report",
"description": "",
"path": "https://2020.entsos-tyndp-scenarios.eu/wp-content/uploads/2020/06/TYNDP_2020_Joint_ScenarioReport_final.pdf",
"licenses": [
{
"name": "CC-BY 4.0",
"title": "Attribution 4.0 International",
"path": "https://creativecommons.org/licenses/by/4.0/",
"instruction": "You are free: To Share, To Adapt, Under the following terms: Attribution, No additional restrictions",
"attribution": "@ENTSO-E @ENSTOG"
}
]
},
{
"title": "OpenEnergyPlatform Metadata Example",
"description": "Metadata description",
"path": "https://github.com/OpenEnergyPlatform",
"licenses": [
{
"name": "CC0-1.0",
"title": "Creative Commons Zero v1.0 Universal",
"path": "https://creativecommons.org/publicdomain/zero/1.0/legalcode",
"instruction": "You are free: To Share, To Create, To Adapt",
"attribution": "© Reiner Lemoine Institut"
}
]
}
],
"licenses": [
{
"name": "CC-BY 4.0",
"title": "Attribution 4.0 International",
"path": "https://creativecommons.org/licenses/by/4.0/",
"instruction": "You are free: To Share, To Adapt, Under the following terms: Attribution, No additional restrictions",
"attribution": "@ENTSO-E @ENSTOG"
}
],
"contributors": [
{
"title": "Martin Hertwig",
"email": "martin.hertwig@iee.fraunhofer.de",
"date": "2021-09-15",
"object": null,
"comment": "preparation of tables, upload to OEP and metadata"
},
{
"title": "Mirjam Stappel",
"email": "mirjam.stappel@iee.fraunhofer.de",
"date": "2022-03-29",
"object": null,
"comment": "support and metadata"
}
],
"resources": [
{
"profile": "tabular-data-resource",
"name": "tyndp_2020_scenario_data_all",
"path": "TYNDP_2020_Scenario_Datafile_all.xlsx",
"format": "xlsx",
"encoding": "UTF-8",
"schema": {
"fields": [
{
"name": "id",
"description": "unique identifier",
"type": "serial",
"unit": null
},
{
"name": "Node/Line",
"description": "Node/Line abbreviation which determine the coordinates and region of grid node / grid line",
"type": "character varying",
"unit": null
},
{
"name": "Generator_ID",
"description": "Type of Generator",
"type": "character varying",
"unit": null
},
{
"name": "Parameter",
"description": "Parameter is natural gas",
"type": "character varying",
"unit": null
},
{
"name": "Category",
"description": "Category determines, if the associated value is a production- or a demand-value ",
"type": "character varying",
"unit": null
},
{
"name": "Case",
"description": "Type of function wich was applied on the data e.g. min, max ,average",
"type": "character varying",
"unit": null
},
{
"name": "Scenario",
"description": "Name of scenario for which the data was calculated",
"type": "character varying",
"unit": null
},
{
"name": "Year",
"description": "Year for which the scenario data was calculated",
"type": "integer",
"unit": "year"
},
{
"name": "Climate Year",
"description": "",
"type": "integer",
"unit": "year"
},
{
"name": "Value",
"description": "Resulting value of the case calculation",
"type": "double precision",
"unit": "TWh/y"
},
{
"name": "Simulation_ID",
"description": "",
"type": "character varying",
"unit": "null"
},
{
"name": "Node 1",
"description": "",
"type": "character varying",
"unit": null
},
{
"name": "Path",
"description": "",
"type": "integer",
"unit": null
},
{
"name": "Simulation_type",
"description": "",
"type": "character varying",
"unit": null
},
{
"name": "Sector",
"description": "Gas- or electricity-sector",
"type": "character varying",
"unit": null
},
{
"name": "Note",
"description": "Note to the associated scenario case",
"type": "character varying",
"unit": null
},
{
"name": "Unit",
"description": "Unit of the value",
"type": "character varying",
"unit": null
}
],
"primaryKey": [
"id"
],
"foreignKeys": [
{
"fields": [
""
],
"reference": {
"resource": "",
"fields": [
""
]
}
}
]
},
"dialect": {
"delimiter": null,
"decimalSeparator": "."
}
}
],
"review": {
"path": "",
"badge": ""
},
"metaMetadata": {
"metadataVersion": "OEP-1.4.0",
"metadataLicense": {
"name": "CC0-1.0",
"title": "Creative Commons Zero v1.0 Universal",
"path": "https://creativecommons.org/publicdomain/zero/1.0/"
}
},
"_comment": {
"metadata": "Metadata documentation and explanation (https://github.com/OpenEnergyPlatform/organisation/wiki/metadata)",
"dates": "Dates and time must follow the ISO8601 including time zone (YYYY-MM-DD or YYYY-MM-DDThh:mm:ss±hh)",
"units": "Use a space between numbers and units (100 m)",
"languages": "Languages must follow the IETF (BCP47) format (en-GB, en-US, de-DE)",
"licenses": "License name must follow the SPDX License List (https://spdx.org/licenses/)",
"review": "Following the OEP Data Review (https://github.com/OpenEnergyPlatform/data-preprocessing/wiki)",
"null": "If not applicable use (null)"
}
}

[Ludee]

Finally I had a look at the table and metadata.
I suggest some corrections.

• Add two additional sources: download page and excel file
• Adjust license, the report is not licensed, only the data file
• harmonise the column names and descriptions

Edit: I don't really understand which sheet from the excel file has been used.
The title says "datafile_all" but it only has the natural gas data.
Please explain

{
  "name": "TYNDP_2020_all_scenarios_supply_potential_natural_gas",
  "title": "Supply potential of natural gas for all scenarios of TYNDP 2020",
  "id": "https://openenergy-platform.org/dataedit/view/model_draft/tyndp_2020_scenario_datafile_all",
  "description": "The TYNDP and the economic and technical studies performed to produce it generate a great quantity of valuable information on the future of the European power system. ENTSO-E, the European Network of Transmission System Operators for Electricity, is the association for the cooperation of the European transmission system operators (TSOs). ENTSO-E’s 10-year network development plan (TYNDP) is the European electricity infrastructure development plan.",
  "language": [
    "en-GB"
  ],
  "keywords": [
    "Fraunhofer IEE",
    "TYNDP",
    "2020",
    "ENSTO-E",
    "ENTSOG"
  ],
  "publicationDate": "2020-06-01",
  "context": {
    "homepage": "https://tyndp.entsoe.eu/",
    "documentation": "https://eepublicdownloads.blob.core.windows.net/public-cdn-container/tyndp-documents/TYNDP2020/FINAL/entso-e_TYNDP2020_Main_Report_2108.pdf"
  },
  "spatial": {
    "extent": "Europe"
  },
  "temporal": {
    "referenceDate": "2020-01-01",
    "timeseries": [
      {
        "start": "2020-01-01T00:00+01",
        "end": "2040-12-31T23:00+01",
        "resolution": "5 a",
        "aggregationType": "current"
      }
    ]
  },
  "sources": [
    {
      "title": "TYNDP 2020 – Scenario Report",
      "description": "Main Report - ENTSOs’ TYNDP 2020 Scenario Report describes possible European energy futures up to 2050. Scenarios are not forecasts; they set out a range of possible futures used by the ENTSOs to test future electricity and gas infrastructure needs and projects.  The scenarios are ambitious as they deliver a low carbon energy system for Europe by 2050. The ENTSOs have developed credible scenarios that are guided by technically sound pathways, while reflecting country by country specifics, so that a pan-European low carbon future is achieved.",
      "path": "https://2020.entsos-tyndp-scenarios.eu/wp-content/uploads/2020/06/TYNDP_2020_Joint_ScenarioReport_final.pdf",
      "licenses": []
    },
    {
      "title": "TYNDP 2020 – Scenario Report - Download Data",
      "description": "Download Page",
      "path": "https://2020.entsos-tyndp-scenarios.eu/download-data/",
      "licenses": []
    },
    {
      "title": "TYNDP 2020 – Scenario Data",
      "description": "Scenario Data Sets - The joint scenario package provides an extensive data set resource that is used by each ENTSO TYNDP and other studies. The ENTSOs invite stakeholders to use the scenario data sets for their own studies. Whereas Distributed Energy and Global Ambition have been built as full-energy scenarios until 2050, National Trends is based on electricity and gas related data from the NECP and developed until 2040.",
      "path": "https://2020.entsos-tyndp-scenarios.eu/wp-content/uploads/2020/06/TYNDP-2020-Scenario-Datafile.xlsx.zip",
      "licenses": [
        {
          "name": "CC-BY-4.0",
          "title": "Creative Commons Attribution 4.0 International",
          "path": "http://creativecommons.org/licenses/by/4.0/",
          "instruction": "You are free: To Share, To Adapt, Under the following terms: Attribution, No additional restrictions",
          "attribution": "©ENTSO-E ©ENTSOG TYNDP 2020 Scenarios"
        }
      ]
    }
  ],
  "licenses": [
    {
      "name": "CC-BY-4.0",
      "title": "Creative Commons Attribution 4.0 International",
      "path": "https://creativecommons.org/licenses/by/4.0/",
      "instruction": "You are free: To Share, To Adapt, Under the following terms: Attribution, No additional restrictions",
      "attribution": "©ENTSO-E ©ENTSOG TYNDP 2020 Scenarios"
    }
  ],
  "contributors": [
    {
      "title": "Martin Hertwig",
      "email": "martin.hertwig@iee.fraunhofer.de",
      "date": "2021-09-15",
      "object": "Data, Metadata",
      "comment": "Preparation of tables, upload to OEP and metadata"
    },
    {
      "title": "Mirjam Stappel",
      "email": "mirjam.stappel@iee.fraunhofer.de",
      "date": "2022-03-29",
      "object": "Metadata",
      "comment": "Support and metadata"
    },
    {
      "title": "Ludee",
      "date": "2023-02-17",
      "object": "Metadata",
      "comment": "Review and update metadata"
    }
  ],
  "resources": [
    {
      "profile": "tabular-data-resource",
      "name": "tyndp_2020_scenario_data_all",
      "encoding": "UTF-8",
      "schema": {
        "fields": [
          {
            "name": "id",
            "description": "unique identifier",
            "type": "serial"
          },
          {
            "name": "node",
            "description": "Node abbreviation which determine the coordinates and region of grid node",
            "type": "text"
          },
          {
            "name": "generator_id",
            "description": "Type of Generator",
            "type": "varchar(128)"
          },
          {
            "name": "parameter",
            "description": "Parameter is natural gas",
            "type": "text"
          },
          {
            "name": "category",
            "description": "Category determines, if the associated value is a production- or a demand-value ",
            "type": "text"
          },
          {
            "name": "case",
            "description": "Type of function wich was applied on the data e.g. min, max ,average",
            "type": "text"
          },
          {
            "name": "scenario",
            "description": "Name of scenario for which the data was calculated",
            "type": "text"
          },
          {
            "name": "year",
            "description": "Year for which the scenario data was calculated",
            "type": "integer"
          },
          {
            "name": "Climate Year",
            "description": "climate_year",
            "type": "integer"
          },
          {
            "name": "value",
            "description": "Resulting value of the case calculation",
            "type": "float",
            "unit": "TWh/y"
          },
          {
            "name": "simulation_id",
            "description": "Simulation_ID",
            "type": "varchar(128)"
          },
          {
            "name": "node_1",
            "description": "Node 1",
            "type": "varchar(128)"
          },
          {
            "name": "path",
            "description": "Path",
            "type": "bigint"
          },
          {
            "name": "simulation_type",
            "description": "Simulation type",
            "type": "text"
          },
          {
            "name": "sector",
            "description": "Gas- or electricity-sector",
            "type": "text"
          },
          {
            "name": "note",
            "description": "Note to the associated scenario case",
            "type": "text"
          },
          {
            "name": "unit",
            "description": "Unit of the value",
            "type": "varchar(128)"
          }
        ],
        "primaryKey": [
          "id"
        ],
        "foreignKeys": [
          {
            "fields": [
              ""
            ],
            "reference": {
              "fields": [
                ""
              ]
            }
          }
        ]
      },
      "dialect": {
        "decimalSeparator": "."
      }
    }
  ],
  "metaMetadata": {
    "metadataVersion": "OEP-1.4.0",
    "metadataLicense": {
      "name": "CC0-1.0",
      "title": "Creative Commons Zero v1.0 Universal",
      "path": "https://creativecommons.org/publicdomain/zero/1.0/"
    }
  },
  "_comment": {
    "metadata": "Metadata documentation and explanation (https://github.com/OpenEnergyPlatform/organisation/wiki/metadata)",
    "dates": "Dates and time must follow the ISO8601 including time zone (YYYY-MM-DD or YYYY-MM-DDThh:mm:ss±hh)",
    "units": "Use a space between numbers and units (100 m)",
    "languages": "Languages must follow the IETF (BCP47) format (en-GB, en-US, de-DE)",
    "licenses": "License name must follow the SPDX License List (https://spdx.org/licenses/)",
    "review": "Following the OEP Data Review (https://github.com/OpenEnergyPlatform/data-preprocessing/wiki)",
    "null": "If not applicable use (null)"
  }
}