JSON metadata file with data export

[BradyPlanden]

Is your feature request related to a problem? Please describe.
The data export side of Galv would benefit from a metadata file that describes the testing, cell definition, experimental equipment, etc.

Describe the solution you'd like
Given the slack discussion, it would be ideal to have the ability to export data from Galv with a corresponding JSON-LD file that defines the metadata for the given experimental test. We have been working on a workflow for this type of use case here: https://github.com/Battery-Intelligence-Lab/IntelLiGent-Data-Standard. I foresee Galv's internal metadata interface acting as the input method for the yellow bubbles below, with the cycler data coming from the sql database.

Additional context
Given the work on Batt-INFO: https://github.com/BIG-MAP/BattINFO it would be ideal to point the @context definition for the JSON-LD towards their database(?). @jsimonclark has done a lot of work on this and can probably comment on the state of Batt-INFO further (and whether modifications would need to be made to include experiemental electrochemical testing parameters i.e. terminal voltage, applied current, surface temperature, etc.)

[martinjrobins]

I think it would be great if Galvcould use this standard you and @jsimonclark are working on. At the moment we store some cell properties, basically just those requested from @davidhowey's group. We also store some experimental metadata, and for this we use the same syntax as PyBaMM's experiment class.

As I understand it, the spec for this JSON-LD file has yet to be determined. I guess this would be the main blocker for this issue then? What is the timescale for this spec to be finalised?

[mjaquiery]

Also worth considering how strict to make this stuff I the data input side: if Galv I'd going to have to provide that data on export we may want a way of requiring/requesting that users input that data on the import side.

[jsimonclark]

@martinjrobins and @BradyPlanden, I think this would be an ideal case for using BattINFO to create RDF compliant data / metadata. If you already have an idea for the JSON schema, it's not a big leap to make the extension to JSON-LD. We just need to add the context and possibly tweak the structure to make it more "triple-friendly".

I've got the last terms we need to implement in a PR under review at EMMO, which I hope will be approved in the next 1-2 weeks.

[mjaquiery]

PR mentioned above

Looks good. We can map JSON to JSON-LD as you say, @jsimonclark - perhaps the difficulty will be in working out where we need to create stuff in our own schema and where it would make more sense to elevate the definition to e.g. BattINFO.

[Do correct me if I'm getting stuff confused - currently doing a deep dive on RDF, Turtle, JSON-LD, EMMO, &c &c]

[jsimonclark]

Here are some resources that can help you get started:

Background:
https://en.wikipedia.org/wiki/Semantic_Web
https://www.w3.org/standards/semanticweb/data
https://www.w3.org/RDF/
https://json-ld.org/
https://www.w3.org/TR/rdf-sparql-query/

Tools:
https://rdflib.readthedocs.io/en/stable/

Other RDF Vocabularies:
https://schema.org/
https://www.dublincore.org/specifications/dublin-core/dcmi-terms/

Triplestores:
https://blazegraph.com/
https://jena.apache.org/

[BradyPlanden]

Hi All,

I've updated the example JSON-LD file for us to iterate / discuss. Would appreciate feedback / pull requests!

A few items I've been thinking on:

• We need a way to generate this file given the metadata and cycler data in Galv. Something like https://pypi.org/project/PyLD/?
• How much metadata do we want to include in the export? There could be justification to include particle model parameters if we are feeling ambitious (i.e. load directly into PyBaMM / PRISM with corresponding experimental data).
• @jsimonclark for the @context part of the JSON-LD, my understanding is this defines the ontology used later in the file. So, for this work, we would point towards parameter/variable definitions in EMMO via the "https...", correct?

[jsimonclark]

This is going in the right direction. A few comments and suggestions:

• Try to avoid spaces in the key names. Most RDF vocabularies use UpperCamelCase for class names (nodes) and lowerCamelCase for property names (edges)
• The context defines the mappings between keys and IRIs. If you are using just one vocabulary like schema.org then you can just set that as the context and be done with it. But in more complex cases like this we end up having to combine vocabularies. That is typically handled by defining a pre-fix for the namespace and then adding the name of the term. For example, in the context you could define "schema": "http://schema.org/" and then in the body you could say "schema:affiliation" : ... This says that you are using the term affiliation as defined in the schema.org namespace.
• Because EMMO uses non-human-readable IRIs, it helps to also create human-readable mappings for the terms. So, in the context you could say, "BatteryCell": "https://emmo.info/...whatever the IRI is for that Term". In EMMO, units are an exception to that rule because they are always unique. In that case you can just write, "emmo:Metre" (we use UK English spelling rules).
• Remember that RDF and linked data is all about building graphs - which means that two nodes are connected by an edge. So, it's not advisable to put two nodes (classes) next to each other in the hierarchy. Rather, try to connect them with a relevant edge (property)