MMIF `describe` as a part of `mmif_utils` module

[keighrim]

New Feature Summary

As a sibling to rewind command, it'd be nice to have a command line tool to describe/summarize input MMIF . This thread to discuss and brainstorm such a feature.

Related

Maybe we can start by exploring possibility of merging https://github.com/clamsproject/mmif-summarizer .

Alternatives

No response

Additional context

No response

[keighrim]

#339 currently provides a very limited set of statistics for the input MMIF. Although this describe command is not aiming to fully replicate all that complex graph building and html rendering from the mmif-summarizer implementation, I think we could benefit from having a little more features to the describe command.

To fully address and close this issue, I have two key items on my wishlist:

Naming the Annotation Process Identifier

We need to decide on a proper and consistent name for the identifier that tracks the sequence of CLAMS apps used to generate a MMIF's views.

• basic mmif describe command #339 uses pipeline_id, inherited from the older mmif-storage implementation.
• In the past, we have historically mixed the terms "pipeline" and "workflow" for this object (e.g., using "workflow" for our "workflow engine" context and wfe- repositories). There is currently no official consensus or glossary term to my understanding.
• New Suggestion
  • @owencking suggested the name "recipe" and recipeId, which I find very appealing as a clear alternative.

Handling Collections of MMIFs

The tool should be able to accept a directory containing multiple MMIFs as input. It should then:

• Summarize the statistics for each individual MMIF.
• Provide a collection-level summary, including metrics like:
  • How many MMIFs originated from the same "recipe" (or chosen name).
  • How many MMIFs failed to process or contain errors/warnings.
  • and more

---

I am happy to hear any additional features or wishlists from others. Please leave comments below!

[marcverhagen]

To me "workflow" was always a more general and high-level term focused at the methodology around running pipelines (like using chowda) whereas "pipeline" was more concrete and pointed at the pipeline itself.

Both pipeline_id and recipe_id (or recipeId) are fine with me.

There now is overlap between summarize and describe. If the goal is not to fully replicate the summarizer it does seem to be to replicate some of the summarizer and add a couple of new things (mostly the pipeline_id, which is very useful). So in that case we should be clear on what kind of information is in the result of one and what is in the result of the other. I do not have a clear dividing line. For example, I see the suggestion to have collection-level statistics for MMIF files. After a suggestion from Owen I was working on that for the summarizer and was including collection-level error/warning statistics (since the summarizer does retain the warnings), it was also including annotation statistics (which is not envisioned for describe I think); on the other hand, I had no plans to put in how many MMIFs came from the same recipe.

The describe command also has some overlap with the mmif-storage status API. This brings up something that I had thought about before which is to integrate summarize (and now describe) into the data server.

[keighrim]

Regarding

The describe command also has some overlap with the mmif-storage status API. This brings up something that I had thought about before which is to integrate summarize (and now describe) into the data server.

I worked on this clamsproject/aapb-brandeis-datahousing@b1cdb4b Haven't merged yet, waiting for some further decisions from this discussion (like the naming).

The summarizer should also be updated to declutter the duplication of code, and I have thought that was the plan: splitting the summarizer into a CLI and more "advance" features like HTML generation and merge CLI to the main mmif cmd. So there shouldn't be any "overlap" ideally.

[marcverhagen]

I worked on this clamsproject/aapb-brandeis-datahousing@b1cdb4b Haven't merged yet, waiting for some further decisions from this discussion (like the naming).

Nice.

The summarizer should also be updated to declutter the duplication of code, and I have thought that was the plan: splitting the summarizer into a CLI and more "advance" features like HTML generation and merge CLI to the main mmif cmd. So there shouldn't be any "overlap" ideally.

Agreed. I am hoping to have that done by the time we meet this afternoon. To me, that's the easy part. What I was trying to say was that it is unclear to me how much the CLI part of the summarizer overlaps with describe and how best to separate them. The one thing that sticks out a bit to me is that the summary is always going to be more about the annotation content rather than any pipeline-related statistics, which make the pipeine_id one of those things that should be done by describe.

[keighrim]

After giving some thoughts, I decided to go with more generic term "workflow" (I really liked "recipe", though!)

Also noticed that the describe and the "ID" both don't have information about the source file. Assuming the MMIF file name is encoding the asset ID (AAPB-GIUD for AAPB data for example), we should at least show how many source files were used in a given MMIF. Then, since the source documents always come at the top-level, source info must be the first prefix in the ID construction. For example, IDs for

• A singleton SWT workflow ran with a single source video
  • VideoDocument-1/app-swt-detection/v8.1/xxxx
• A singleton FA workflow ran with a single source video and its associated transcript
  • TextDcoument-1-VideoDocument-1/app-nfa-wrapper/v0.8/xxxx

Some notes on the first prefix

• adding this will invalidate all exsiting mmif paths (not the files themselves) in our current mmif-storage server, which is fine, as none of paths are being used in any other places (except for manual effort)
• the syntax in the example is type-number-type-number-.... I think for the type parts, they MUST BE the canonical short names for document type in the vocabulary (unlike in the mmif source or clams source command, where we use non-canonical short names like video text, ..)
• for the delimiter, I arbitrarily chose - for everywhere in that part of the string, assuming there's no negative number of documents used.
• for document types, they must be alphabetically sorted (e.g., MUST NOT BE VideoDocument-1-TextDcoument-1)