Unitsml schema template using Lutaml::Xsd and Metanorma command

[suleman-uzair]

This PR adds adoc and liquid files for Unitsml schema documentation generation via Lutaml::Xsd gem.

closes #5

[ronaldtse]

@suleman-uzair thanks! Please remove document.html from the latest commit since we shouldn't keep artifacts in the repo.

Also, the document.adoc requires some metadata, please copy the structure from say, sources/unitsdb/document.adoc. Thanks!

[ronaldtse]

We don't want any "Ruby class names" in Liquid... The attribute variable here should already be the Drop object. Every XSD element should be here as a Drop object.

[ronaldtse]

Do we have to handcraft XML elements here? We actually want the source directly, which should be provided by the drop (add a "source" method?).

You can see how Expressir stores the source of the parsed content and provides it through the method.

[ronaldtse]

@suleman-uzair I have updated the document and there are many things to be changed in the way it works.

There is a problem that needs to be comprehensively fixed in this implementation (the combined implementation) that the Liquid code "assumes" Metanorma syntax. There should be no Metanorma AsciiDoc syntax inside Lutaml or Liquid.

For example, items in schema | used_by: attribute_group gives you attribute groups but each is actually rendered as <<{link},{name}>>. There are 2 problems:

• link: All anchors/links are outside of your Lutaml/Liquid code. There should not be any assumptions on what they will be. The user assigns and uses them.
• name: the name of the used_by item should just be item.name.

Remove non-default Liquid::Drop implementations

lutaml-xsd and the plugin must minimize custom Liquid::Drop classes.

The Liquid::Drop classes are automatically generated by Lutaml::Model, and there is very little reason why we need to customize them (any customization needs justification).

We need to remove all (if not most) custom Drop classes. If any is needed, they should be defined using the Lutaml::Model method:

• Allow customization of Liquid Drop classes lutaml/lutaml-model#319

Remove Liquid filters unless absolutely necessary

We do not want to add Liquid filters because they are difficult to use -- the user does not know what data types a filter acts on and what output the filter provides.

Always prefer Drop methods over custom filters.

e.g.

{{ attr | attr_type: }}

Should just be:

{{ attr.type }}

{{ attr | min_max_arg }}

Should just be:

{{ attr.cardinality }}

And any of these need to be documented.

No Metanorma-related assumptions in Liquid

lutaml-xsd does not know about Metanorma and therefore should not have anything to do with Metanorma.

When you compile the document, you can see there are cross-references called from the Liquid filter that assume the patterns of element_*, complex_type_* and attribute_group_*.

These all need to be removed.

Missing XML Schema element relationships: groups, sequence, any of, order

UnitType contains a <xsd:sequence> but it is not reflected anywhere:

	<xsd:complexType name="UnitType">
		<xsd:annotation>
			<xsd:documentation>Type for the unit.</xsd:documentation>
		</xsd:annotation>
		<xsd:sequence>
			<xsd:element ref="UnitSystem" minOccurs="0" maxOccurs="unbounded">
				<xsd:annotation>
					<xsd:documentation>Container for describing the system(s) of units.</xsd:documentation>
				</xsd:annotation>
			</xsd:element>
			<xsd:element ref="UnitName" maxOccurs="unbounded"/>
			<xsd:element ref="UnitSymbol" minOccurs="0" maxOccurs="unbounded"/>
			<xsd:element ref="CodeListValue" minOccurs="0" maxOccurs="unbounded"/>
			<xsd:element ref="RootUnits" minOccurs="0">
				<xsd:annotation>
					<xsd:documentation>Container for defining derived units in terms of their root units. This allows a precise definition of a wide range of units. The goal is to improve interoperability among applications and databases which use derived units based on commonly encountered root units.</xsd:documentation>
				</xsd:annotation>
			</xsd:element>
			<xsd:element ref="Conversions" minOccurs="0"/>
			<xsd:element ref="QuantityReference" minOccurs="0" maxOccurs="unbounded"/>
			<xsd:element ref="UnitVersionHistory" minOccurs="0" maxOccurs="unbounded">
				<xsd:annotation>
					<xsd:documentation>Element for descriptive information, including version changes to the unit.</xsd:documentation>
				</xsd:annotation>
			</xsd:element>
			<xsd:element ref="UnitDefinition" minOccurs="0" maxOccurs="unbounded"/>
			<xsd:element ref="UnitHistory" minOccurs="0" maxOccurs="unbounded"/>
			<xsd:element ref="UnitRemark" minOccurs="0" maxOccurs="unbounded"/>
		</xsd:sequence>
		<xsd:attribute ref="xml:id" use="required"/>
		<xsd:attribute name="timeStamp" type="xsd:dateTime">
			<xsd:annotation>
				<xsd:documentation>Used to indicate the version of the unit output from the Units Database. Changes in the time-stamp are made if a substantive change has been made to the unit, such as a change in the unit definition or changes in conversion factors.</xsd:documentation>
			</xsd:annotation>
		</xsd:attribute>
		<xsd:attributeGroup ref="dimensionURL">
			<xsd:annotation>
				<xsd:documentation>Reference to a representation of the unit or quantity in terms of the 7 SI base dimensions.</xsd:documentation>
			</xsd:annotation>
		</xsd:attributeGroup>
	</xsd:complexType>

Missing simpleType/enumerations

EnumeratedRootUnitType has an enumeration list, but it is not displayed in the document.

	<xsd:complexType name="EnumeratedRootUnitType">
		<xsd:annotation>
			<xsd:documentation>Type for the element for a root unit (from an extensive enumerated list) allowing an optional prefix and power. E.g., mm^2</xsd:documentation>
		</xsd:annotation>
		<xsd:attribute name="unit" use="required">
			<xsd:annotation>
				<xsd:documentation>Unit identifier; the enumerated list is basically English unit names in lowercase, with a few upper case exceptions, e.g., 32F, mmHg, pH.</xsd:documentation>
			</xsd:annotation>
			<xsd:simpleType>
				<xsd:restriction base="xsd:token">
					<xsd:enumeration value="meter"/>
					<xsd:enumeration value="gram"/>
					<xsd:enumeration value="second"/>
					<xsd:enumeration value="ampere"/>
					<xsd:enumeration value="kelvin"/>
					<xsd:enumeration value="mole"/>
					<xsd:enumeration value="candela"/>
...

While the rendered sample XML does display the list using this:

[example]
.Illustrative usage of `<{{ item.name }}>`
====
[source,xml]
-----
<{{item.name}}{{ item_attributes | xml_representations: schema }}
{%- if item_children.size > 0 -%}>
{%- for child in item_children -%}
{% assign item_name = child.ref || child.name %}
{%- if item_name %}
  <{{item_name}}> ... </{{ item_name }}> {{ child | cardinality_representation: }}
{%- else %}
  {{ child }}
{%- endif %}
{%- endfor %}
</{{item.name}}>
{%- else -%}
/>
{% endif %}
-----
====

But this is not what we want.

The {{ child }} line should not render unit="xsd:token (value comes from list: {'meter'|'gram'|'second... at all. child is a Drop itself that allows us to obtain enumerated values, not render it in a random string format.

The pattern of value comes from list should not be in any implementation -- this is to be done in the Liquid template of the document.

Liquid template FileSystem not setup

In the lutaml_xsd block where Liquid's #render is called, you must setup the FileSystem properly to enable the usage of {% render ...%} to resolve the template paths.

Please refer to how lutaml_xmi is implemented in metanorma-plugin-lutaml. These must share the same file system code.

For example, if you extract the attribute rendering code to share amongst the 3 templates, this will fail:

{% render "attribute", attribute: attribute %}

With:

Liquid error: No documents in template path: ~/unitsml/docs/attribute

Because the Liquid FileSystem root is assumed to be PWD which is absolutely not what we want. Instead, it has to be the location of that particular file that defined it, and/or the document root.

lutaml_xsd needs to also accept a direct template path input

Right now lutaml_xsd is done as:

[lutaml_xsd,schemas/unitsml-v1.0-csd04.xsd,schema,skip_rendering_of=annotation]
----
...
----

This is useful, an approach where the user provides an inline Liquid block.

But we need to also support:

lutaml_xsd::schemas/unitsml-v1.0-csd04.xsd[context=schema,template=templates/schema.liquid]

In this case, you would need to add the templates/ directory (the source of the template) into the Liquid file system.

XSD source should be pretty formatted through the canon gem

A Liquid filter that accepts source XML and pretty-prints (as an option?) it using the canon gem.

Target document structure

We need to revamp the implementation to support this structure from NISO STS documentation:

• https://groups.niso.org/higherlogic/ws/public/download/27991/Z39.102-2022.pdf

Elements

Attributes

[ronaldtse]

@suleman-uzair can you please help copy/paste the tickets to the correct repositories for individual implementation? Thanks.

Please also message me so we can resolve this interactively, thanks!

[suleman-uzair]

can you please help copy/paste the tickets to the correct repositories for individual implementation?

@ronaldtse, I have created the following issues in their respective repositories:

1. Expand Drops with methods required by Liquid templates lutaml/lutaml-xsd#19
2. Set up Liquid FileSystem for {% render %} in lutaml_xsd rendering metanorma/metanorma-plugin-lutaml#254
3. Support direct template path input with lutaml_xsd command metanorma/metanorma-plugin-lutaml#253
4. Format XML output using canon gem lutaml/lutaml-xsd#18

We need to remove all (if not most) custom Drop classes. If any is needed, they should be defined using the Lutaml::Model method:

• Allow customization of Liquid Drop classes lutaml/lutaml-model#319

Additionally, please review the following PR for the mentioned issue:

1. Feat: Added custom methods to liquid drop classes lutaml/lutaml-model#466