Recognize acronyms, initialisms, and other all-caps sequences

[Witiko]

Acronyms, initialisms, and other all-caps sequences often benefit from being typeset differently from ordinary text, but marking them up manually quickly becomes tedious. Our parser could accept a comma-separated list of words to be treated specially. When these words appear in regular text, the parser would emit dedicated renderers that would apply appropriate typographic treatment such as small caps and letter-spacing.

Here is a demonstration of the proposed interface and the expected output:

\documentclass{article}
\usepackage{markdown}
\markdownSetup {
  % Format the following words as acronyms.
  acronyms = {HTML, YAML},  % We can also easily fill this list from e.g. YAML and other external sources.
  renderers = {
    % Format acronyms as small caps.
    acronym = \textsc{\MakeLowercase{#1}},
    % Don't format acronyms in headings.
    heading* ^= \begingroup \markdownSetup { renderers = { acronym = ##1 } },
    heading* += \endgroup,
  },
}
\begin{document}
\begin{markdown}

HTML and YAML are two staples of modern tooling that often get mentioned
in the same breath, even though they live in very different layers of the stack.

\end{markdown}
\end{document}

[Witiko]

Let's discuss the difficult parts of the implementation:

1. What counts as an occurrence? If HTML is listed, should it match inside XHTML, HTML5, HTML-based, APIs, etc.?
2. Longest-match rules: If users list both HTML and HTML5, the longest (HTML5) should be formatted.
3. Unicode normalization: The options unicodeNormalization and unicodeNormalizationForm should be observed.
4. Efficiency: Checking potentially thousands of acronyms at most points in the text seems slow. We may need to build a trie before parsing to allow efficient scanning. This also automatically solves the longest-match requirement.
5. “Acronyms” that aren’t simple words: If we scan for acronyms at word boundaries, we should take care to still be able to recognize sequences that include punctuation, such as C++, C#, F#, Objective-C, UTF-8, U.S.A., etc.

[Witiko]

1. What counts as an occurrence? If HTML is listed, should it match inside XHTML, HTML5, HTML-based, APIs, etc.?

Let's only recognize acronyms at word boundaries. This will require "XHTML", "HTML5", and "APIs" to be explicitly listed but not "HTML-based". The plurals like "APIs" may be especially annoying to users (and will require a smarter renderer to only apply small caps to the upper case letters) but we should refuse the temptation to guess. It seems comparatively easier to explicitly list plurals than trying to remove false acronyms from the output.

2. Longest-match rules: If users list both HTML and HTML5, the longest (HTML5) should be formatted.
3. Unicode normalization: The options unicodeNormalization and unicodeNormalizationForm should be observed.
4. Efficiency: Checking potentially thousands of acronyms at most points in the text seems slow. We may need to build a trie before parsing to allow efficient scanning. This also automatically solves the longest-match requirement.

Indeed. Nothing to decide here, we just need to keep these in mind.

5. “Acronyms” that aren’t simple words: If we scan for acronyms at word boundaries, we should take care to still be able to recognize sequences that include punctuation, such as C++, C#, F#, Objective-C, UTF-8, U.S.A., etc.

This is achieved by not prohibiting any characters within the acronyms themselves, just at the boundaries.

[Witiko]

When these words appear in regular text, the parser would emit dedicated renderers that would apply appropriate typographic treatment such as small caps and letter-spacing.

Additionally, we may want to produce the renderers even in some places where Markdown markup is typically not allowed, such as the titles of hyperlinks and images.

[michal-h21]

You may also want to declare the acronym together with the expansion. That would enable expansion of the acronym on the first use and also to create a list of acronyms, like what the Gloassaries package does.

[Witiko]

You may also want to declare the acronym together with the expansion. That would enable expansion of the acronym on the first use and also to create a list of acronyms, like what the Glo[a]ssaries package does.

Thanks for the feedback!

I considered that but the focus of this ticket is on a complementary problem that packages like Glossaries do not currently address: automatically recognizing acronyms in running text without requiring explicit markup.

Once such recognition is available, it should be straightforward to integrate it with packages like Glossaries, for example by recording acronym usage from within the acronym renderer when a list of acronyms is present.

[michal-h21]

I realized that myself too after I post my question. I guess that acronym=\gls{#1} should work. But it would be needed to declare acronym twice - once for Glossaries and then for Markdown. So it would be nice to have an interface to do that just once.

[Witiko]

So it would be nice to have an interface to do that just once.

Absolutely! It would be good to have an interface for doing this just once. At the very least, we should provide a documented example (and ideally also a built-in snippet) for inheriting acronym definitions from the Glossaries package when using the Markdown package.

[Witiko]

@TeXhackse raised an important point on our Matrix.org server about the trade-off between explicit and implicit markup. In addition to implicit acronym recognition outlined in the original post of this issue ticket, it might be useful to predefine the HTML class .acronym in our default definitions. This would allow users to write, for example, [APIs]{.acronym} and have the text "APIs" formatted via the acronym renderer.

This approach should give authors an escape hatch for edge cases where automatic recognition falls short, while still benefiting from general implicit rules. In workflows where implicit definitions are controlled by editors and explicit markup is added by authors, this enables a cooperative model: authors can mark up cases missed by the editors, and vice versa.

[Witiko]

This would allow users to write [...] [APIs]{.acronym} and have the text "APIs" formatted via the acronym renderer.

We just need to ensure that later specifying acronyms += APIs does not cause [APIs]{.acronym} to trigger the acronym renderer twice. Let's simply disable nested acronyms altogether in the default definitions? We can use an approach analogous to how acronyms are disabled in headings in the example from the original post of this issue ticket.