Relative reference support

[SamWilsn]

Currently docc supports absolute references with the form [link text](ref:ethereum.frontier.blocks.Header.difficulty). It's a pain to have to write out the full path to types when they could be resolved relatively, like for example:

class Foo:
    """
   This a some cool class documentation.
    """

    def some_function(self):
        """
        See [`Foo`](ref:Foo) for more details.
        """

I'm not exactly sure what the final form of this would look like. You might want to use a different prefix (py:Foo) or something else entirely.

[rainwaters11]

@SamWilsn —I'd like to take this on - as part of WIEP

[SamWilsn]

Be my guest! After you have a good idea of how you'll implement it, but before you start coding in earnest, reach out so we can discuss how it'll look for documentation authors.

[rainwaters11]

My plan is to add relative references for Python via a py: prefix while keeping current ref: (absolute) behavior unchanged by default. I’ll do this in small phases: Phase 1: teach the parser to recognize {"ref:", "py:"}, store prefix and target on the node, add a feature flag (default off), and add parsing tests. Phase 2: plumb scope context to the resolver (current_module, current_class, current_function) with tests but no behavior change. Phase 3 (flag on): resolve py:Name in order class → module → global/absolute fallback, emit clear warnings when unresolved, keep ref: exactly as-is, and add regression tests. Phase 4: introduce a small prefix registry/dispatcher for future prefixes (e.g., js:) and update docs with examples and troubleshooting. Backward compatibility: default behavior unchanged; unresolved relative refs warn rather than hard-fail. Before I start, could you confirm (1) is py: your preferred prefix or do you have an existing convention, (2) where the feature flag should live (env var, project config, or module constant) and that default-off is acceptable, (3) whether unresolved py: should warn-and-render-plain or error, (4) preferred files to touch for parser and resolver hooks, (5) any performance or caching concerns for scope lookups, and (6) logging/diagnostics style to follow? If this direction looks good, please assign me and I’ll open a small Phase-1 draft PR with tests.

[SamWilsn]

This seems like a reasonable plan. I have some general concerns about conflicts between a module foo/bar.py and a function bar in foo/__init__.py, but that isn't specific to this feature.

Adding tests is ambitious (and extremely appreciated), but don't stress if you don't end up writing them. If you're sure you want to, you should add the plumbing for them in a separate pull request. It feels like that might be a big one on its own.

My initial thought on how to implement the backend of this is to:

• Change identifier from a str to a list of path segments (eg. segment0.segment1 -> ['segment0', 'segment1'];
• Introduce an enum (see ¹) to represent .. in identifier.
• Add a field (from, source, departure, name to be bikeshedded...) to Reference that holds the location of the Reference itself; and
• Adding a transform that knows how to resolve PathSegments.

That said, it's been over a year since I've made a substantial contribution to docc, so if you have a better/simpler approach, by all means go for it.

a feature flag (default off)

Just go for it. Ain't nobody got time for that!

(1) is py: your preferred prefix or do you have an existing convention

Hm, I think there's a lot of design space here. Off the top of my head, I see two potential approaches:

1. ref:.foo, ref:..foo - use a leading dot to indicate a relative path; or
2. py:foo, py:..foo - use a completely separate scheme.

We could look at other documentation tools for inspiration?

Rustdoc has no special syntax, and uses the native resolution rules (closest to 2).

Sphinx uses a py prefix (but is a bit too loose with its name resolution IMO). This also seems closest to 2.

YARD (Ruby) also has a specific notation for Ruby links ({...}). Again, closest to 2.

Doxygen seems to use C++-style syntax when making explicit links using \ref, so closest to 1.

I think option (2) is the most compatible, especially if we end up supporting other languages with different syntax styles. Unfortunately this approach crosscuts between docc.plugins.mistletoe and docc.plugins.python: how do you tell the mistletoe plugin what valid reference schemes are, or alternatively, does the python plugin need to learn how to process markdown?

One possible solution is to add a way to register special scheme handlers (like how docc.plugins.html registers new node types). The docc.plugins.references plugin could register for ref:, while the docc.plugins.python plugin could register for py:. The docc.plugins.mistletoe plugin could lookup schemes in that registry to see how to handle them (falling back to normal markdown behaviour).

(3) whether unresolved py: should warn-and-render-plain or error

Error

[...]

Go wild. The codebase isn't the best organized, so use your best judgment.

Footnotes

1. Something like:

   class PathSegment(Enum):
       SELF
       PARENT
   

   ↩

[rainwaters11]

Hi Sam,

Thank you for the detailed feedback and insights. Thinking through your comments about future scalability and how other documentation tools handle this, I agree that a more integrated approach is better than a simple, one-off py: prefix.

Based on your suggestion to look at how different plugins could register handlers, I've outlined a new plan that I think is more robust.

The New Approach: A Central Scheme Registry

Instead of just adding a py: prefix, I propose creating a central "scheme registry" in the core docc application. This would allow different plugins to register themselves as the handler for a specific link prefix (like ref:, py:, or a future js:). The markdown parser would check this registry to see how to process a link, making the entire system modular and easily extensible.

Implementation Play-by-Play:

I plan to tackle this in a few small, sequential pull requests:

PR 1: The Core Registry. I'll start by implementing the scheme registry itself and updating the markdown parser to recognize registered schemes. This PR will be the foundational change with no new user-facing features.

PR 2: Implement the py: Handler. Next, I'll build the Python-specific logic for relative linking and register it as the handler for the py: scheme, all within the docc.plugins.python plugin. This will implement the original feature request.

PR 3: Refactor the ref: Handler. Finally, to prove the system's utility, I'll refactor the existing ref: logic to also use this new registry system, keeping its logic cleanly inside the docc.plugins.references plugin.

This approach directly incorporates your vision for a more scalable, plugin-driven architecture. Let me know what you think about this new direction before I start coding.

[SamWilsn]

I'd flip the order of 2 and 3. It'll be easier to verify that the registry works without the added complication of py:

That said, I'm okay with whatever order you want to go with.