Feature Request: User-specified Sections

[BrianPugh]

Motivation: BrianPugh/cyclopts#477

Basically, I don't want docstring_parser to handle "Examples" sections. I haven't looked to hard at the code base, so I'm not sure where the best place to change is. Ideally I'd want this available through the common parse function.

I'm willing to help out on a PR if someone (@mauvilsa ?) could give me pointers/recommendations.

[mauvilsa]

It would have been nicer if you explained here, instead of just adding links and have to go through two conversations to figure out what you want. Anyway, I went through them. And I don't understand the reason why docstring_parser should change. When you parse the docstring, the examples can be accessed through the examples attribute. You are free to use it however you see fit. Furthermore, docstring_parser is used in many other projects which can potentially break. Suggesting that examples should be handled differently is already a no go. Maybe you have in mind something that wouldn't break other projects. But since you haven't explained, then no way to know.

[BrianPugh]

Why the harsh tone?

Some parsers, like the GoogleParser and NumpydocParser, take in sections that specifies which sections to parse from the docstring. If not set, they default to their respective DEFAULT_SECTIONS. Some parsers, like epydoc and rest, do not have this feature (I'm also not familiar with these documentation styles). Despite GoogleParser and NumypdocParser both taking in sections, they actually use independent Section classes.

The main function, parse, does not provide a mechanism to specify sections, and docstring_parser as a whole does not provide a mechanism to identify the style of a supplied docstring. Between the two (and the separate Section classes), there's no real way of taking advantage of this feature.

It might be reasonable to add a new parameter sections: Sequence | None = None to compose that gets passed along to GoogleParser and NumpydocParser, but we would need to unify the specification a little bit between the two parsers. They both should probably just have sections in their constructors be a Sequence[Literal['Arguments', 'Parameters', ...]], and then we can make a mapping between the strings and their instantiated Section objects. This can be implemented in a backwards compatible way.

Because of the bad ergonomics of specifying a 10-or-20-long array of strings, it would also be good to add an exclude_sections option.

I opened this issue to get guidance on how we would want to go about implementing this, because there are many ways to go about it. I did not intend for the initial post to come across as lazy or lacking in information.

[mauvilsa]

Why the harsh tone?

Sorry about that. It was not my intention to sound harsh. It was just my honest opinion about the short initial description.

From your last post, I would say that you are more familiar with the code base than me. I can guide. But maybe more from the perspective of the feature making sense and being useful for a broad audience.

You say you want to add a sections parameter to parse(). And you already mention issues, like sections not in all parsers, which would make it a bit weird to have such a parameter. But it is still not clear to me why this specific change. As I mentioned before, the examples can be accessed through the examples attribute. Why isn't this useful for you? Or why can't that be improved so that it becomes more useful?

On the other hand, it is not clear to me what exactly this means:

I don't want docstring_parser to handle "Examples" sections

If a section is not handled, then what is the behavior? If I go to the google parser and comment-out the examples section, then the content of the examples seems to either be 1) just lost if it appears after some other section, or 2) be part of the long description if before any other section.

I think it would be good for you to explain what is the actual problem you want to solve, to then figure out what could be the simplest solution. Instead of immediately thinking about how to implement a sections parameter which already looks to be a complex feature.