Infinite loop in NautilusCTSResolver.inventory

[rillian]

Testing a new corpus I ran into an infinite loop in the resolver. I wrote a simple testcase if that's helpful.

test_resolver_loop (tests.cts.test_resolver.TestXMLFolderResolverBehindTheScene)
Test to verify an infinite loop is fixed ... Fatal Python error: Cannot recover from stack overflow.

Current thread 0x00007ff8194bb680 (most recent call first):
  File "${site-packages}/rdflib/term.py", line 156 in __eq__
  File "${site-packages}/rdflib/plugins/memory.py", line 460 in __obj2id
  File "${site-packages}/rdflib/plugins/memory.py", line 471 in __encodeTriple
  File "${site-packages}/rdflib/plugins/memory.py", line 309 in triples
  File "${site-packages}/rdflib/graph.py", line 428 in triples
  File "${site-packages}/rdflib/graph.py", line 638 in objects
  File "${site-packages}/MyCapytain/common/metadata.py", line 224 in getOr
  File "${site-packages}/MyCapytain/resources/prototypes/metadata.py", line 39 in __init__
  File "${site-packages}/MyCapytain/resources/prototypes/cts/inventory.py", line 40 in __init__
  File "${site-packages}/MyCapytain/resources/prototypes/cts/inventory.py", line 555 in __init__
  File "${site-packages}/MyCapytain/resources/collections/cts.py", line 267 in parse
  File "capitains_nautilus/cts/resolver.py", line 175 in parse
  File "capitains_nautilus/cts/resolver.py", line 133 in get_or
  File "capitains_nautilus/cts/resolver.py", line 89 in inventory
  File "capitains_nautilus/cts/resolver.py", line 258 in parse
  File "capitains_nautilus/cts/resolver.py", line 133 in get_or
  File "capitains_nautilus/cts/resolver.py", line 89 in inventory
[...and so on until the recursion limit is reached.]

[PonteIneptique]

Yes, this is kind of a "known" bug: if you have nothing in your corpus, it breaks Nautilus. I'd need to have a look into that

[rillian]

Thanks for taking a look!

In my actual repo, there is something there, of course, but perhaps none of them are loaded due to other validation problems.

[PonteIneptique]

Can you check if the issue is in the DTS branch ?

[rillian]

I...am not sure. With the dts-draft-1 branch and the current dev branch of MyCapitain, the resolver get_or called into the local resolver, which raised a KeyError for the the default collection.

Traceback (most recent call last):
  File "test.py", line 6, in <module>
    resolver.parse()
  File "/home/giles/cdli/Nautilus/capitains_nautilus/cts/resolver/base.py", line 125, in parse
    self._parse(resource)
  File "/home/giles/cdli/Nautilus/capitains_nautilus/cts/resolver/base.py", line 134, in _parse
    return super(ProtoNautilusCtsResolver, self).parse(resource=resource)
  File "/home/giles/cdli/Nautilus/MyCapytain/build/lib/MyCapytain/resolvers/cts/local.py", line 312, in parse
    return self.inventory
  File "/home/giles/cdli/Nautilus/capitains_nautilus/cts/resolver/base.py", line 294, in inventory
    self.__inventory__ = self.get_or(self.inventory_cache_key, self.parse, self.__resources__)
  File "/home/giles/cdli/Nautilus/capitains_nautilus/cts/resolver/base.py", line 90, in get_or
    raise E
  File "/home/giles/cdli/Nautilus/capitains_nautilus/cts/resolver/base.py", line 80, in get_or
    output = callback(*args, **kwargs)
  File "/home/giles/cdli/Nautilus/capitains_nautilus/cts/resolver/base.py", line 125, in parse
    self._parse(resource)
  File "/home/giles/cdli/Nautilus/capitains_nautilus/cts/resolver/base.py", line 134, in _parse
    return super(ProtoNautilusCtsResolver, self).parse(resource=resource)
  File "/home/giles/cdli/Nautilus/MyCapytain/build/lib/MyCapytain/resolvers/cts/local.py", line 306, in parse
    self._dispatch_container(textgroup, textgroup_path)
  File "/home/giles/cdli/Nautilus/capitains_nautilus/cts/resolver/base.py", line 157, in _dispatch_container
    super(ProtoNautilusCtsResolver, self)._dispatch_container(textgroup, directory)
  File "/home/giles/cdli/Nautilus/MyCapytain/build/lib/MyCapytain/resolvers/cts/local.py", line 262, in _dispatch_container
    self._dispatch(textgroup, directory)
  File "/home/giles/cdli/Nautilus/MyCapytain/build/lib/MyCapytain/resolvers/cts/local.py", line 246, in _dispatch
    self.dispatcher.dispatch(textgroup, path=directory)
  File "/home/giles/cdli/Nautilus/MyCapytain/build/lib/MyCapytain/resolvers/utils.py", line 80, in dispatch
    collection.parent = self.collection.children[inventory]
KeyError: 'default'

But after switching to a dev checkout and back it's recursing forever again. I don't know what I've messed up. Are you able to reproduce with the test branch I linked above?

[rillian]

Steps to reproduce, in case there's some build step I'm missing:

$ git clone https://github.com/Capitains/Nautilus
$ cd Nautilus
$ git checkout dts-draft-1
$ git clone https://github.com/Capitains/MyCapytain
$ pipenv install werkzeug typing linkheader six lxml future rdflib-jsonld pyld typing rdflib-sqlalchemy
$ PYTHONPATH=$PWD/MyCapytain pipenv run python << END
from capitains_nautilus.cts.resolver import NautilusCtsResolver
resolver = NautilusCtsResolver(['/dev/null'])
resolver.parse()
END
[...]
RecursionError: maximum recursion depth exceeded while calling a Python object

[PonteIneptique]

Trying to reproduce this morning

[PonteIneptique]

Oh, I thought the reproduction contained your corpora.... I actually know of this bug, never took care of it....
So, I am gonna give it a minute or two

[rillian]

Well, it reproduces with an empty corpus, but the same problem occurs if no works in the corpus validate. :)