Fix #186: MinerU 2.7.0+ caused filepath errors with new default backend #188
+46
−12
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Modified _read_output_files() to scan subdirectories for actual output - Added backend mapping for hybrid-* backends to use hybrid_auto method - Maintains backward compatibility with pipeline, vlm, and other backends - Successfully tested with pipeline, hybrid_auto, vlm, and custom backends Co-authored-by: hanlianlu <7419836+hanlianlu@users.noreply.github.com>
- Updated README.md and README_zh.md with hybrid-auto-engine backend option - Added hybrid-auto-engine to CLI argument choices - Enhanced code comments explaining backend-to-directory mapping - Documents MinerU 2.7.0+ backend directory naming conventions Co-authored-by: hanlianlu <7419836+hanlianlu@users.noreply.github.com>
- Replace outdated backend names (vlm-transformers, vlm-sglang-engine, vlm-sglang-client) - Add current MinerU 2.7.1+ backend options (hybrid-http-client, vlm-auto-engine, vlm-http-client) - Update vlm_url references to use vlm-http-client instead of vlm-sglang-client - Changes made in parser.py, README.md, and README_zh.md Co-authored-by: hanlianlu <7419836+hanlianlu@users.noreply.github.com>
…x-change-requirements
Co-authored-by: hanlianlu <7419836+hanlianlu@users.noreply.github.com>
…-issue Fix MinerU hybrid-auto-engine output directory detection and update backend options
Update backend choices to match MinerU 2.7.1 CLI specification
Co-authored-by: hanlianlu <7419836+hanlianlu@users.noreply.github.com>
Fix trailing whitespace in parser.py
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
MinerU 2.7.0+ uses hybrid_auto/ subdirectory for hybrid-auto-engine backend output, but RAG-Anything hardcoded auto/ paths, causing file-not-found errors. Additionally, backend options have been updated to match MinerU 2.7.1+ CLI specification.
Related Issues
Addresses issue #186
Changes Made
Core Fix: Dynamic Directory Detection
_read_output_files() now scans subdirectories for *_content_list.json marker files instead of hardcoding paths
Automatically detects output in auto/, hybrid_auto/, vlm/, or any future backend directories
Falls back to method-based path construction when subdirectory scan finds nothing
Backend Mapping Enhancement
Added hybrid-* backend detection to map method="auto" → method="hybrid_auto"
Extends existing vlm-* backend pattern for consistency
Backend Options Update
Updated all backend references to match MinerU 2.7.1+ CLI specification
Replaced outdated backend names (vlm-transformers, vlm-sglang-engine, vlm-sglang-client)
Added current backend options: pipeline, hybrid-auto-engine, hybrid-http-client, vlm-auto-engine, vlm-http-client
Updated CLI argument parser, README.md, README_zh.md, and all vlm_url references
Documentation
Added hybrid-auto-engine and other current backends to CLI choices and README backend options
Inline comments document MinerU 2.7.0+ directory naming conventions
Before (fails with hybrid-auto-engine)
file_stem_subdir = output_dir / file_stem
if file_stem_subdir.exists():
md_file = file_stem_subdir / method / f"{file_stem}.md" # Assumes method matches directory name
After (works with any backend)
if file_stem_subdir.exists():
found = False
for subdir in file_stem_subdir.iterdir():
if (subdir / f"{file_stem}_content_list.json").exists():
md_file = subdir / f"{file_stem}.md" # Uses actual directory
found = True
break
Checklist
Changes tested locally
Code reviewed
Documentation updated (if necessary)
Unit tests added (if applicable)