Skip to content

Feature/auto prep deal scheduling reopened#512

Merged
ianconsolata merged 94 commits intodata-preservation-programs:mainfrom
anjor:feature/auto-prep-deal-scheduling-reopened
Jun 27, 2025
Merged

Feature/auto prep deal scheduling reopened#512
ianconsolata merged 94 commits intodata-preservation-programs:mainfrom
anjor:feature/auto-prep-deal-scheduling-reopened

Conversation

@anjor
Copy link
Collaborator

@anjor anjor commented Jun 14, 2025

No description provided.

anjor added 9 commits June 14, 2025 08:04
Core auto-deal functionality including:
- Onboard command for single-step data onboarding
- Auto-deal trigger service
- Unified service for managed workers
- Workflow orchestrator for automatic job progression
- Notification system for observability
- Validation handlers for wallets and storage providers

Fixes data-preservation-programs#495

This is a reopened version of data-preservation-programs#494 which was closed due to history rewrite issues.
- Add auto-deal configuration fields to Preparation model
- Add Notification model for system observability
- Integrate onboard command in main app
- Update database migration tables
- Extend dataprep create handler with auto-deal parameters and validation
- Update pack job handler to trigger auto-deals on completion
- Add notification and validation systems integration
- Add workflow progression triggering in worker threads
- Integrate with auto-deal trigger service
- Add lotus client configuration for deal creation
- Add DEMO_AUTO_PREP_DEALS.md with complete demo script
- Add auto-deal system documentation
- Update CLI reference documentation
- Update Swagger API documentation
- Add comprehensive usage examples and troubleshooting
- Add detailed feature overview and quick start guide
- Document the onboard command and auto-deal workflow
- Include architecture diagrams and usage examples
- Add troubleshooting and migration guides
- Provide comprehensive configuration options
- Add auto-create-deals flag and all deal configuration options
- Add wallet and provider validation flags
- Add workflow automation flags (auto-start, auto-progress)
- Implement automatic scanning and workflow orchestration
- Update AutoDealService to use schedule.Handler interface correctly
- Fix method calls to match the actual schedule handler implementation
- Ensure proper integration with existing schedule creation system
The version.json file is required by the Go embed directive in singularity.go but was not committed to the repository, causing CI Docker builds to fail with "pattern version.json: no matching files found".
@anjor
Copy link
Collaborator Author

anjor commented Jun 16, 2025

add id for tracing?

@anjor anjor force-pushed the feature/auto-prep-deal-scheduling-reopened branch from 00e4b7d to 3b434aa Compare June 16, 2025 19:19
@anjor
Copy link
Collaborator Author

anjor commented Jun 16, 2025

Closes #495

@Sankara-Jefferson
Copy link
Contributor

  • Failing CLI Tests (cmd/). Failure Reason: dial tcp 127.0.0.1:3306: connect: connection refused. Tests assume a live MySQL instance; this breaks on CI/local environments. Either mock, stub, or add Docker bootstrap config.
  • 0% Coverage in Key Services workflow, downloadserver, workermanager, testdb. These packages are runtime-critical and currently untested. At minimum, basic lifecycle/startup and error-path tests should be added.
  • Premature Service Shutdowns. Context cancelled + all services stopped logs appear right after startup. Tests should keep services running long enough to observe real behavior. Consider sleeping or triggering test hooks before shutdown.
  • More feedback on respective files.

@ianconsolata ianconsolata merged commit 469ff6e into data-preservation-programs:main Jun 27, 2025
12 of 14 checks passed
@anjor anjor deleted the feature/auto-prep-deal-scheduling-reopened branch June 27, 2025 21:17
parkan pushed a commit that referenced this pull request Oct 22, 2025
Co-authored-by: Jefferson Sankara <135276815+Sankara-Jefferson@users.noreply.github.com>
parkan pushed a commit that referenced this pull request Nov 6, 2025
Co-authored-by: Jefferson Sankara <135276815+Sankara-Jefferson@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants