Skip to content

adding retries to delete task to handle info service being down #215

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
fcollman wants to merge 1 commit into
base: master
Choose a base branch
from
Open

adding retries to delete task to handle info service being down #215

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion materializationengine/workflows/periodic_database_removal.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,11 @@ def get_valid_versions(session):
return [str(version) for version in valid_versions]


@celery.task(name="workflow:remove_expired_databases")
@celery.task(name="workflow:remove_expired_databases",
bind=True,
acks_late=True,
autoretry_for=(Exception,),
max_retries=3)
Copy link

@cursor cursor bot Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing self parameter for bound Celery task

High Severity

The bind=True option was added to the Celery task decorator, but the function signature is missing the required self parameter as the first argument. When Celery invokes this task, it will pass the task instance as the first positional argument, which will be incorrectly assigned to delete_threshold. This causes delete_threshold to receive a task object instead of an integer, leading to a TypeError when comparing len(databases) > delete_threshold at runtime. The task will fail on every invocation and retry 3 times with the same error.

Fix in Cursor Fix in Web

def remove_expired_databases(delete_threshold: int = 5, datastack: str = None) -> str:
"""Remove expired databases and clean up their metadata.

Expand Down
Loading