[QUESTION] Has `repeat_every` changed behaviour since fastapi-restful 0.5.0?

[ThomasA]

Description

I have used fastapi-restful 0.5.0 for a long time in a project to repeat a task every five minutes. The task used to start every 5 minutes, regardless of how long each task took to complete.

I have switched to fastapi-utils 0.7.0. After the upgrade, it seems like my tasks start about five minutes after the previous task has completed. That is, it seems to have gone from five minutes between task starts to five minutes from the end of each task to the start of the next.

Can this be due to the switch to fastapi-utils? I notice there are some differences in @repeat_every between the packages, but I am not familiar enough with asyncio to conclude whether these can explain it. If this is the case, can I do something to get the behaviour back to five minutes between task starts?

Additional context
I have also upgraded from FastAPI 0.110.2 to 0.115.4.

[ThomasA]

I tried testing this effect with the this script:

from fastapi_restful.tasks import repeat_every as repeat_every_r
from fastapi_utils.tasks import repeat_every as repeat_every_u
from fastapi import FastAPI
from time import sleep
import datetime
import uvicorn
from contextlib import asynccontextmanager


@asynccontextmanager
async def lifespan(app: FastAPI):
    await repeated_task_r()
    await repeated_task_u()
    yield


app = FastAPI(name="My application", lifespan=lifespan)


@repeat_every_r(seconds=5)
def repeated_task_r() -> None:
    print(f"Running task with fastapi-restful ({datetime.datetime.now()})")
    sleep(5)


@repeat_every_u(seconds=5)
def repeated_task_u() -> None:
    print(f"Running task with fastapi-utils ({datetime.datetime.now()})")
    sleep(5)


if __name__ == "__main__":
    uvicorn.run(app, log_level="info")

versus this script:

from fastapi_restful.tasks import repeat_every as repeat_every_r
from fastapi_utils.tasks import repeat_every as repeat_every_u
from fastapi import FastAPI
from time import sleep
import datetime
import uvicorn
from contextlib import asynccontextmanager


app = FastAPI(name="My application")


@app.on_event("startup")
@repeat_every_r(seconds=5)
def repeated_task_r() -> None:
    print(f"Running task with fastapi-restful ({datetime.datetime.now()})")
    sleep(5)


@app.on_event("startup")
@repeat_every_u(seconds=5)
def repeated_task_u() -> None:
    print(f"Running task with fastapi-utils ({datetime.datetime.now()})")
    sleep(5)


if __name__ == "__main__":
    uvicorn.run(app, log_level="info")

In both of these examples, both tasks run every 5 minutes, so none of these differences explain the behaviour.

It turns out that I made another change to my application (not the example here) as well: before, I created a threading.Thread in my task function which in turn ran the actual workload of the task. Because I saw the run_in_threadpool mechanism in fastapi_utils.tasks, I thought I could probably do without this mechanism. However, this seems to make the difference.
If I create a thread inside my task function for running the workload, including some blocking I/O, then tasks start at the interval specified to repeat_every, but if I omit the thread and just call my workload function inside the task function, then each new task starts relative to the end of the previous task.

This means I can get to the "usual" behaviour. I just do not quite understand why this thread mechanism is necessary.

[adaschevici]

I suspect that when creating the thread a new asyncio event loop runs in the new thread, and does not have to wait for any other async operations to finish. This also probably makes it so that the repeat every is respected.