Running Async Python Inside Celery Is Harder Than You Think.

Short summary

asyncio.run() fails when event loops already exist, and Celery's fork() model leaves inherited loops orphaned. Solution: create a fresh event loop per worker on a daemon thread, then schedule async tasks using asyncio.run_coroutine_threadsafe() to avoid re-entrant loops and dead threads.

• •asyncio.run() in pre-existing loops causes RuntimeError or unpredictable behavior, even with nest_asyncio
• •Celery's fork() leaves the parent's event loop object intact but with no thread driving it, causing tasks to hang silently
• •Correct pattern: spawn fresh event loop per worker on daemon thread, use run_coroutine_threadsafe() for thread-safe scheduling

Generated with AI, which can make mistakes.