Is your feature request related to a problem?
Our cron jobs consume memory and don't release it after running. This causes memory to keep growing over time, contributing to the spikes we've seen in CloudWatch.
Describe the solution you'd like
Move the cron jobs over to Celery tasks so memory is released after each run. This should:
- Migrate the cron jobs to Celery tasks
- Ensure memory is released once a task completes
- Fit the broader Celery architecture
- Be verified against the memory spike previously observed in CloudWatch
Original issue
Is your feature request related to a problem?
Our cron jobs consume memory and don't release it after running. This causes memory to keep growing over time, contributing to the spikes we've seen in CloudWatch (eval resync cron jobs / resync calls were flagged as suspects in the Jun 16 sync).
Describe the solution you'd like
Move the cron jobs over to Celery tasks so memory is released after each run. This should:
- Migrate the cron jobs to Celery tasks
- Ensure memory is released once a task completes (no lingering growth)
- Fit the broader Celery architecture (single default queue, LLM calls prioritized via priority number)
- Be verified against the memory spike previously observed in CloudWatch
Is your feature request related to a problem?
Our cron jobs consume memory and don't release it after running. This causes memory to keep growing over time, contributing to the spikes we've seen in CloudWatch.
Describe the solution you'd like
Move the cron jobs over to Celery tasks so memory is released after each run. This should:
Original issue
Is your feature request related to a problem?
Our cron jobs consume memory and don't release it after running. This causes memory to keep growing over time, contributing to the spikes we've seen in CloudWatch (eval resync cron jobs / resync calls were flagged as suspects in the Jun 16 sync).
Describe the solution you'd like
Move the cron jobs over to Celery tasks so memory is released after each run. This should: