Enhancement #34
Scheduling: queue vs. cron job
| Status: | New | Start date: | ||
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assignee: | Anonymous | % Done: | 0% | |
| Category: | ProCKSI/scheduling | |||
| Target version: | 9.0 | |||
| Resolution: |
Description
- Current State: The complete scheduling is done by cron jobs including the entire post-processing, e.g. uncompressing the TGZ output, registering results, finishing tasks/requests.
- Potential Problem: The cron jobs start at fixes times, e.g. every 5 minutes, check the status and react accordingly. If the post-processing takes longer than the interval between two cron job starts, two or more cron jobs might run at the same time concurrently.
- Possible Solutions:
1. Lock the program that is being started by a cron job, e.g. check_tasks.lock, and don't start a new cron job before this lock file has been deleted again at the end of the first cron job's run.
2. Do not perform any "real tasks", e.g. post-processing, from within the cron job, but rather submit a new job to a special administration queue. Introduce new task states for post-processing as to exclude the job from being submitted multiple times into the queue within multiple cron job runs.
3. Implement a daemon that controls all activities instead of running several cron jobs potentially in parallel. Possible candidate: Schedule::Cron
History
#1 Updated by Anonymous over 16 years ago
- Temporary Solution: Solution 1 is being implemented.
- Final Solution: A combination of Solution 2 and 3 will be favourable