Pause a long running job when queue worker shuts down

When running a long job, if your queue worker gets shutdown by

  • Stopping the worker.
  • Sending signal SIGTERM (SIGINT for Horizon).
  • Pressing CTRL + C (Linux/Windows).

Then the job process may get corrupted while it is doing something.

By checking with app('queue.worker')->shouldQuit, we can determine if the worker is shutting down. This way, we can save the current process and requeue the job so that when the queue worker runs again, it can resume from where it left.

This is very useful in the Containerized world (Kubernetes, Docker etc.) where the container gets destroyed and re-created anytime.

1<?php
2 
3namespace App\Jobs;
4 
5use App\Models\User;
6use Illuminate\Bus\Queueable;
7use Illuminate\Contracts\Queue\ShouldQueue;
8use Illuminate\Foundation\Bus\Dispatchable;
9use Illuminate\Queue\InteractsWithQueue;
10use Illuminate\Queue\SerializesModels;
11use Illuminate\Support\Facades\Cache;
12use Illuminate\Support\Facades\Log;
13 
14class MyLongJob implements ShouldQueue
15{
16 use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
17 
18 public $timeout = 3600;
19 
20 private const CACHE_KEY = 'user.last-process-id';
21 
22 public function handle()
23 {
24 $processedUserId = Cache::get(self::CACHE_KEY, 0); // Get last processed item id
25 $maxId = Users::max('id');
26 
27 if ($processedUserId >= $maxId) {
28 Log::info("All users have already been processed!");
29 return;
30 }
31 
32 while ($user = User::where('id', '>', $processedUserId)->first()) {
33 Log::info("Processing User ID: {$user->id}");
34 
35 // Your long work here to process user
36 // Ex. Calling Billing API, Preparing Invoice for User etc.
37 
38 $processedUserId = $user->id;
39 Cache::put(self::CACHE_KEY, $processedUserId, now()->addSeconds(3600)); // Updating last processed item id
40 
41 if (app('queue.worker')->shouldQuit) {
42 $this->job->release(60); // Requeue the job with delay of 60 seconds
43 break;
44 }
45 }
46 
47 Log::info("All users have processed successfully!");
48 }
49}

Tip given by @a-h-abid

Like our articles?

Become a Premium Member for $129/year or $29/month
What else you will get:
  • 22 courses (477 lessons, total 38 h 20 min)
  • 2 long-form tutorials (one new every week)
  • access to project repositories
  • access to private Discord