Home Backend Development PHP Tutorial The reason why Laravel repeatedly executes the same queue task

The reason why Laravel repeatedly executes the same queue task

Dec 25, 2017 pm 04:44 PM
laravel queue

Laravel's queue service provides a unified API for various background queue services. Queues allow you to delay the execution of time-consuming tasks, such as sending an email. This can effectively reduce the request response time. This article introduces to you the reasons why Laravel repeatedly executes the same queue task through source code analysis. The article introduces it in detail through sample code. Friends in need can refer to it. Let’s take a look together.

Discover the problem

Use Redis in Laravel to process queue tasks. The functions provided by the framework are very powerful, but recently I encountered a problem, that is, I found that a task was executed multiple times. Why is this? ?

Let’s talk about the reason first:

Because in Laravel, if a queue (task) takes more than 60 seconds to execute, it will be considered to have failed and will be rejoined. Queue, which will cause the same task to be executed repeatedly.

The logic of this task is to push content to users. Users need to be retrieved and traversed based on the queue content, and sent through the request backend HTTP interface. For example, if there are 10,000 users, if the number of users is large or the interface processing speed is not that fast, the execution time will definitely be greater than 60 seconds, so the task will be re-added to the queue. The situation is even worse. If the previous tasks are not executed within 60 seconds, they will be re-added to the queue, so that the same task will be executed not only once, but multiple times.

Let’s find the culprit from the Laravel source code.

Source code file: vendor/laravel/framework/src/Illuminate/Queue/RedisQueue.php

1

2

3

4

5

6

/**

 * The expiration time of a job.

 *

 * @var int|null

 */

protected $expire = 60;

Copy after login

This $expire member variable is a fixed value, Laravel thinks that a queue is no matter how 60 It should be completed within seconds. How to get the queue:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

public function pop($queue = null)

{

 $original $queue ?: $this->default

 $queue $this->getQueue($queue); 

 $this->migrateExpiredJobs($queue.':delayed'$queue); 

 if (! is_null($this->expire)) {

  $this->migrateExpiredJobs($queue.':reserved'$queue);

 

 list($job$reserved) = $this->getConnection()->eval(

  LuaScripts::pop(), 2, $queue$queue.':reserved'$this->getTime() + $this->expire

 ); 

 if ($reserved) {

  return new RedisJob($this->container, $this$job$reserved$original);

 }

}

Copy after login

There are several steps to get the queue. Because the queue execution fails, or the execution times out, etc., it will be put into another collection and saved for retry. The process is as follows:

1 .Repush the queue that failed due to execution from the delayed collection to the currently executed queue.

2. Re-rpush the queue due to execution timeout from the reserved collection to the currently executed queue.

3. Then take the task from the queue and start executing it, and put the queue into the reserved ordered collection.

The eval command is used here to execute this process, and several lua scripts are used.

Get the task from the queue to be executed:

1

2

3

4

5

6

7

8

9

local job = redis.call('lpop', KEYS[1])

local reserved = false

if(job ~= false) then

 reserved = cjson.decode(job)

 reserved['attempts'] = reserved['attempts'] + 1

 reserved = cjson.encode(reserved)

 redis.call('zadd', KEYS[2], ARGV[1], reserved)

end

return {job, reserved}

Copy after login

You can see that when Laravel gets the queue to be executed by Redis, it will also put a copy into an ordered set and use Expiration timestamp as score.

Only when the task is completed, the task will be removed from the ordered set. The code to remove the queue from this ordered collection is omitted. Let's take a look at how Laravel handles queues whose execution time is greater than 60 seconds.

This is the operation performed by this Lua script:

1

2

3

4

5

6

7

8

local val = redis.call('zrangebyscore', KEYS[1], '-inf', ARGV[1])

if(next(val) ~= nil) then

 redis.call('zremrangebyrank', KEYS[1], 0, #val - 1)

 for i = 1, #val, 100 do

  redis.call('rpush', KEYS[2], unpack(val, i, math.min(i+99, #val)))

 end

end

return true

Copy after login

Here zrangebyscore finds the elements whose scores range from infinitesimal to the current timestamp, that is, the tasks added to the collection 60 seconds ago. These elements are then removed from the set via zremrangebyrank and rpush to the queue.

You should suddenly realize it when you see this.

If a queue has not been executed within 60 seconds, the process will rpush the tasks from the reserved set to the queue again when fetching the queue.

Related recommendations:

Some practical tips to improve the performance of Laravel 5

Cross-domain solutions in laravel development Detailed explanation

Laravel WeChat payment development example

The above is the detailed content of The reason why Laravel repeatedly executes the same queue task. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Roblox: Bubble Gum Simulator Infinity - How To Get And Use Royal Keys
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Nordhold: Fusion System, Explained
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Mandragora: Whispers Of The Witch Tree - How To Unlock The Grappling Hook
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Clair Obscur: Expedition 33 - How To Get Perfect Chroma Catalysts
2 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Hot Topics

Java Tutorial
1677
14
PHP Tutorial
1278
29
C# Tutorial
1257
24
Laravel Introduction Example Laravel Introduction Example Apr 18, 2025 pm 12:45 PM

Laravel is a PHP framework for easy building of web applications. It provides a range of powerful features including: Installation: Install the Laravel CLI globally with Composer and create applications in the project directory. Routing: Define the relationship between the URL and the handler in routes/web.php. View: Create a view in resources/views to render the application's interface. Database Integration: Provides out-of-the-box integration with databases such as MySQL and uses migration to create and modify tables. Model and Controller: The model represents the database entity and the controller processes HTTP requests.

Solve caching issues in Craft CMS: Using wiejeben/craft-laravel-mix plug-in Solve caching issues in Craft CMS: Using wiejeben/craft-laravel-mix plug-in Apr 18, 2025 am 09:24 AM

When developing websites using CraftCMS, you often encounter resource file caching problems, especially when you frequently update CSS and JavaScript files, old versions of files may still be cached by the browser, causing users to not see the latest changes in time. This problem not only affects the user experience, but also increases the difficulty of development and debugging. Recently, I encountered similar troubles in my project, and after some exploration, I found the plugin wiejeben/craft-laravel-mix, which perfectly solved my caching problem.

How to learn Laravel How to learn Laravel for free How to learn Laravel How to learn Laravel for free Apr 18, 2025 pm 12:51 PM

Want to learn the Laravel framework, but suffer from no resources or economic pressure? This article provides you with free learning of Laravel, teaching you how to use resources such as online platforms, documents and community forums to lay a solid foundation for your PHP development journey from getting started to master.

Laravel framework installation method Laravel framework installation method Apr 18, 2025 pm 12:54 PM

Article summary: This article provides detailed step-by-step instructions to guide readers on how to easily install the Laravel framework. Laravel is a powerful PHP framework that speeds up the development process of web applications. This tutorial covers the installation process from system requirements to configuring databases and setting up routing. By following these steps, readers can quickly and efficiently lay a solid foundation for their Laravel project.

What versions of laravel are there? How to choose the version of laravel for beginners What versions of laravel are there? How to choose the version of laravel for beginners Apr 18, 2025 pm 01:03 PM

In the Laravel framework version selection guide for beginners, this article dives into the version differences of Laravel, designed to assist beginners in making informed choices among many versions. We will focus on the key features of each release, compare their pros and cons, and provide useful advice to help beginners choose the most suitable version of Laravel based on their skill level and project requirements. For beginners, choosing a suitable version of Laravel is crucial because it can significantly impact their learning curve and overall development experience.

Laravel user login function Laravel user login function Apr 18, 2025 pm 12:48 PM

Laravel provides a comprehensive Auth framework for implementing user login functions, including: Defining user models (Eloquent model), creating login forms (Blade template engine), writing login controllers (inheriting Auth\LoginController), verifying login requests (Auth::attempt) Redirecting after login is successful (redirect) considering security factors: hash passwords, anti-CSRF protection, rate limiting and security headers. In addition, the Auth framework also provides functions such as resetting passwords, registering and verifying emails. For details, please refer to the Laravel documentation: https://laravel.com/doc

How to view the version number of laravel? How to view the version number of laravel How to view the version number of laravel? How to view the version number of laravel Apr 18, 2025 pm 01:00 PM

The Laravel framework has built-in methods to easily view its version number to meet the different needs of developers. This article will explore these methods, including using the Composer command line tool, accessing .env files, or obtaining version information through PHP code. These methods are essential for maintaining and managing versioning of Laravel applications.

The difference between laravel and thinkphp The difference between laravel and thinkphp Apr 18, 2025 pm 01:09 PM

Laravel and ThinkPHP are both popular PHP frameworks and have their own advantages and disadvantages in development. This article will compare the two in depth, highlighting their architecture, features, and performance differences to help developers make informed choices based on their specific project needs.

See all articles