Table of Contents
消息队列处理后台任务带来的问题
Resque对后台任务的设计与角色划分
后台任务的角色划分
php-resque的安装
安装Redis
安装Composer
使用Composer安装php-resque
php-resque的使用
编写一个Worker
将Job插入队列
查看Job运行情况
运行Worker
Home Backend Development PHP Tutorial 用PHP实现守护进程任务后台运行与多线程(php-resque使用说明)

用PHP实现守护进程任务后台运行与多线程(php-resque使用说明)

Jun 23, 2016 pm 01:29 PM

消息队列处理后台任务带来的问题

项目中经常会有后台运行任务的需求,比如发送邮件时,因为要连接邮件服务器,往往需要5-10秒甚至更长时间,如果能先给用户一个成功的提示信息,然后在后台慢慢处理发送邮件的操作,显然会有更好的用户体验。

为了实现类似的需求,Web项目中一般的实现方法是使用消息队列(Message Queue),比如MemcacheQ,RabbitMQ等等,都是很著名的产品。

消息队列说白了就是一个最简单的先进先出队列,队列的一个成员就是一段文本。正是因为消息队列实在太简单了,当拿着消息队列时,反而有点无从下手的感觉,因为这仅仅一个发送邮件的任务,就会引申出很多问题:

消息队列只能存储字符串类型的数据,如何将一个发送邮件这样的“任务”,转换为消息队列中的一个“消息”?

消息队列只负责数据的存放与进出,本身不能执行任何程序,那么我们要如何从消息队列中一个一个取出数据,再将这些数据转化回任务并执行。

我们无法预知消息队列何时会有数据产生,所以我们的任务执行程序还需要具备监控消息队列的能力,也就是一个常驻后台的守护进程。

一般的Web应用PHP都以cgi方式运行,无法常驻内存。我们知道php还有cli模式,那么守护进程是否能以php cli来实现,效率如何?

当守护进程运行时,Web应用能否与后台守护进程交互,实现开启/杀死进程的功能以及获得进程的运行状态?

Resque对后台任务的设计与角色划分

对以上这些问题,目前为止我能找到的最好答案,并不是来自php,而是来自Ruby的项目Resque,正是由于Resque清晰简单的解决了后台任务带来的一系列问题,Resque的设计也被Clone到Python、php、NodeJs等语言:比如Python下的pyres以及PHP下的php-resque等等,这里有各种语言版本的Resque实现,而在本篇日志里,我们当然要以PHP版本为例来说明如何用php-resque运行一个后台任务,可能一些细节方面会与Ruby版有出入,但是本文中以php版为准。

Resque是这样解决这些问题的:

后台任务的角色划分

其实从上面的问题已经可以看出,只靠一个消息队列是无法解决所有问题的,需要新的角色介入。在Resque中,一个后台任务被抽象为由三种角色共同完成:

Job | 任务 : 一个Job就是一个需要在后台完成的任务,比如本文举例的发送邮件,就可以抽象为一个Job。在Resque中一个Job就是一个Class。

Queue | 队列 : 也就是上文的消息队列,在Resque中,队列则是由Redis实现的。Resque还提供了一个简单的队列管理器,可以实现将Job插入/取出队列等功能。

Worker | 执行者 : 负责从队列中取出Job并执行,可以以守护进程的方式运行在后台。

那么基于这个划分,一个后台任务在Resque下的基本流程是这样的:

将一个后台任务编写为一个独立的Class,这个Class就是一个Job。

在需要使用后台程序的地方,系统将Job Class的名称以及所需参数放入队列。

以命令行方式开启一个Worker,并通过参数指定Worker所需要处理的队列。

Worker作为守护进程运行,并且定时检查队列。

当队列中有Job时,Worker取出Job并运行,即实例化Job Class并执行Class中的方法。

至此就可以完整的运行完一个后台任务。

在Resque中,还有一个很重要的设计:一个Worker,可以处理一个队列,也可以处理很多个队列,并且可以通过增加Worker的进程/线程数来加快队列的执行速度。

php-resque的安装

需要提前说明的是,由于涉及到进程的开辟与管理,php-resque使用了php的PCNTL函数,所以只能在Linux下运行,并且需要php编译PCNTL函数。如果希望用Windows做同样的工作,那么可以去找找Resque的其他语言版本,php在Windows下非常不适合做后台任务。

以Ubuntu12.04LTS为例,Ubuntu用apt安装的php已经默认编译了PCNTL函数,无需任何配置,以下指令均为root帐号

安装Redis

1

apt-get install redis-server

Copy after login

安装Composer

1

apt-get install curlcd /usr/local/bincurl -s http://getcomposer.org/installer | phpchmod a+x composer.pharalias composer='/usr/local/bin/composer.phar'

Copy after login

使用Composer安装php-resque

假设web目录在/opt/htdocs

1

apt-get install git git-corecd /opt/htdocsgit clone git://github.com/chrisboulton/php-resque.gitcd php-resquecomposer install

Copy after login

php-resque的使用

编写一个Worker

其实php-resque已经给出了简单的例子, demo/job.php文件就是一个最简单的Job:

1

class PHP_Job{    public function perform()    {        sleep(120);        fwrite(STDOUT, 'Hello!');    }}

Copy after login


这个Job就是在120秒后向STDOUT输出字符Hello!

在Resque的设计中,一个Job必须存在一个perform方法,Worker则会自动运行这个方法。

将Job插入队列

php-resque也给出了最简单的插入队列实现 demo/queue.php:

1

if(empty($argv[1])) {    die('Specify the name of a job to add. e.g, php queue.php PHP_Job');}require __DIR__ . '/init.php';date_default_timezone_set('GMT');Resque::setBackend('127.0.0.1:6379');$args = array(    'time' => time(),    'array' => array(        'test' => 'test',    ),);$jobId = Resque::enqueue('default', $argv[1], $args, true);echo "Queued job ".$jobId."\n\n";

Copy after login

在这个例子中,queue.php需要以cli方式运行,将cli接收到的第一个参数作为Job名称,插入名为'default'的队列,同时向屏幕输出刚才插入队列的Job Id。在终端输入:

1

php demo/queue.php PHP_Job

Copy after login

结果可以看到屏幕上输出:

1

Queued job b1f01038e5e833d24b46271a0e31f6d6

Copy after login

即Job已经添加成功。注意这里的Job名称与我们编写的Job Class名称保持一致:PHP_Job

查看Job运行情况

php-resque同样提供了查看Job运行状态的例子,直接运行:

1

php demo/check_status.php b1f01038e5e833d24b46271a0e31f6d6

Copy after login
Copy after login

可以看到输出为:

1

Tracking status of b1f01038e5e833d24b46271a0e31f6d6. Press [break] to stop. Status of b1f01038e5e833d24b46271a0e31f6d6 is: 1

Copy after login

我们刚才创建的Job状态为1。在Resque中,一个Job有以下4种状态:

Resque_Job_Status::STATUS_WAITING = 1; (等待)

Resque_Job_Status::STATUS_RUNNING = 2; (正在执行)

Resque_Job_Status::STATUS_FAILED = 3; (失败)

Resque_Job_Status::STATUS_COMPLETE = 4; (结束)

因为没有Worker运行,所以刚才创建的Job还是等待状态。

运行Worker

这次我们直接编写demo/resque.php:

1

<?php    date_default_timezone_set('GMT');    require 'job.php';    require '../bin/resque';

Copy after login

可以看到一个Worker至少需要两部分:

可以直接包含Job类文件,也可以使用php的自动加载机制,指定好Job Class所在路径并能实现自动加载

包含Resque的默认Worker: bin/resque

在终端中运行:

1

QUEUE=default php demo/resque.php

Copy after login

前面的QUEUE部分是设置环境变量,我们指定当前的Worker只负责处理default队列。也可以使用

1

QUEUE=* php demo/resque.php

Copy after login

来处理所有队列。

运行后输出为

1

#!/usr/bin/env php*** Starting worker

Copy after login

用ps指令检查一下:

1

ps aux | grep resque

Copy after login

可以看到有一个php的守护进程已经在运行了

1

1000      4607  0.0  0.1  74816 11612 pts/3    S+   14:52   0:00 php demo/resque.php

Copy after login

再使用之前的检查Job指令

1

php demo/check_status.php b1f01038e5e833d24b46271a0e31f6d6

Copy after login
Copy after login

2分钟后可以看到

1

Status of b1f01038e5e833d24b46271a0e31f6d6 is: 4

Copy after login

任务已经运行完毕,同时屏幕上应该可以看到输出的Hello!

至此我们已经成功的完成了一个最简单的Resque实例的全部演示,更复杂的情况以及遗留的问题会在下一次的日志中说明。

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Hot Topics

Java Tutorial
1655
14
PHP Tutorial
1253
29
C# Tutorial
1227
24
Explain JSON Web Tokens (JWT) and their use case in PHP APIs. Explain JSON Web Tokens (JWT) and their use case in PHP APIs. Apr 05, 2025 am 12:04 AM

JWT is an open standard based on JSON, used to securely transmit information between parties, mainly for identity authentication and information exchange. 1. JWT consists of three parts: Header, Payload and Signature. 2. The working principle of JWT includes three steps: generating JWT, verifying JWT and parsing Payload. 3. When using JWT for authentication in PHP, JWT can be generated and verified, and user role and permission information can be included in advanced usage. 4. Common errors include signature verification failure, token expiration, and payload oversized. Debugging skills include using debugging tools and logging. 5. Performance optimization and best practices include using appropriate signature algorithms, setting validity periods reasonably,

How does session hijacking work and how can you mitigate it in PHP? How does session hijacking work and how can you mitigate it in PHP? Apr 06, 2025 am 12:02 AM

Session hijacking can be achieved through the following steps: 1. Obtain the session ID, 2. Use the session ID, 3. Keep the session active. The methods to prevent session hijacking in PHP include: 1. Use the session_regenerate_id() function to regenerate the session ID, 2. Store session data through the database, 3. Ensure that all session data is transmitted through HTTPS.

What is REST API design principles? What is REST API design principles? Apr 04, 2025 am 12:01 AM

RESTAPI design principles include resource definition, URI design, HTTP method usage, status code usage, version control, and HATEOAS. 1. Resources should be represented by nouns and maintained at a hierarchy. 2. HTTP methods should conform to their semantics, such as GET is used to obtain resources. 3. The status code should be used correctly, such as 404 means that the resource does not exist. 4. Version control can be implemented through URI or header. 5. HATEOAS boots client operations through links in response.

What are anonymous classes in PHP and when might you use them? What are anonymous classes in PHP and when might you use them? Apr 04, 2025 am 12:02 AM

The main function of anonymous classes in PHP is to create one-time objects. 1. Anonymous classes allow classes without names to be directly defined in the code, which is suitable for temporary requirements. 2. They can inherit classes or implement interfaces to increase flexibility. 3. Pay attention to performance and code readability when using it, and avoid repeatedly defining the same anonymous classes.

How do you handle exceptions effectively in PHP (try, catch, finally, throw)? How do you handle exceptions effectively in PHP (try, catch, finally, throw)? Apr 05, 2025 am 12:03 AM

In PHP, exception handling is achieved through the try, catch, finally, and throw keywords. 1) The try block surrounds the code that may throw exceptions; 2) The catch block handles exceptions; 3) Finally block ensures that the code is always executed; 4) throw is used to manually throw exceptions. These mechanisms help improve the robustness and maintainability of your code.

Explain different error types in PHP (Notice, Warning, Fatal Error, Parse Error). Explain different error types in PHP (Notice, Warning, Fatal Error, Parse Error). Apr 08, 2025 am 12:03 AM

There are four main error types in PHP: 1.Notice: the slightest, will not interrupt the program, such as accessing undefined variables; 2. Warning: serious than Notice, will not terminate the program, such as containing no files; 3. FatalError: the most serious, will terminate the program, such as calling no function; 4. ParseError: syntax error, will prevent the program from being executed, such as forgetting to add the end tag.

What is the difference between include, require, include_once, require_once? What is the difference between include, require, include_once, require_once? Apr 05, 2025 am 12:07 AM

In PHP, the difference between include, require, include_once, require_once is: 1) include generates a warning and continues to execute, 2) require generates a fatal error and stops execution, 3) include_once and require_once prevent repeated inclusions. The choice of these functions depends on the importance of the file and whether it is necessary to prevent duplicate inclusion. Rational use can improve the readability and maintainability of the code.

PHP and Python: Comparing Two Popular Programming Languages PHP and Python: Comparing Two Popular Programming Languages Apr 14, 2025 am 12:13 AM

PHP and Python each have their own advantages, and choose according to project requirements. 1.PHP is suitable for web development, especially for rapid development and maintenance of websites. 2. Python is suitable for data science, machine learning and artificial intelligence, with concise syntax and suitable for beginners.

See all articles