用Python实现一个简单的线程池
线程池的概念是什么？
在面向对象编程中，创建和销毁对象是很费时间的，因为创建一个对象要获取内存资源或者其它更多资源。在Java中更是 如此，虚拟机将试图跟踪每一个对象，以便能够在对象销毁后进行垃圾回收。所以提高服务程序效率的一个手段就是尽可能减少创建和销毁对象的次数，特别是一些 很耗资源的对象创建和销毁。如何利用已有对象来服务就是一个需要解决的关键问题，其实这就是一些"池化资源"技术产生的原因。
我理解为线程池是一个存放很多线程的单位，同时还有一个对应的任务队列。整个执行过程其实就是使用线程池中已有有限的线程把任务 队列中的任务做完。这样做的好处就是你不需要为每个任务都创建一个线程，因为当你创建第100个线程来执行第100个任务的时候，可能前面已经有50个线 程结束工作了。因此重复利用线程来执行任务，减少系统资源的开销。
一个不怎么恰当的比喻就是，有100台电脑主机箱需要从1楼搬到2楼，你不需要喊来100人帮忙搬，你只需要叫十个或者二十个人就足以，每个人分配十个或者五个甚至是谁搬的快谁就多搬知道完成未知。（这个比喻好像。。。。。）
不管如何吧，大体上理解了线程池的概念。那么怎么用python实现呢？
代码如下
# !/usr/bin/env python
# -*- coding:utf-8 -*-
# ref_blog:http://www.open-open.com/home/space-5679-do-blog-id-3247.html
import Queue
import threading
import time
class WorkManager(object):
def __init__(self, work_num=1000,thread_num=2):
self.work_queue = Queue.Queue()
self.threads = []
self.__init_work_queue(work_num)
self.__init_thread_pool(thread_num)
"""
初始化线程
"""
def __init_thread_pool(self,thread_num):
for i in range(thread_num):
self.threads.append(Work(self.work_queue))
"""
初始化工作队列
"""
def __init_work_queue(self, jobs_num):
for i in range(jobs_num):
self.add_job(do_job, i)
"""
添加一项工作入队
"""
def add_job(self, func, *args):
self.work_queue.put((func, list(args)))#任务入队，Queue内部实现了同步机制
"""
检查剩余队列任务
"""
def check_queue(self):
return self.work_queue.qsize()
"""
等待所有线程运行完毕
"""
def wait_allcomplete(self):
for item in self.threads:
if item.isAlive():item.join()
class Work(threading.Thread):
def __init__(self, work_queue):
threading.Thread.__init__(self)
self.work_queue = work_queue
self.start()
def run(self):
#死循环，从而让创建的线程在一定条件下关闭退出
while True:
try:
do, args = self.work_queue.get(block=False)#任务异步出队，Queue内部实现了同步机制
do(args)
self.work_queue.task_done()#通知系统任务完成
except Exception,e:
print str(e)
break
#具体要做的任务
def do_job(args):
print args
time.sleep(0.1)#模拟处理时间
print threading.current_thread(), list(args)
if __name__ == '__main__':
start = time.time()
work_manager = WorkManager(10, 2)#或者work_manager = WorkManager(10000, 20)
work_manager.wait_allcomplete()
end = time.time()
print "cost all time: %s" % (end-start)
这个代码清晰易懂。
整个代码只有两个类：WorkManager和Work，前者确实如命名所示，是一个管理者，管理线程池和任务队列，而后者就是具体的一个线程。
它的整个运行逻辑就是，给WorkManager分配制定的任务量和线程数，然后每个线程都从任务队列中获取任务来执行，直到队列中没有任务。这里面也用到了Queue内部的同步机制（至于是啥同步机制目前还没去研究）。
总结一下这样一个线程池的作用，对于我本来的目的其实这个东西是永不上的，因为我需要在web页面来控制线程的启动和停止，而这个线程池看起来只是用来并发完任务的。不过我想虽然在控制线程方面没有作用，但是它的并发执行任务的作用还是蛮不错，或许可以用在爬网页的部分。
Hot AI Tools
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Undress AI Tool
Undress images for free
Clothoff.io
AI clothes remover
Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
PHP and Python: Different Paradigms Explained
Apr 18, 2025 am 12:26 AM
PHP is mainly procedural programming, but also supports object-oriented programming (OOP); Python supports a variety of paradigms, including OOP, functional and procedural programming. PHP is suitable for web development, and Python is suitable for a variety of applications such as data analysis and machine learning.
Choosing Between PHP and Python: A Guide
Apr 18, 2025 am 12:24 AM
PHP is suitable for web development and rapid prototyping, and Python is suitable for data science and machine learning. 1.PHP is used for dynamic web development, with simple syntax and suitable for rapid development. 2. Python has concise syntax, is suitable for multiple fields, and has a strong library ecosystem.
Python vs. JavaScript: The Learning Curve and Ease of Use
Apr 16, 2025 am 12:12 AM
Python is more suitable for beginners, with a smooth learning curve and concise syntax; JavaScript is suitable for front-end development, with a steep learning curve and flexible syntax. 1. Python syntax is intuitive and suitable for data science and back-end development. 2. JavaScript is flexible and widely used in front-end and server-side programming.
Can vs code run in Windows 8
Apr 15, 2025 pm 07:24 PM
VS Code can run on Windows 8, but the experience may not be great. First make sure the system has been updated to the latest patch, then download the VS Code installation package that matches the system architecture and install it as prompted. After installation, be aware that some extensions may be incompatible with Windows 8 and need to look for alternative extensions or use newer Windows systems in a virtual machine. Install the necessary extensions to check whether they work properly. Although VS Code is feasible on Windows 8, it is recommended to upgrade to a newer Windows system for a better development experience and security.
PHP and Python: A Deep Dive into Their History
Apr 18, 2025 am 12:25 AM
PHP originated in 1994 and was developed by RasmusLerdorf. It was originally used to track website visitors and gradually evolved into a server-side scripting language and was widely used in web development. Python was developed by Guidovan Rossum in the late 1980s and was first released in 1991. It emphasizes code readability and simplicity, and is suitable for scientific computing, data analysis and other fields.
Can visual studio code be used in python
Apr 15, 2025 pm 08:18 PM
VS Code can be used to write Python and provides many features that make it an ideal tool for developing Python applications. It allows users to: install Python extensions to get functions such as code completion, syntax highlighting, and debugging. Use the debugger to track code step by step, find and fix errors. Integrate Git for version control. Use code formatting tools to maintain code consistency. Use the Linting tool to spot potential problems ahead of time.
How to run programs in terminal vscode
Apr 15, 2025 pm 06:42 PM
In VS Code, you can run the program in the terminal through the following steps: Prepare the code and open the integrated terminal to ensure that the code directory is consistent with the terminal working directory. Select the run command according to the programming language (such as Python's python your_file_name.py) to check whether it runs successfully and resolve errors. Use the debugger to improve debugging efficiency.
Is the vscode extension malicious?
Apr 15, 2025 pm 07:57 PM
VS Code extensions pose malicious risks, such as hiding malicious code, exploiting vulnerabilities, and masturbating as legitimate extensions. Methods to identify malicious extensions include: checking publishers, reading comments, checking code, and installing with caution. Security measures also include: security awareness, good habits, regular updates and antivirus software.
