python多线程 内存错误？

Question

代码：

pool = ThreadPool(3)
results = pool.map(main, articles)
pool.close()
pool.join()

错误提示：

python(32471,0x700000501000) malloc: *** error for object 0x7fef9ade6da0: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
[1]    32471 abort      python thread_main.py
(py3) 

是内存错误还是啥意思？ Macbook air

Answer 1

单看log信息只能得出是内存分配的问题

不知道你是的main是 CPU 密集型还是 IO 密集型

• CPU 密集型

from multiprocessing import Pool, cpu_count


def main(i):
    print("{}: hello world!".format(i))

if __name__ == "__main__":
    pool = Pool(cpu_count())
    pool.map(main, (i for i in range(10000)))
    pool.close()
    pool.join()

• IO 密集型

from multiprocessing.dummy import Pool as ThreadPool
from multiprocessing import cpu_count


def main(i):
    print("{}: hello world!".format(i))

if __name__ == "__main__":
    pool = ThreadPool(cpu_count())
    pool.map(main, (i for i in range(10000)))
    pool.close()
    pool.join()
    

两者分不清楚怎么办, 写两种代码, 反正改动小, 写完自己测一测时间, 哪个快用哪个

猜测你这个应该是 CPU 密集型, 应该用多进程, 而不是多线程