串行:同一個(gè)時(shí)間段只干一件事
并行:同一個(gè)時(shí)間段可以干多件事
-
并發(fā) V.S. 并行
- 并發(fā)是指一個(gè)時(shí)間段內(nèi),有幾個(gè)程序在同一個(gè)CPU上運(yùn)行,但是任意時(shí)刻只有一個(gè)程序在CPU上運(yùn)行。
- 并行是指任意時(shí)刻點(diǎn)上,有多個(gè)程序同時(shí)運(yùn)行在多個(gè)CPU上,即每個(gè)CPU獨(dú)立運(yùn)行一個(gè)程序。
- 并行的最大數(shù)量和CPU的數(shù)量是一致的。
-
同步 V.S. 異步:
- 同步是指代碼調(diào)用IO操作時(shí),必須等待IO操作完成返回才調(diào)用的方式
- 異步是指代碼調(diào)用IO操作時(shí),不必等待IO操作完成返回才調(diào)用的方式
多線程:交替執(zhí)行,另一種意義上的串行
比進(jìn)程的概念小,減少了進(jìn)程切換過(guò)程中的消耗
多進(jìn)程:并行執(zhí)行,真正意義上的并行
通過(guò)切換不同的進(jìn)程來(lái)實(shí)現(xiàn)多任務(wù)的并行
-
多進(jìn)程 V.S. 多線程:
- 由于全局解釋鎖GIL的存在,Python的多線程無(wú)法利用多核優(yōu)勢(shì),所以不適合計(jì)算密集型任務(wù),適合IO密集型任務(wù)。
- CPU密集型任務(wù)適合使用多進(jìn)程
- 進(jìn)程切換代價(jià)要高于線程
- 線程之間可以通過(guò)全局變量來(lái)進(jìn)行通信,但是進(jìn)程不行。進(jìn)程之間的數(shù)據(jù)是完全隔離的。
-
threading類(lèi):
?
threading.Lock()
:同步鎖(互斥鎖),解決數(shù)據(jù)安全問(wèn)題?
threading.RLock()
:遞歸鎖,解決線程死鎖問(wèn)題?
threading.Semaphore()
:信號(hào)量,最多允許同時(shí)有n個(gè)線程,超過(guò)信號(hào)量的線程會(huì)被堵塞,直到有線程的信號(hào)量被釋放。?
threading.Event()
:事件,讓不同的線程保持同步而不是獨(dú)立的運(yùn)行,彼此交互:你給我發(fā)一個(gè)信號(hào),我接到了執(zhí)行一個(gè)操作,再給你發(fā)一個(gè)信號(hào),你接到了執(zhí)行操作并繼續(xù)發(fā)信號(hào)……
鎖
import threading
#生成鎖對(duì)象,全局唯一
lock = threading.Lock()
#獲取鎖
lock.acquire()
#釋放鎖
lock.release()
- 可以使用上下文管理協(xié)議來(lái)管理鎖
lock = threading.Lock()
with lock:
pass
#等價(jià)于:
lock.acquire()
try:
pass
finally:
lock.release()
- 加鎖會(huì)影響性能,獲取鎖和釋放鎖都需要時(shí)間。
- 使用鎖可能會(huì)引起死鎖
多線程
import threading
import time
def sleeper(n,name):
print('Hi, I`m {}. I`m going to sleep 5 seconds'.format(name))
time.sleep(n)
print('{} has woken up from sleep'.format(name))
e.g.1
def main():
t = threading.Thread(target = sleeper, args = (5,'thread1'))
t.start()
#調(diào)用.join()方法來(lái)阻塞當(dāng)前線程,直到該線程結(jié)束后,才跳轉(zhuǎn)執(zhí)行其余的線程
#t.join() 如果加上這個(gè)語(yǔ)句,程序需要等整個(gè)sleeper執(zhí)行完畢后才會(huì)跳回執(zhí)行后面的兩次print
print('hello')
print('hello')
>>
Hi, I`m thread1. I`m going to sleep 5 seconds.
hello
hello
thread1 has woken up from sleep.
e.g.2
def main():
start = time.time()
thread_list = []
for i in range(1,6):
t = threading.thread(target=sleeper, args=(5,'thread{}'.format(i)))
thread_list.append(t)
t.start()
print('{} has started.'.format(t.name)
for t in thread_list:
t.join()
end = time.time()
print('total time: {}'.format(end - start))
print('aaa')
>>
Hi, I`m thread1. I`m going to sleep 5 seconds
Thread-1 has started.
Hi, I`m thread2. I`m going to sleep 5 seconds
Thread-2 has started.
Hi, I`m thread3. I`m going to sleep 5 seconds
Thread-3 has started.
Hi, I`m thread4. I`m going to sleep 5 seconds
Thread-4 has started.
thread2 has woken up from sleep
thread1 has woken up from sleep
thread4 has woken up from sleep
thread3 has woken up from sleep
total time: 5.001285791397095
aaa
e.g.3
def main():
start = time.time()
thread_list = []
for i in range(1,6):
t = threading.thread(target=sleeper, args=(5,'thread{}'.format(i)))
thread_list.append(t)
t.start()
t.join() #阻塞進(jìn)程
print('{} has started.'.format(t.name)
end = time.time()
print('total time: {}'.format(end - start))
print('aaa')
>>
Hi, I`m thread1. I`m going to sleep 5 seconds
thread1 has woken up from sleep
Thread-1 has started.
Hi, I`m thread2. I`m going to sleep 5 seconds
thread2 has woken up from sleep
Thread-2 has started.
Hi, I`m thread3. I`m going to sleep 5 seconds
thread3 has woken up from sleep
Thread-3 has started.
Hi, I`m thread4. I`m going to sleep 5 seconds
thread4 has woken up from sleep
Thread-4 has started.
total time: 20.00114393234253
aaa
線程之間通信的方法
queue:
- queue是線程安全的,其源碼中已經(jīng)使用了鎖
- queue也是共享變量的一種,只是它自身已經(jīng)具有線程安全性。如果換成其他變量可能會(huì)出現(xiàn)不同步的問(wèn)題。
- 常用方法:
from queue import Queue
-
put(self, item, block=True, timeout=None)
:將元素加入隊(duì)列,隊(duì)列為滿時(shí)為阻塞,可以設(shè)定timeout參數(shù) -
get(self, block=True, timeout=None)
:從隊(duì)列中取出元素,隊(duì)列為空時(shí)阻塞,可以設(shè)定timeout參數(shù) -
qsize()
:返回隊(duì)列的大小 -
empty()
:判斷是否為空 -
full()
:判斷是否為滿 -
join()
:阻塞直到隊(duì)列中的所有元素都被取出執(zhí)行。在調(diào)用該方法前要使用task_done()
方法,否則隊(duì)列將一直處在阻塞狀態(tài) -
task_done()
:指示之前的隊(duì)列任務(wù)已經(jīng)完成 -
put_nowait(self, item)
:非阻塞模式,隊(duì)列滿時(shí)直接拋出異常 -
get_nowait(self)
:非阻塞模式,隊(duì)列空時(shí)直接拋出異常
condition
from threading import Condition
條件變量,用于復(fù)雜的線程間的同步。
threading中的
Queue
和semaphore
的實(shí)現(xiàn)都使用了conditionCondition
中定義了__enter__
和__exit__
方法,所以可以使用with
語(yǔ)句實(shí)現(xiàn)上下文管理-
常用方法:
-
Condition中實(shí)際上有兩層鎖:
- 初始化時(shí)會(huì)上一把默認(rèn)的遞歸鎖鎖住condition,這把鎖會(huì)在調(diào)用
wait()
方法時(shí)被解除,稱(chēng)為底層鎖 - 在調(diào)用
wait()
方法時(shí)還會(huì)分配一把鎖將當(dāng)前進(jìn)程鎖住,并將這把鎖添加到一個(gè)雙端隊(duì)列中去。等到調(diào)用notify()
方法時(shí)該鎖會(huì)被釋放掉
def __init__(self, lock=None): if lock is None: lock = RLock() self._lock = lock # Export the lock's acquire() and release() methods self.acquire = lock.acquire self.release = lock.release def wait(self, timeout=None): if not self._is_owned(): raise RuntimeError("cannot wait on un-acquired lock") waiter = _allocate_lock() waiter.acquire() #分配一把鎖鎖住當(dāng)前進(jìn)程 self._waiters.append(waiter) #將鎖加入到一個(gè)雙端隊(duì)列中去 saved_state = self._release_save() #釋放掉初始化conditon時(shí)加上的底層鎖 def notify(self, n=1): if not self._is_owned(): raise RuntimeError("cannot notify on un-acquired lock") all_waiters = self._waiters waiters_to_notify = _deque(_islice(all_waiters, n)) if not waiters_to_notify: return for waiter in waiters_to_notify: waiter.release() #釋放掉雙端隊(duì)列中的鎖 try: all_waiters.remove(waiter) except ValueError: pass
- 初始化時(shí)會(huì)上一把默認(rèn)的遞歸鎖鎖住condition,這把鎖會(huì)在調(diào)用
-
wait(self, predicate, timeout=None)
:Wait until notified or until a timeout occurs.
This method releases the underlying lock, and then blocks until it is awakened by a notify() or notify_all() for the same condition variable in another thread, or until the optional timeout occurs.
釋放掉當(dāng)前線程的底層鎖,使得其他線程可以訪問(wèn)當(dāng)前線程。同時(shí)為當(dāng)前線程加上一把鎖,在收到
notify()
的通知之前阻塞當(dāng)前線程剩余的部分繼續(xù)執(zhí)行。即:在調(diào)用了
wait()
方法后,需要等待notify()
的通知,才能啟動(dòng)。
-
notify(self, n=1)
:Wake up one or more threads waiting on this condition.
釋放掉當(dāng)前線程的鎖,使得其余線程的
wait()
方法可以獲得鎖
-
-
class PersonA(threading.Thread): def __init__(self,cond): super().__init__(name="A") self.cond = cond def run(self): with self.cond: #為PersonA加上一把底層鎖 print('{}: Hello B,你在嗎?'.format(self.name)) self.cond.notify() #給B發(fā)出通知,釋放掉B中wait加上的鎖 self.cond.wait() #釋放掉A的底層鎖,確保下一次收到notify()通知后線程能夠切入。同時(shí)給A加上一把鎖,阻塞之后程序的運(yùn)行 class PersonB(threading.Thread): def __init__(self,cond): super().__init__(name="B") self.cond = cond def run(self): with self.cond: #為PersonB加上一把底層鎖 self.cond.wait() #使用wait方法阻塞后續(xù)程序,需要等待A發(fā)出通知,同時(shí)釋放掉B的底層鎖,保證線程可以切入進(jìn)來(lái) print('{}: Hello A,我在'.format(self.name)) self.cond.notify() #給A發(fā)出一個(gè)通知,同時(shí)釋放掉A中wait加上的鎖 if __name__ == '__main__': cond = threading.Condition() A = PersonA(cond) B = PersonB(cond) B.start() A.start() #如果先調(diào)用A.start(),程序會(huì)阻塞,無(wú)法繼續(xù)進(jìn)行,原因在于先啟動(dòng)A時(shí),notify()方法已經(jīng)被發(fā)出,而此時(shí)B的wait()方法還沒(méi)有啟動(dòng),也就沒(méi)法接受到notify()的通知,所以程序無(wú)法再繼續(xù) >> A: Hello B,你在嗎? B: Hello A,我在
- 注意
start()
的順序很重要,錯(cuò)誤的start()
順序會(huì)導(dǎo)致阻塞,程序無(wú)法繼續(xù)進(jìn)行。
- 注意
信號(hào)量(semaphore)
- 用于控制進(jìn)入數(shù)量的鎖
- 文件的讀寫(xiě),通常情況下讀可以由多個(gè)線程來(lái)讀,而寫(xiě)僅由一個(gè)線程來(lái)寫(xiě)
-
threading.semaphore(n)
:可以同時(shí)允許n個(gè)線程執(zhí)行,大于n的部分必須等之前有線程釋放后才能加入繼續(xù)執(zhí)行。
concurrent.futures
封裝好的方法,為多線程和多進(jìn)程提供了統(tǒng)一接口,方便調(diào)用。
-
優(yōu)勢(shì):
- 主線程中可以獲取某一個(gè)線程的狀態(tài)或者某一任務(wù)的狀態(tài),以及返回值
- 當(dāng)一個(gè)線程完成的時(shí)候我們主線程能立即知道
from concurrent.futures import ThreadPoolExecutor from concurrent.futures import Future def get_html(times): time.sleep(times) print('get page {} success'.format(times)) return times executor = ThreadPoolExecutor(max_workers=2) #最大線程數(shù)設(shè)定為2 task1 = executor.submit(get_html,(3)) #使用submit方法來(lái)提交線程 task2 = executor.submit(get_html,(2)) #executor返回的是一個(gè)Future對(duì)象 task1.done() #用于判定某個(gè)任務(wù)是否完成,該方法沒(méi)有阻塞,是Future的方法 task2.cancel() #取消線程,只能在線程還沒(méi)有開(kāi)始前取消,若線程已經(jīng)開(kāi)始執(zhí)行,則無(wú)法取消 task1.result() #返回線程執(zhí)行的結(jié)果,有阻塞,只有在線程結(jié)束后才有返回值
-
常用方法:
-
as_completed(fs, timeout=None)
:- 生成器,返回已經(jīng)完成的
Future
- 返回結(jié)果不一定與參數(shù)的傳入順序相同,誰(shuí)先完成就先返回誰(shuí)
#上例中若只獲取已經(jīng)成功了的task的結(jié)果 from concurrent.futures import as_completed urls = [3,4,2] all_task = [executor.submit(get_html, (url)) for url in urls] for future in as_completed(all_task): #無(wú)阻塞的方法,一旦子線程完成執(zhí)行,主線程中即可立馬返回相應(yīng)的結(jié)果 data = future.result() print("get {} page success".format(data))
- 生成器,返回已經(jīng)完成的
-
.map()
:- 類(lèi)似于python中的
map()
函數(shù),將可迭代的參數(shù)列表作用在同一函數(shù)上,其本質(zhì)依舊是一個(gè)生成器,直接返回future.result()
- 返回的順序與調(diào)用可迭代參數(shù)的順序一致
#只獲取已經(jīng)成功了的task的結(jié)果的另一種方法 urls = [3,4,2] for future in executor.mao(get_html, urls): print('get {} page success'.format(data))
- 類(lèi)似于python中的
wait(fs, timeout=None, return_when=ALL_COMPLETED)
:阻塞主線程,直到指定的某些子線程完成后才能繼續(xù)執(zhí)行主線程(在return_when參數(shù)中設(shè)定)with ThreadPoolExecutor(n) as executor:
使用with語(yǔ)句來(lái)進(jìn)行調(diào)用
-
Future
-
submit(self, fn, *args, **kwargs)
:class ThreadPoolExecutor(_base.Executor): # Used to assign unique thread names when thread_name_prefix is not supplied. _counter = itertools.count().__next__ def __init__(self, max_workers=None, thread_name_prefix=''): """Initializes a new ThreadPoolExecutor instance. Args: max_workers: The maximum number of threads that can be used to execute the given calls. thread_name_prefix: An optional name prefix to give our threads. """ if max_workers is None: # Use this number because ThreadPoolExecutor is often # used to overlap I/O instead of CPU work. max_workers = (os.cpu_count() or 1) * 5 if max_workers <= 0: raise ValueError("max_workers must be greater than 0") self._max_workers = max_workers self._work_queue = queue.Queue() self._threads = set() self._shutdown = False self._shutdown_lock = threading.Lock() self._thread_name_prefix = (thread_name_prefix or ("ThreadPoolExecutor-%d" % self._counter())) def submit(self, fn, *args, **kwargs): with self._shutdown_lock: #添加鎖,為了鎖住后面的一段代碼 if self._shutdown: raise RuntimeError('cannot schedule new futures after shutdown') f = _base.Future() #生成一個(gè)Future對(duì)象,最后這個(gè)對(duì)象會(huì)被返回 w = _WorkItem(f, fn, args, kwargs) #WorkItem才是整個(gè)線程池的真正執(zhí)行單元,F(xiàn)uture對(duì)象會(huì)被傳入WorkItem中 self._work_queue.put(w) #整個(gè)WorkItem實(shí)例被放入到_work_queue中 self._adjust_thread_count() return f submit.__doc__ = _base.Executor.submit.__doc__ def _adjust_thread_count(self): # When the executor gets lost, the weakref callback will wake up # the worker threads. def weakref_cb(_, q=self._work_queue): q.put(None) # TODO(bquinlan): Should avoid creating new threads if there are more # idle threads than items in the work queue. num_threads = len(self._threads) if num_threads < self._max_workers: #判斷啟動(dòng)了多少線程 thread_name = '%s_%d' % (self._thread_name_prefix or self, num_threads) t = threading.Thread(name=thread_name, target=_worker, args=(weakref.ref(self, weakref_cb), self._work_queue)) #線程執(zhí)行的函數(shù)是_worker,而_worker中執(zhí)行的函數(shù)來(lái)自self._work_queue #_worker函數(shù)的作用:運(yùn)行線程,不停的從_work_queue中取出_WorkItem t.daemon = True t.start() self._threads.add(t) #將啟動(dòng)了的線程數(shù)量加入到_threads中 _threads_queues[t] = self._work_queue class _WorkItem(object): #設(shè)定這個(gè)類(lèi)的目的其實(shí)就是執(zhí)行函數(shù),并將執(zhí)行的結(jié)果設(shè)置到Future中 def __init__(self, future, fn, args, kwargs): self.future = future self.fn = fn #需要執(zhí)行的函數(shù)在這里傳入 self.args = args self.kwargs = kwargs def run(self): if not self.future.set_running_or_notify_cancel(): return try: result = self.fn(*self.args, **self.kwargs) except BaseException as exc: self.future.set_exception(exc) # Break a reference cycle with the exception 'exc' self = None else: self.future.set_result(result) #將函數(shù)執(zhí)行的結(jié)果設(shè)置到Future中
-
Future
的常用方法:-
cancel(self)
:取消futuredef cancel(self): """Cancel the future if possible. Returns True if the future was cancelled, False otherwise. A future cannot be cancelled if it is running or has already completed. """ with self._condition: #使用條件變量 if self._state in [RUNNING, FINISHED]: #判斷future的狀態(tài)使否是在執(zhí)行或已完成,若是則無(wú)法取消 return False if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]: return True self._state = CANCELLED self._condition.notify_all() self._invoke_callbacks() return True
cancelled(self)
:狀態(tài)判斷,是否已經(jīng)取消running(self)
:狀態(tài)判斷,是否在執(zhí)行done(self)
:狀態(tài)判斷,是否已完成或已取消-
result(self, timeout=None)
:- 返回調(diào)用的結(jié)果,是有阻塞的方法,依舊是使用condition來(lái)實(shí)現(xiàn)
- 其中調(diào)用了condition的
wait()
方法
-
set_result(self, result)
:set the return value of work associated with the future
調(diào)用了condition的
notify_all()
方法
-
多進(jìn)程
在windows下使用多進(jìn)程,不管是ProcessPoolExecutor
還是multiprocessing
,代碼要放到if __ name__ == '__ main __'
模塊中,否則會(huì)報(bào)錯(cuò)
from concurrent.futures import ProcessPoolExecutor, as_completed
def fib(b):
if n <= 2:
return 1
return fib(n-1)+fib(n-2)
if __name__ == '__main__':
with ProcessPoolExecutor(3) as executor:
all_task = [executor.submit(fib,(num)) for num in range(15,45)]
for future in as_completed(all_task):
data = future.result()
print('result: {}'.format(data))
- 在處理CPU密集型問(wèn)題時(shí),使用多進(jìn)程會(huì)比使用多線程快,但是很難做到翻倍,因?yàn)檫M(jìn)程的切換比線程的切換更耗時(shí)。
multiprocessing
import time
import multiprocessing
def get_html(n):
time.sleep(n)
return n
if __name__ == '__main__':
progress = multiprocessing.Process(target=get_html, args=(2,))
progress.start()
progress.join()
print(progress.pid) #打印進(jìn)程的ID
multiprocessing 的進(jìn)程池
pool = multiprocessing.Pool(multiprocessing.cpu_count())
result = pool.apply_async(get_html, args=(3,))#進(jìn)程的提交
pool.close() #在調(diào)用.join之前要先調(diào)用.close,關(guān)閉進(jìn)程池使其不再接受進(jìn)程,否則.join方法會(huì)出錯(cuò)
pool.join()
print(result.get())
-
常用方法:
apply_async(self, func, args=(), kwds={}, callback=None, error_callback=None)
:apply()
方法的異步版本apply(self, fuc, args=(), kwds={})
:添加進(jìn)程-
imap(self, func, iterable, chunksize=1)
:對(duì)應(yīng)到之前executor里的map
方法pool = multiprocessing.Pool(multiprocessing.cpu_count()) for result in pool.imap(get_html, [1,5,3]): print('{} sleep success'.format(result)) >> 1 sleep success 5 sleep success 3 sleep success
-
imap_unordered(self, func, iterable, chunksize=1)
:先執(zhí)行完的先返回for result in pool.imap_unordered(get_html, [1,5,3]): print('{} sleep success'.format(result)) >> 1 sleep success 3 sleep success 5 sleep success
進(jìn)程之間的通信
-
多進(jìn)程中的
Queue
與多線程中的不一樣,這里要使用來(lái)自maltiprocessing
里的Queue
from multiprocessing import Process,Queue import multiprocessing def producer(queue): queue.put("a") time.sleep(2) def consumer(queue): time.sleep(2) data = queue.get() print(data) if __name__ == "__main__": queue = Queue(10) my_producer = Process(target=producer, args=(queue,)) my_consumer = Process(target=consumer, args=(queue,)) my_producer.start() my_consumer.start() my_producer.join() my_consumer.join()
-
maltiprocessing
里創(chuàng)建的Queue
不能用在Pool
創(chuàng)建的進(jìn)程池中from multiprocessing import Pool,Queue if __name__ == "__main__": queue = Queue(10) pool = Pool(2) pool.apply_async(producer, args=(queue,)) pool.apply_async(consumer, args=(queue,)) pool.close() pool.join() #沒(méi)有任何結(jié)果返回
-
進(jìn)程池中進(jìn)程的通信需要用來(lái)自
Manager
里的Queue
from maltiprocessing import Manager if __name__ == "__main__": queue = Manager().Queue(10) pool = Pool(2) pool.apply_async(producer, args=(queue,)) pool.apply_async(consumer, args=(queue,)) pool.close() pool.join()
-
使用pipe進(jìn)行通信
pipe只能適用于兩個(gè)進(jìn)程間的通信
-
pipe返回的是
Connection
的實(shí)例,Connection
的常用方法有:send(self, obj)
和recv(self)
def Pipe(duplex=True): return Connection(), Connection() class Connection(object): def send(self, obj): pass def recv(self): pass
pipe的性能比queue要高
from maltiprocessing import Pipe def producer(pipe): pipe.send("bobby") def consumer(pipe): print(pipe.recv()) if __name__ == "__main__": recevie_pipe, send_pipe=Pipe() #管道的兩端,發(fā)送端和接收端 my_producer= Process(target=producer, args=(send_pipe, )) my_consumer = Process(target=consumer, args=(recevie_pipe,)) my_producer.start() my_consumer.start() my_producer.join() my_consumer.join()
共享全局變量在多進(jìn)程中是不適用的
def producer(a):
a += 100
time.sleep(2)
def consumer(a):
time.sleep(2)
print(a)
if __name__ == '__main__':
a = 1
my_producer = Process(target=producer,args=(a,))
my_consumer = Process(target=consumer,args=(a,))
my_producer.start()
my_consumer.start()
my_producer.join()
my_consumer.join()
>> 1 #返回值還是1,不是101
-
進(jìn)程間的同步(維護(hù)一個(gè)公共的內(nèi)存/變量)使用
Manager()
def Manager(): return multiprocessing.SyncManager() class SyncManager(multiprocessing.managers.BaseManager): def Condition(self, lock=None): return threading.Condition(lock) def Event(self): return threading.Event() def Lock(self): return threading.Lock() def Namespace(self): pass def Queue(self, maxsize=None): return queue.Queue() def RLock(self): return threading.RLock() def Semaphore(self, value=None): return threading.Semaphore(value) def Array(self, typecode, sequence): pass def Value(self, typecode, value): pass def dict(self, mapping_or_sequence): pass def list(self, sequence): pass
def add_data(p_dict, key, value): p_dict[key] = value if __name__ == '__main__': progress_dict = Manager().dict() first = Process(target=add_data, args=(progress_dict, 'Bobby1',22)) second = Process(target=add_data, args=(progress_dict, 'Bobby2',23)) first.start() second.start() first.join() second.join() print(progress_dict) >> {'bobby1': 22, 'bobby2': 23}
使用時(shí)注意數(shù)據(jù)的同步,必要時(shí)需要加鎖
文檔閱讀
Thread Objects
- Thread類(lèi):在單獨(dú)的控制線程中運(yùn)行的活動(dòng)(activity),有兩種指定活動(dòng)的方法:
- 將可調(diào)用對(duì)象傳遞給構(gòu)造器
- 在子類(lèi)中重寫(xiě)
run()
方法
- 通過(guò)調(diào)用thread對(duì)象的
start()
方法來(lái)激活對(duì)象,is_alive()
方法用來(lái)判斷線程是否處在激活狀態(tài)。 - 其他線程可以調(diào)用
join()
方法來(lái)阻塞正在調(diào)用的線程,直到調(diào)用join()
方法的線程結(jié)束。 -
threading.Thread(group=None, target=None, name=None, args=(), kwargs={}, *, daemon=None)
- target:傳入被
run()
方法調(diào)用的可調(diào)用對(duì)象 - name:線程的名字
- args:目標(biāo)調(diào)用的參數(shù)的元組
- kwargs:目標(biāo)調(diào)用的參數(shù)的字典
- target:傳入被
Lock Objects
-
threading.Lock()
生成一個(gè)鎖對(duì)象。一旦某個(gè)線程獲取了鎖,之后的所有獲取鎖的請(qǐng)求都會(huì)被阻塞,直到鎖被釋放。任何線程都可以釋放鎖,不一定是獲得鎖的線程。
RLock Objects
- 可重入鎖允許同一線程多次獲取同一把鎖。
Event Objects
- 一個(gè)事件對(duì)象管理線程內(nèi)的一個(gè)內(nèi)部標(biāo)記,可以使用
set()
方法將標(biāo)記設(shè)置為true
,使用clear()
方法將標(biāo)記重置為false
。標(biāo)記為false
時(shí),wait()
方法被阻塞,直到標(biāo)記被設(shè)定為true
。 -
threading.Event()
構(gòu)造一個(gè)事件對(duì)象,內(nèi)部標(biāo)記初始化為false
。 -
is_set()
:檢查內(nèi)部標(biāo)記的狀態(tài),只有在狀態(tài)為為true
時(shí)返回true
。 -
set()
:將內(nèi)部標(biāo)記設(shè)定為true
。完成設(shè)定后,所有調(diào)用了wait()
方法的線程都將不再受到阻塞。 -
clear()
:將內(nèi)部標(biāo)記重置為false
。 -
wait(timeout=None)
:阻塞進(jìn)程直到內(nèi)部標(biāo)記設(shè)定為true
或滿足選定的timeout。- timeout:a floating point number,以秒為單位設(shè)定超時(shí)時(shí)間
queue
- 同步隊(duì)列:隊(duì)列模塊實(shí)現(xiàn)了多生產(chǎn)者、多消費(fèi)者隊(duì)列。當(dāng)必須在多個(gè)線程之間安全地交換信息時(shí),它在線程編程中特別有用。
- 該模塊實(shí)現(xiàn)了三種隊(duì)列:
- FIFO隊(duì)列:先添加的任務(wù)先讀取(先入先出)
- LIFO隊(duì)列:后添加的任務(wù)先讀取(后入先出)
- 優(yōu)先隊(duì)列:輸入的數(shù)據(jù)排序,值最小的數(shù)據(jù)先讀取
- 類(lèi):
- Queue(maxsize=0):FIFO隊(duì)列的構(gòu)造函數(shù)
- LifoQueue(maxsize=0):LIFO隊(duì)列的構(gòu)造函數(shù)
- PriorityQueue(maxsize=0):優(yōu)先隊(duì)列的構(gòu)造函數(shù)
- maxsize參數(shù)用于設(shè)定可以放入隊(duì)列的項(xiàng)目數(shù)的上限。若maxsize小于或等于零,則隊(duì)列大小為無(wú)限大。
Queue Objects
-
Queue.qsize()
:返回隊(duì)列的大小 -
Queue.empty()
:檢查隊(duì)列是否為空,是則返回True
-
Queue.full()
:檢查隊(duì)列是否為滿,是則返回True
-
Queue.put(item, block=True, timeout=None)
:將元素放入隊(duì)列-
block=True, timeout=None
:在必要時(shí)阻塞,當(dāng)隊(duì)列有空位時(shí)執(zhí)行插入 - timeout為正數(shù):阻塞最多timeout秒,若隊(duì)列還是沒(méi)有空位則拋出Full異常
- block=False:在有空位時(shí)直接插入,否則拋出異常
-
-
Queue.get(block=True,timeout=None)
:將元素從隊(duì)列中移除并返回
(未完)