如果Python Books是一些指導(dǎo),那么,coroutines是最少被記載,晦澀的,看上去沒(méi)什么用的功能。 -- David Beazley, Python author
caller從generator中拉取數(shù)據(jù)。
一個(gè)coroutine從結(jié)構(gòu)上像是一個(gè)generator,只是一個(gè)包含yield關(guān)鍵字的函數(shù)而已。
coroutine可以從caller中接收數(shù)據(jù),通過(guò).send() 代替 .next()。甚至,yield關(guān)鍵字也可以沒(méi)有數(shù)據(jù)流入流出。除了數(shù)據(jù)流,yield也是一個(gè)控制流程的設(shè)備,使多任務(wù)間協(xié)作。(yield的本身意思就是“放棄”)每個(gè)coroutine yields 控制返還給scheduler,因此,其他的coroutine被激活。
當(dāng)你把yield主要想成流程控制的手段時(shí),你就快理解coroutine了。
Python coroutine是一系列改進(jìn)簡(jiǎn)陋的generator的成果。coroutine就是從PEP342中引入,即Coroutines via Enhandced Generators,Python2.5。通過(guò)send傳入數(shù)據(jù),這讓generator可以被用作coroutine,一個(gè)協(xié)作的過(guò)程,yielding and receiving values from the caller。
除了send,PEP 342還加入了throw和close,允許caller拋出異常,該異常在generator里面處理,也可以結(jié)束generator。PEP 380讓coroutine支持return和yield from。
coroutine可以處于4種狀態(tài),通過(guò)inspect.getgeneratorstate()函數(shù)來(lái)確定狀態(tài):
1.GEN_CREATED,等待開(kāi)始
2.GEN_RUNNING,被解釋器執(zhí)行
3.GEN_SUSPENED,在yield表達(dá)式處掛起
4.GEN_CLOSED,執(zhí)行結(jié)束
caller通過(guò)send傳入coroutine的數(shù)據(jù),會(huì)在yield處獲得,所以,coroutine必須處于yield出掛起時(shí)(GEN_SUSPENDED),才能通過(guò)send傳入數(shù)據(jù)。這也就解釋了,coroutine第一次被激活,一定是通過(guò)next,否則處于GEN_CREADED狀態(tài)下,無(wú)法send。初始的next調(diào)用,是為了讓coroutine準(zhǔn)備好。值得說(shuō)的是,如果coroutine中有這樣的表達(dá)式“b = yield a”,那么此處,caller是必須send一個(gè)值進(jìn)來(lái),也就是綁定給b的值,要不然b為None。
重要的一點(diǎn),要理解為什么coroutine的執(zhí)行流程,會(huì)恰好在yield處掛起,確切的說(shuō),如果b = yield a,是在yield a掛起,b =在下次執(zhí)行。因?yàn)檫@個(gè)表達(dá)式“b = yield a”,只有在協(xié)程被客戶端激活之后,b的值才能被設(shè)置。這對(duì)理解異步編程有用。
第一次調(diào)用next,讓coroutine處于第一個(gè)yield處掛起,這個(gè)準(zhǔn)備過(guò)程叫做coroutine priming,為了更方便,引入decorator,@coroutine。
對(duì)于yield from,在調(diào)用時(shí)自動(dòng)prime協(xié)程,所以,與yield from搭配的asyncio.coroutine,其實(shí)沒(méi)有做prime工作。
終止協(xié)程和異常處理
協(xié)程內(nèi)一個(gè)未處理的異常會(huì)隨著send或next傳播到caller中。
generator.throw
導(dǎo)致generator被掛起的yield處,拋出異常。如果generator處理異常,則當(dāng)前yield為throw本身,next會(huì)繼續(xù)到下一個(gè)yield。如果generator不處理異常,異常傳播給caller,狀態(tài)為GEN_CLOSED。
generator.close
導(dǎo)致generator被掛起的yield處,拋出GeneratorExit異常,狀態(tài)為GEN_CLOSED。close不會(huì)yield 值。
從coroutine中返回值
一些coroutine不會(huì)yield有趣的值,而是被設(shè)計(jì)成返回一個(gè)結(jié)果,一個(gè)累積的最后結(jié)果。
在coroutine中的return value語(yǔ)句,value回作為StopIteration的值偷偷地傳回給caller。這是一個(gè)hack,但是又符合coroutine的行為:在exhausted時(shí)拋出StopIteration,我們不得不這樣去獲取coroutine的返回值:
try:
r = coro.send(None)
except StopIteration as exc:
r = exc.value
Ok,PEP380引入了yield from語(yǔ)法,它會(huì)自動(dòng)地在內(nèi)部捕獲StopIteration。
PEP380的標(biāo)題是“Syntax for Delegating to a Subgenerator”。
yield from語(yǔ)法最主要的功能是,打開(kāi)一個(gè)雙向的通道,在外部的caller與內(nèi)部的subgenerator之間,我們可以send數(shù)據(jù)進(jìn)去,也可以yield數(shù)據(jù)回來(lái),而不用寫異常處理代碼。來(lái)看3個(gè)名詞解釋:
delegating generator
The generator function that contain the yield from <iterable> expression。
subgenerator
The generator obtained from the <iterable> part of the yield from expression。
caller
PEP380 uses the term "caller" to refer to the client code that calls the delegating generator. Depending on the context, I use "client" instead of "caller", to distinguish from the delegating generator, which is also a "caller"(it calls the subgenerator).
While the delegating generator is suspended at yield from, the caller sends data directly to the subgenerator, which yields data back to the caller. thee delegating generator resumes when the subgenerator returns and the interpreter raise StopIteration with the returned value attached.
delegating generator看不到caller傳給subgenerator的值。
書中一個(gè)例子16-17非常好,結(jié)論:
如果一個(gè)subgenerator不曾終止,那么,delegating generator就不會(huì)被激活,一直處于掛起狀態(tài),一直掛在yield from處。但是,程序仍會(huì)進(jìn)行,因?yàn)閥ield from返還控制權(quán)給client,就想yield一樣,只是有些任務(wù)處于未完成狀態(tài)。
one delegating generator uses yield from to call a subgenerator, which itself is a delegating generator calling another subgenerator with yield from, and so on. Eventually, this chain must end in a simple generator that use just yield, or a iterable object.
yield from must be driven by a client that calls next() or send().
"when the iterator is another generator, the effect is the same as if the body of the subgenerator were inlined at the point of the yield from expression. Furthermore, the subgenerator is allowed to execute a return statement with a value, and that value become the value of the yield from expression.
PEP380中關(guān)于yield from語(yǔ)法的六點(diǎn)說(shuō)明:
1.any values that the subgenerator yields are passed directly to the caller of the delegating generator(i.e., the client code).
2.any values sent to the delegating generator using send() are passed directly to the subgenerator. If the sent value is None, the subgenerator's next() method is called. If the sent value is not none the subgenerator's send() method is called. If the call raise StopIteration, the delegating generator is resumed. Any other exception is propagated to the delegating generator.
3.return expr in a generator or a subgenerator causes StopIteration(expr) to be raised upon exit from the generator.
4.The value of the yield from expression is the first argument to the StopIteration exception raised by the subgenerator when it terminates.
5.Exceptions other than GeneratorExit thrown into the delegating generator are passed to the throw() method of the subgenerator. If the call raise StopIteration, the delegating generator is resumed. Any other exception is propagated to the delegating generator.
6.If a GeneratorExit exception is thrown into the delegating generator, or the close() method of the delegating generator is called, then the close() method of the subgenerator is called if it has one. If this call results in an exception, it is propagated to the delegating generator. Otherwise, GeneratorExit is raised in the delegating generator.
現(xiàn)實(shí)是復(fù)雜的,因?yàn)槲覀冃枰幚砜蛻舳说膖hrow和close。PEP380的偽代碼被作者給予高度評(píng)價(jià),讀了三遍才明白。要好好讀這個(gè)代碼,因?yàn)榇蠖鄶?shù)的yield from例子都在asyncio中,不是獨(dú)立的例子。
再?gòu)?qiáng)調(diào)一下,yield from是auto prime哦。
This is a form of multitasking: coroutines voluntarily and explicitly yield control to the central scheduler.
廣義協(xié)程vsAsyncio協(xié)程
a broad, informal definition of a coroutine: a generator function driven by a client sending it data through send() calls or yield from. This broad definition is the one used in PEP 342 -- Coroutines via Enhanced Generators and in most existing Python books.
the asyncio coroutine, a stricter definition: asyncio coroutines are (usually) decorated with an @asyncio.coroutine decorator, and they are always driven by yield from, not by calling send() directly on them. Of course, asyncio coroutines are driven by next() and send() under the covers, but in user code, we only use yield from to make them run.