golang中啟動一個協程不會消耗太多資源，有人認為可以不用協程池。但是當訪問量增大時，可能造成內存消耗完，程序崩潰。于是寫了一個協程池的Demo。
Demo中有worker和job。worker是一個協程，在worker中完成一個job。Jobs是一個channel，使用Jobs記錄job。當生成一個新任務，就發送到Jobs中。程序啟動時，首先啟動3個worker協程，每個協程都嘗試從Jobs中接收job。如果Jobs中沒有job，worker協程就等待。
基本邏輯如下：
- Jobs管道存放job，Results管道存放結果。
- 程序一啟動，啟動3個worker協程，等待從Jobs管道中取數據。
- 向Jobs管道中發送3個數據。
- 關閉Jobs管道。
- worker協程從Jobs管道中接收到數據以后，執行程序，把結果放到Results管道中。然后繼續等待。
- 當Jobs管道中沒有數據，并且Results有3個數據時。退出主程序。
代碼如下：
package main
import (
"fmt"
"time"
)
func worker(id int) {
go func() {
for {
fmt.Println("Waiting for job...")
select {
// Receive from channel
case j := <-Jobs :
fmt.Println("worker", id, "started job", j)
time.Sleep(time.Second)
fmt.Println("worker", id, "finished job", j)
Results <- true
}
}
}()
}
const channelLength = 3
var (
Jobs chan int
Results chan bool
)
func main() {
Jobs = make(chan int, channelLength)
Results = make(chan bool, channelLength)
// Start worker goroutines
for i:= 0; i < channelLength; i++ {
worker(i)
}
// Send to channel
time.Sleep(time.Second)
for j := 0; j < channelLength; j++ {
Jobs <- j
}
close(Jobs)
for len(Jobs) != 0 || len(Results) != channelLength {
time.Sleep(100 * time.Millisecond)
}
fmt.Println("Complete main")
}
運行結果如下：
Waiting for job...
Waiting for job...
Waiting for job...
worker 1 started job 2
worker 2 started job 0
worker 0 started job 1
worker 0 finished job 1
Waiting for job...
worker 0 started job 0
worker 2 finished job 0
Waiting for job...
worker 2 started job 0
worker 1 finished job 2
Waiting for job...
worker 1 started job 0
Complete main
這個程序出現問題了，bug在哪里？
開始的3次，協程運行都是正常。
worker 1 started job 2
worker 2 started job 0
worker 0 started job 1
worker 0 finished job 1
worker 2 finished job 0
worker 1 finished job 2
根據設計，向Jobs管道中發送3個數據以后，就關閉了管道。此后，協程不應該再從Jobs管道中接收到數據。
for j := 0; j < channelLength; j++ {
jobs <- j
}
close(jobs)
實際運行中，協程接收完3個數據以后，worker還能不斷的從Jobs管道中接收到數據。與設計不符。
worker 0 started job 0
worker 2 started job 0
worker 1 started job 0
開始以為問題出在worker()中，j := <- job，只有當job中有返回，才會打印worker started。但是后面的job id都是0，說明沒有向jobs管道中發送新數據。
for {
fmt.Println("Waiting for job...")
select {
case j := <-Jobs :
fmt.Println("worker", id, "started job", j)
time.Sleep(time.Second)
fmt.Println("worker", id, "finished job", j)
Results <- true
}
}
研究向Jobs管道發送數據的代碼，突發奇想，把close(Jobs)注釋掉，看看如何。
for j := 0; j < channelLength; j++ {
Jobs <- j
}
//close(Jobs)
程序居然正常了。
Waiting for job...
Waiting for job...
Waiting for job...
worker 1 started job 0
worker 0 started job 2
worker 2 started job 1
worker 1 finished job 0
worker 0 finished job 2
Waiting for job...
Waiting for job...
worker 2 finished job 1
Waiting for job...
Complete main
原來問題出在close()上，馬上查注釋。close()是在sender中調用，當管道中最后一個數據被接收以后，就關閉管道。此時，不能再向管道中發送數據。否則會報錯panic: send on closed channel。
使用x, ok := <-c可以判斷一個管道是否關閉，如果管道已經關閉，ok的值為false。
管道關閉以后，并且管道中的數據被接收完以后，居然還能從管道中接收到數據0。于是就造成了后續協程接收到job 0的問題。
// The close built-in function closes a channel, which must be either
// bidirectional or send-only. It should be executed only by the sender,
// never the receiver, and has the effect of shutting down the channel after
// the last sent value is received. After the last value has been received
// from a closed channel c, any receive from c will succeed without
// blocking, returning the zero value for the channel element. The form
// x, ok := <-c
// will also set ok to false for a closed channel.
func close(c chan<- Type)
如果要使用close，應該怎么做
管道不用時，close()管道是個好習慣。此時，應該怎么解決這個問題呢？首先要在協程中檢查接收到的數據，j:=<-jobs，判斷j是否為0。如果Jobs中存放的是非指針數據，不能分辨0是真正的0值，還是close以后接收到的0。因此需要在Jobs管道中存放指針。管道打開時，接收的都是非nil指針。close以后才返回0，也就是nil指針。
修改程序。新生成一個機構體Job。
type Job struct {
JobId int
}
Jobs保存指向Job的指針。
Jobs chan *Job
func main() {
Jobs = make(chan *Job, channelLength)
...
for j := 0; j < channelLength; j++ {
Jobs <- &Job{JobId:j}
}
close(Jobs)
...
}
在worker協程中，從管道取出Job指針以后，判斷指針是否為nil。如果為nil，說明管道已經關閉，協程退出。
func worker(id int) {
go func() {
for {
fmt.Println("Waiting for job...")
select {
// Receive from channel
case j := <-Jobs :
if j == nil {
fmt.Println("Close the worker", id)
return
}
fmt.Println("worker", id, "started job", j.JobId)
time.Sleep(time.Second)
fmt.Println("worker", id, "finished job", j.JobId)
Results <- true
}
}
}()
}
運行結果達到預期。
Waiting for job...
Waiting for job...
Waiting for job...
worker 0 started job 0
worker 1 started job 1
worker 2 started job 2
worker 2 finished job 2
worker 0 finished job 0
Waiting for job...
Waiting for job...
Close the worker 2
Close the worker 0
worker 1 finished job 1
Waiting for job...
Close the worker 1
Complete main
附上最終的代碼。
package main
import (
"fmt"
"time"
)
type Job struct {
JobId int
}
func worker(id int) {
go func() {
for {
fmt.Println("Waiting for job...")
select {
// Receive from channel
case j := <-Jobs :
if j == nil {
fmt.Println("Close the worker", id)
return
}
fmt.Println("worker", id, "started job", j.JobId)
time.Sleep(time.Second)
fmt.Println("worker", id, "finished job", j.JobId)
Results <- true
}
}
}()
}
const channelLength = 3
var (
Jobs chan *Job
Results chan bool
)
func main() {
Jobs = make(chan *Job, channelLength)
Results = make(chan bool, channelLength)
// Start worker goroutines
for i:= 0; i < channelLength; i++ {
worker(i)
}
// Send to channel
time.Sleep(time.Second)
for j := 0; j < channelLength; j++ {
Jobs <- &Job{JobId:j}
}
close(Jobs)
for len(Jobs) != 0 || len(Results) != channelLength {
time.Sleep(100 * time.Millisecond)
}
fmt.Println("Complete main")
}
