golang 1.7之后高級(jí)測(cè)試方法之子測(cè)試,子基準(zhǔn)測(cè)試(subtest sub-benchmarks)

介紹

在go1.7之后,testing包T和B的引入了一個(gè)Run方法,用于創(chuàng)建subtests 和 sub-benchmarks. subtests 和 sub-benchmarks可以讓開(kāi)發(fā)者更好的處理測(cè)試中的失敗,更好的控制運(yùn)行哪個(gè)測(cè)試用例,控制并行測(cè)試操作,測(cè)試代碼更加簡(jiǎn)潔和可維護(hù)性更強(qiáng)。

Table-driven tests 基礎(chǔ)

首先我們先討論下Go中常見(jiàn)的測(cè)試代碼編寫方式。

一系列相關(guān)的測(cè)試校驗(yàn)可以通過(guò)遍歷測(cè)試用例的切片來(lái)實(shí)現(xiàn),代碼如下:

func TestTime(t *testing.T) {
    testCases := []struct {
        gmt  string
        loc  string
        want string
    }{
        {"12:31", "Europe/Zuri", "13:31"},     // incorrect location name
        {"12:31", "America/New_York", "7:31"}, // should be 07:31
        {"08:08", "Australia/Sydney", "18:08"},
    }
    for _, tc := range testCases {
        loc, err := time.LoadLocation(tc.loc)
        if err != nil {
            t.Fatalf("could not load location %q", tc.loc)
        }
        gmt, _ := time.Parse("15:04", tc.gmt)
        if got := gmt.In(loc).Format("15:04"); got != tc.want {
            t.Errorf("In(%s, %s) = %s; want %s", tc.gmt, tc.loc, got, tc.want)
        }
    }
}

測(cè)試函數(shù)必須以Test開(kāi)頭,Test后跟的名字也必須首字母大寫。
上面的測(cè)試方式稱為table-driven 測(cè)試法,可以降低重復(fù)代碼。

Table-driven benchmarks

在go1.7之前是不能夠?qū)enchmarks采用table-driven的方法的,如果要測(cè)試不同的參數(shù)就需要編寫不同的benchmark函數(shù),在go1.7之前常見(jiàn)的benchmarks測(cè)試代碼如下:

func benchmarkAppendFloat(b *testing.B, f float64, fmt byte, prec, bitSize int) {
    dst := make([]byte, 30)
    b.ResetTimer() // Overkill here, but for illustrative purposes.
    for i := 0; i < b.N; i++ {
        AppendFloat(dst[:0], f, fmt, prec, bitSize)
    }
}

func BenchmarkAppendFloatDecimal(b *testing.B) { benchmarkAppendFloat(b, 33909, 'g', -1, 64) }
func BenchmarkAppendFloat(b *testing.B)        { benchmarkAppendFloat(b, 339.7784, 'g', -1, 64) }
func BenchmarkAppendFloatExp(b *testing.B)     { benchmarkAppendFloat(b, -5.09e75, 'g', -1, 64) }
func BenchmarkAppendFloatNegExp(b *testing.B)  { benchmarkAppendFloat(b, -5.11e-95, 'g', -1, 64) }
func BenchmarkAppendFloatBig(b *testing.B)     { benchmarkAppendFloat(b, 123456789123456789123456789, 'g', -1, 64) }

go1.7之后,采用table-drive方法代碼如下:

func BenchmarkAppendFloat(b *testing.B) {
    benchmarks := []struct{
        name    string
        float   float64
        fmt     byte
        prec    int
        bitSize int
    }{
        {"Decimal", 33909, 'g', -1, 64},
        {"Float", 339.7784, 'g', -1, 64},
        {"Exp", -5.09e75, 'g', -1, 64},
        {"NegExp", -5.11e-95, 'g', -1, 64},
        {"Big", 123456789123456789123456789, 'g', -1, 64},
        ...
    }
    dst := make([]byte, 30)
    for _, bm := range benchmarks {
        b.Run(bm.name, func(b *testing.B) {
            for i := 0; i < b.N; i++ {
                AppendFloat(dst[:0], bm.float, bm.fmt, bm.prec, bm.bitSize)
            }
        })
    }
}

每個(gè)b.Run單獨(dú)創(chuàng)建一個(gè)benchmark。
可以看到新的編碼方式可讀性和可維護(hù)行上更強(qiáng)。

如果想要子測(cè)試并發(fā)執(zhí)行,則使用 b.RunParallel

Table-driven tests using subtests

Go1.7之后引用Run方法用于創(chuàng)建subtests,對(duì)之前 Table-driven tests 基礎(chǔ) 中的代碼重新寫為:

func TestTime(t *testing.T) {
    testCases := []struct {
        gmt  string
        loc  string
        want string
    }{
        {"12:31", "Europe/Zuri", "13:31"},
        {"12:31", "America/New_York", "7:31"},
        {"08:08", "Australia/Sydney", "18:08"},
    }
    for _, tc := range testCases {
        t.Run(fmt.Sprintf("%s in %s", tc.gmt, tc.loc), func(t *testing.T) {
            loc, err := time.LoadLocation(tc.loc)
            if err != nil {
                t.Fatal("could not load location")
            }
            gmt, _ := time.Parse("15:04", tc.gmt)
            if got := gmt.In(loc).Format("15:04"); got != tc.want {
                t.Errorf("got %s; want %s", got, tc.want)
            }
        })
    }
}

go1.7之前的 Table-driven tests 基礎(chǔ) 的測(cè)試代碼運(yùn)行結(jié)果為:

--- FAIL: TestTime (0.00s)
    time_test.go:62: could not load location "Europe/Zuri"

雖然兩個(gè)用例都是錯(cuò)誤的,但是 第一個(gè)用例Fatalf 后,后面的用例也就沒(méi)能進(jìn)行運(yùn)行。

使用Run的測(cè)試代碼運(yùn)行結(jié)果為:

--- FAIL: TestTime (0.00s)
    --- FAIL: TestTime/12:31_in_Europe/Zuri (0.00s)
        time_test.go:84: could not load location
    --- FAIL: TestTime/12:31_in_America/New_York (0.00s)
        time_test.go:88: got 07:31; want 7:31

Fatal 導(dǎo)致subtest被跳過(guò),不過(guò)不影響其他subtest以及父test的測(cè)試。

針對(duì)每一個(gè)子測(cè)試,go test命令都會(huì)打印出一行測(cè)試摘要。它們是分離的、獨(dú)立統(tǒng)計(jì)的。這可以讓我們進(jìn)行更加精細(xì)的測(cè)試,細(xì)到每次輸入輸出。

過(guò)濾執(zhí)行測(cè)試用例

subtests和sub-benchmarks可以使用 -run or -bench flag
來(lái)對(duì)測(cè)試用例進(jìn)行過(guò)濾運(yùn)行。 -run or -bench flag后跟以'/'分割的正則表達(dá)式,用來(lái)制定特定的測(cè)試用例。

  • 執(zhí)行TestTime下匹配"in Europe" 的子測(cè)試
$ go test -run=TestTime/"in Europe"
--- FAIL: TestTime (0.00s)
    --- FAIL: TestTime/12:31_in_Europe/Zuri (0.00s)
        time_test.go:85: could not load location
  • 執(zhí)行TestTime下匹配"12:[0-9] " 的子測(cè)試
$ go test -run=Time/12:[0-9] -v
=== RUN   TestTime
=== RUN   TestTime/12:31_in_Europe/Zuri
=== RUN   TestTime/12:31_in_America/New_York
--- FAIL: TestTime (0.00s)
    --- FAIL: TestTime/12:31_in_Europe/Zuri (0.00s)
        time_test.go:85: could not load location
    --- FAIL: TestTime/12:31_in_America/New_York (0.00s)
        time_test.go:89: got 07:31; want 7:31
$ go test -run=Time//New_York
--- FAIL: TestTime (0.00s)
    --- FAIL: TestTime/12:31_in_America/New_York (0.00s)
        time_test.go:88: got 07:31; want 7:31

func (*T) Parallel

func (t *T) Parallel()

使用t.Parallel(),使測(cè)試和其它子測(cè)試并發(fā)執(zhí)行。

 tc := tc這個(gè)地方很關(guān)鍵,不然多個(gè)子測(cè)試可能使用的tc是同一個(gè)。
func TestGroupedParallel(t *testing.T) {
    for _, tc := range testCases {
        tc := tc // capture range variable
        t.Run(tc.Name, func(t *testing.T) {
            t.Parallel()
            if got := foo(tc.in); got != tc.out {
                t.Errorf("got %v; want %v", got, tc.out)
            }
            ...
        })
    }
}

func (*B) RunParallel

func (b *B) RunParallel(body func(*PB))

RunParallel runs a benchmark in parallel. It creates multiple goroutines and distributes b.N iterations among them. The number of goroutines defaults to GOMAXPROCS. To increase parallelism for non-CPU-bound benchmarks, call SetParallelism before RunParallel. RunParallel is usually used with the go test -cpu flag.

The body function will be run in each goroutine. It should set up any goroutine-local state and then iterate until pb.Next returns false. It should not use the StartTimer, StopTimer, or ResetTimer functions, because they have global effect. It should also not call Run.

RunParallel并發(fā)的執(zhí)行benchmark。RunParallel創(chuàng)建多個(gè)goroutine然后把b.N個(gè)迭代測(cè)試分布到這些goroutine上。goroutine的數(shù)目默認(rèn)是GOMAXPROCS。如果要增加non-CPU-bound的benchmark的并個(gè)數(shù),在執(zhí)行RunParallel之前調(diào)用SetParallelism。

不要使用 StartTimer, StopTimer, or ResetTimer functions這些函數(shù),因?yàn)檫@些函數(shù)都是 global effect的。

package main

import (
    "bytes"
    "testing"
    "text/template"
)

func main() {
    // Parallel benchmark for text/template.Template.Execute on a single object.
    testing.Benchmark(func(b *testing.B) {
        templ := template.Must(template.New("test").Parse("Hello, {{.}}!"))
        // RunParallel will create GOMAXPROCS goroutines
        // and distribute work among them.
        b.RunParallel(func(pb *testing.PB) {
            // Each goroutine has its own bytes.Buffer.
            var buf bytes.Buffer
            for pb.Next() {
                // The loop body is executed b.N times total across all goroutines.
                buf.Reset()
                templ.Execute(&buf, "World")
            }
        })
    })
}

本人測(cè)試實(shí)例

Benchmark測(cè)試代碼


func BenchmarkProductInfo(b *testing.B) {
    // b.ResetTimer()

    testCases := []string{"pn3", "p7", "p666"}
    for _, productId := range testCases {
        // b.SetParallelism
        b.Run(productId, func(b *testing.B) {
            for i := 0; i < b.N; i++ {
                mgoDB.ecnGetProductInfoOfProductId(productId)
            }
        })
    }
}

func BenchmarkProductInfoParalle(b *testing.B) {
    // b.ResetTimer()

    testCases := []string{"pn3", "p7", "p666"}
    for _, tproductId := range testCases {
        // b.SetParallelism
        productId := tproductId
        b.RunParallel(func(b *testing.PB) {
            for b.Next() {
                mgoDB.ecnGetProductInfoOfProductId(productId)
            }

        })
    }
}

func BenchmarkProductLock(b *testing.B) {
    // b.ResetTimer()
    testCases := []string{"pn3", "p7", "p666"}
    for _, productId := range testCases {
        // b.SetParallelism
        b.Run(productId, func(b *testing.B) {
            for i := 0; i < b.N; i++ {
                mgoDB.CheckProductLockStatus(productId)
            }
        })
    }

}
func BenchmarkProductLockParallel(b *testing.B) {
    // b.ResetTimer()
    testCases := []string{"pn3", "p7", "p666"}
    for _, tproductId := range testCases {
        // b.SetParallelism
        productId := tproductId
        b.RunParallel(func(b *testing.PB) {
            for b.Next() {
                mgoDB.CheckProductLockStatus(productId)
            }
        })
    }

}

  • 執(zhí)行如下測(cè)試命令
 go test -bench="."

結(jié)果

BenchmarkProductInfo/pn3-4                 10000            107704 ns/op
BenchmarkProductInfo/p7-4                  10000            108921 ns/op
BenchmarkProductInfo/p666-4                10000            107163 ns/op
BenchmarkProductInfoParalle-4              10000            113386 ns/op
BenchmarkProductLock/pn3-4                 10000            100418 ns/op
BenchmarkProductLock/p7-4                  20000             97373 ns/op
BenchmarkProductLock/p666-4                20000             96905 ns/op
BenchmarkProductLockParallel-4             10000            108399 ns/op
  • 執(zhí)行如下測(cè)試命令
 go test -bench=ProductInfo

過(guò)濾測(cè)試函數(shù)名中包含ProductInfo的測(cè)試用例,結(jié)果:

BenchmarkProductInfo/pn3-4                 10000            111065 ns/op
BenchmarkProductInfo/p7-4                  10000            118515 ns/op
BenchmarkProductInfo/p666-4                10000            111723 ns/op
BenchmarkProductInfoParalle-4              10000            118641 ns/op
  • 執(zhí)行如下測(cè)試命令
go test -bench=oductInfo

過(guò)濾測(cè)試函數(shù)名中包含oductInfo的測(cè)試用例,結(jié)果:

BenchmarkProductInfo/pn3-4                 10000            107338 ns/op
BenchmarkProductInfo/p7-4                  10000            109848 ns/op
BenchmarkProductInfo/p666-4                10000            109344 ns/op
BenchmarkProductInfoParalle-4              10000            114351 ns/op
  • 執(zhí)行如下測(cè)試命令
 go test -bench=ProductInfo/p7

過(guò)濾測(cè)試函數(shù)名中包含ProductInfo且子測(cè)試名稱包含p7的測(cè)試用例,同時(shí)我們可以注意到并行的測(cè)試也執(zhí)行了。結(jié)果:

BenchmarkProductInfo/p7-4                  10000            109045 ns/op
BenchmarkProductInfoParalle-4              10000            117569 ns/op

Test測(cè)試代碼


func TestCheckProductLockt(t *testing.T) {
    testCases := []string{"a1", "a2", "a3"}
    for _, productID := range testCases {

        t.Log(productID)
        t.Run(productID, func(t *testing.T) {
            _, ret := mgoDB.ecnGetProductInfoOfProductId(productID)
            if ret != Success {
                t.Fatalf("faield")
            }

        })

    }
}

func TestCheckProductLocktParalle(t *testing.T) {
    testCases := []string{"a1", "a2", "a3"}
    for _, tproductID := range testCases {
        productID := tproductID
        t.Log(productID)
        t.Run(productID, func(t *testing.T) {
            t.Parallel()
            _, ret := mgoDB.ecnGetProductInfoOfProductId(productID)
            if ret != Success {
                t.Fatalf("faield")
            }

        })

    }
}

func TestUserIDMatchRole(t *testing.T) {
    reqData := []struct {
        ProductID string
        UserID    string
        RoleType  string
    }{
        {"pn2", "48176d26e860975e96518b80a3520407", "HR"},
        {"pn2", "48176d26e860975e96518b80a3520407", "CEO"},
        {"pn2", "48176d26e860975e96518b80a3520407", "CTO"},
    }

    for _, data := range reqData {
        //
        t.Log(data)
        t.Run(fmt.Sprint("%s %s", data.ProductID, data.RoleType), func(t *testing.T) {
            if ret := checkUserMatchProductRole(data.ProductID, data.UserID, data.RoleType); ret != Success {
                t.Error("not match")
            }
        })

    }
}

func TestUserIDMatchRoleParall(t *testing.T) {
    reqData := []struct {
        ProductID string
        UserID    string
        RoleType  string
    }{
        {"pn2", "48176d26e860975e96518b80a3520407", "HR"},
        {"pn2", "48176d26e860975e96518b80a3520407", "CEO"},
        {"pn2", "48176d26e860975e96518b80a3520407", "CTO"},
    }

    for _, tdata := range reqData {
        //
        data := tdata //重要
        t.Log(data)
        t.Run(fmt.Sprint("%s %s", data.ProductID, data.RoleType), func(t *testing.T) {
            t.Parallel()
            if ret := checkUserMatchProductRole(data.ProductID, data.UserID, data.RoleType); ret != Success {
                t.Error("not match")
            }
        })

    }
}

  • 執(zhí)行如下測(cè)試命令
go test -bench="."

結(jié)果

--- FAIL: TestCheckProductLockt (0.00s)
        ecn_test.go:626: a1
    --- FAIL: TestCheckProductLockt/a1 (0.00s)
        ecn_test.go:630: faield
        ecn_test.go:626: a2
    --- FAIL: TestCheckProductLockt/a2 (0.00s)
        ecn_test.go:630: faield
        ecn_test.go:626: a3
    --- FAIL: TestCheckProductLockt/a3 (0.00s)
        ecn_test.go:630: faield
--- FAIL: TestCheckProductLocktParalle (0.00s)
        ecn_test.go:642: a1
        ecn_test.go:642: a2
        ecn_test.go:642: a3
    --- FAIL: TestCheckProductLocktParalle/a1 (0.00s)
        ecn_test.go:647: faield
    --- FAIL: TestCheckProductLocktParalle/a2 (0.00s)
        ecn_test.go:647: faield
    --- FAIL: TestCheckProductLocktParalle/a3 (0.00s)
        ecn_test.go:647: faield
--- FAIL: TestUserIDMatchRole (0.00s)
        ecn_test.go:668: {pn2 48176d26e860975e96518b80a3520407 HR}
    --- FAIL: TestUserIDMatchRole/%s_%spn2HR (0.00s)
        ecn_test.go:671: not match
        ecn_test.go:668: {pn2 48176d26e860975e96518b80a3520407 CEO}
    --- FAIL: TestUserIDMatchRole/%s_%spn2CEO (0.00s)
        ecn_test.go:671: not match
        ecn_test.go:668: {pn2 48176d26e860975e96518b80a3520407 CTO}
    --- FAIL: TestUserIDMatchRole/%s_%spn2CTO (0.00s)
        ecn_test.go:671: not match
--- FAIL: TestUserIDMatchRoleParall (0.00s)
        ecn_test.go:692: {pn2 48176d26e860975e96518b80a3520407 HR}
        ecn_test.go:692: {pn2 48176d26e860975e96518b80a3520407 CEO}
        ecn_test.go:692: {pn2 48176d26e860975e96518b80a3520407 CTO}
    --- FAIL: TestUserIDMatchRoleParall/%s_%spn2HR (0.00s)
        ecn_test.go:696: not match
    --- FAIL: TestUserIDMatchRoleParall/%s_%spn2CTO (0.00s)
        ecn_test.go:696: not match
    --- FAIL: TestUserIDMatchRoleParall/%s_%spn2CEO (0.00s)
        ecn_test.go:696: not match

在測(cè)試代碼中我們添加了t.log的打印,通過(guò)打印對(duì)比并發(fā)版本和非并發(fā)版本的輸出,可以看到非并發(fā)版本的測(cè)試的確時(shí)順序執(zhí)行的,而并發(fā)版本的測(cè)試是并發(fā)執(zhí)行的。

參考網(wǎng)址

Using Subtests and Sub-benchmarks
解讀2016之Golang篇:極速提升,逐步超越

最后編輯于
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
平臺(tái)聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡(jiǎn)書(shū)系信息發(fā)布平臺(tái),僅提供信息存儲(chǔ)服務(wù)。

推薦閱讀更多精彩內(nèi)容