Startup
單元測試的核心價值在于兩點：
- 更加精確地定義某段代碼的作用，從而使代碼的耦合性更低
- 避免程序員寫出不符合預期的代碼，以及因新增功能而帶來的Regression Bug
隨著Test-Driven方法論的流行，測試類庫對于高級語言來說變得不可或缺。Python生態圈中的unit testing framework相當多，不同于Java幾乎只有JUnit與TestNG二選一，Python unittest框架中較為活躍并也有較多使用者的framework就有unittest、unittest2、nose、nose2與py.test等。不計其他較小眾的工具，光是要搞懂這些工具并從中挑選一個合適的出來使用就讓人頭大了。本文因此總結了這些類庫在實戰中的作用，以便讀者在選擇時方便比對參考。
這里是介紹Python測試的官方文檔：
本文在該文檔的基礎上刪減了入門部分，增加了深入講解和實戰案例。
類庫
Unittest
Unittest的標準文檔在這里：
Unittest是Python標準庫的一部分。它是目前最流行的固件測試框架XUnit在Python中的實現，如果你接觸過Junit，nUnit，或者CppUnit，你會非常熟悉它的API。
Unittest框架的單元測試類用例通過繼承unittest.TestCase來實現，看起來像是這樣：
import unittest
def fun(x):
return x + 1
class MyTest(unittest.TestCase):
def test(self):
self.assertEqual(fun(3), 4)
Unittest一共包含4個概念：
- Test Fixture，就是Setup()和TearDown()
- Test Case，一個Test Case就是一個測試用例，他們都是unittest.TestCase類的子類的方法
- Test Suite，Test Suite是一個測試用例集合，基本上你用不到它，用unittest.main()或者其它發現機制來運行所有的測試用例就對了。 :)
- Test runner，這是單元測試結果的呈現接口，你可以定制自己喜歡的呈現方式，比如GUI界面，基本上你也用不到它。
一些實戰中需要用到的技巧：
-
發現機制
python -m unittest discover -s Project/Test/Directory -p "*test*" # 等同于 python -m unittest discover -s Project/Test/Directory -
用Assert，不要用FailUnless（它們已經被廢棄）
Deprecated.png -
常用的Assert
NormalAssert.png -
特殊的Assert
SpecificAssert.pngFor example:
assertAlmostEqual(1.1, 3.3-2.15, places=1)SpecificEqual.png -
AssertException
AssertException.png-
assertRaises
-
assertRaises(exception, callable, *args, **kwds)def raisesIOError(*args, **kwds): raise IOError("TestIOError") class FixtureTest(unittest.TestCase): def test1(self): self.asertRaises(IOError, raisesIOError) if __name__ == '__main__': unittest.main() -
assertRaises(exception)# If only the exception argument is given, # returns a context manager so that the code # under test can be written inline rather # than as a function with self.assertRaises(SomeException): do_something() # The context manager will store the caught # exception object in its exception attribute. # This can be useful if the intention is to # perform additional checks on the exception raised with self.assertRaises(SomeException) as cm: do_something() the_exception = cm.exception self.assertEqual(the_exception.error_code, 3)
-
-
assertRaisesRegexp
self.assertRaisesRegexp(ValueError, "invalid literal for.*XYZ'$", int, 'XYZ') # or with self.assertRaisesRegexp(ValueError, 'literal'): int('XYZ')
-
-
Skip，出于各種原因，你可能需要暫時跳過一些測試用例（而不是刪除它們）
class MyTestCase(unittest.TestCase): @unittest.skip("demonstrating skipping") def test_nothing(self): self.fail("shouldn't happen") @unittest.skipIf(mylib.__version__ < (1, 3), "not supported in this library version") def test_format(self): # Tests that work for only a certain version of the library. pass @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows") def test_windows_support(self): # windows specific testing code pass -
Class level fixtures
import unittest class Test(unittest.TestCase): @classmethod def setUpClass(cls): cls._connection = createExpensiveConnectionObject() @classmethod def tearDownClass(cls): cls._connection.destroy() -
Module level fixtures
# These should be implemented as functions: def setUpModule(): createConnection() def tearDownModule(): closeConnection()
Mock
Mock類庫是一個專門用于在unittest過程中制作（偽造）和修改（篡改）測試對象的類庫，制作和修改的目的是避免這些對象在單元測試過程中依賴外部資源（網絡資源，數據庫連接，其它服務以及耗時過長等）。Mock是一個如此重要的類庫，如果沒有它，Unittest框架從功能上來說就是不完整的。所以不能理解為何它沒有出現在Python2的標準庫里，不過我們可以很高興地看到在Python3中mock已經是unittest框架的一部分。
-
猴子補丁，Monkey-patching is the technique of swapping functions or methods with others in order to change a module, library or class behavior.
>>> class Class(): ... def add(self, x, y): ... return x + y ... >>> inst = Class() >>> def not_exactly_add(self, x, y): ... return x * y ... >>> Class.add = not_exactly_add >>> inst.add(3, 3) 9 -
Mock對象
-
return_value: 設置Mock方法的返回值
>>> from mock import Mock >>> class ProductionClass(): pass ... >>> real = ProductionClass() >>> real.method = Mock(return_value=3) >>> real.method(3, 4, 5, key='value') 3 >>> real.method.assert_called_with(3, 4, 5, key='value') >>> real.method.assert_called_with(3, 4, key='value') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.7/site-packages/mock/mock.py", line 937, in assert_called_with six.raise_from(AssertionError(_error_message(cause)), cause) File "/usr/local/lib/python2.7/site-packages/six.py", line 718, in raise_from raise value AssertionError: Expected call: mock(3, 4, key='value') Actual call: mock(3, 4, 5, key='value') -
side_effect:
-
調用Mock方法時，拋出異常
>>> mock = Mock(side_effect=KeyError('foo')) >>> mock() Traceback (most recent call last): ... KeyError: 'foo' >>> mock = Mock() >>> mock.return_value = 42 >>> mock() 42 -
調用Mock方法時，根據參數得到不同的返回值
>>> values = {'a': 1, 'b': 2, 'c': 3} >>> def side_effect(arg): ... return values[arg] ... >>> mock.side_effect = side_effect >>> mock('a'), mock('b'), mock('c') (1, 2, 3) -
模擬生成器
>>> mock.side_effect = [5, 4, 3, 2, 1] >>> mock() 5 >>> mock(), mock(), mock(), mock() (4, 3, 2, 1) >>> mock() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.7/site-packages/mock/mock.py", line 1062, in __call__ return _mock_self._mock_call(*args, **kwargs) File "/usr/local/lib/python2.7/site-packages/mock/mock.py", line 1121, in _mock_call result = next(effect) File "/usr/local/lib/python2.7/site-packages/mock/mock.py", line 109, in next return _next(obj) StopIteration
-
-
patch：在函數（function）或者環境管理協議（with）中模擬對象，離開函數或者環境管理器范圍后模擬行為結束。
-
在函數中
from mock import patch class AClass(object): pass class BClass(object): pass print id(AClass), id(BClass) @patch('__main__.AClass') @patch('__main__.BClass') def test(x, MockClass2, MockClass1): print id(AClass), id(BClass) print id(MockClass1), id(MockClass2) print AClass print AClass() assert MockClass1.called print x test(42) # output: """ 140254580491744 140254580492688 4525648336 4517777552 4525648336 4517777552 <MagicMock name='AClass' id='4525648336'> <MagicMock name='AClass()' id='4525719824'> 42 """ -
在環境管理協議中
>>> class Class(object): ... def method(self): ... pass ... >>> with patch('__main__.Class') as MockClass: ... instance = MockClass.return_value ... instance.method.return_value = 'foo' ... assert Class() is instance ... assert Class().method() == 'foo' ... -
Spec Set的寫法，你應該用不到
>>> Original = Class >>> patcher = patch('__main__.Class', spec=True) >>> MockClass = patcher.start() >>> instance = MockClass() >>> assert isinstance(instance, Original) >>> patcher.stop()
-
-
patch.object: 在函數或者環境管理協議中模擬對象，但只mock其中一個attribute
from mock import patch class AClass(): def method(self, *arg): return 42 with patch.object(AClass, 'method') as mock_method: mock_method.return_value = "Fake" real = AClass() print real.method(1, 2, 3) # Fake mock_method.assert_called_once_with(1, 2, 3) print real.method(1, 2, 3) # 42 -
patch.dict: patch.dict can be used to add members to a dictionary, or simply let a test change a dictionary, and ensure the dictionary is restored when the test ends.
patch.dict(in_dict, values=(), clear=False, **kwargs)
If clear is True then the dictionary will be cleared before the new values are set.
-
>>> from mock import patch
>>> foo = {}
>>> with patch.dict(foo, {'newkey': 'newvalue'}):
... assert foo == {'newkey': 'newvalue'}
...
>>> assert foo == {}
>>> import os
>>> with patch.dict('os.environ', {'newkey': 'newvalue'}):
... print os.environ['newkey']
...
newvalue
>>> assert 'newkey' not in os.environ
- patch.multiple: Perform multiple patches in a single call.
>>> thing = object()
>>> other = object()
>>> @patch.multiple('__main__', thing=DEFAULT, other=DEFAULT)
... def test_function(thing, other):
... assert isinstance(thing, MagicMock)
... assert isinstance(other, MagicMock)
...
>>> test_function()
>>> @patch('sys.exit')
... @patch.multiple('__main__', thing=DEFAULT, other=DEFAULT)
... def test_function(mock_exit, other, thing):
... assert 'other' in repr(other)
... assert 'thing' in repr(thing)
... assert 'exit' in repr(mock_exit)
...
>>> test_function()
-
MagicMock: MagicMock是Mock的子類。MagicMock is a subclass of Mock with default implementations of most of the magic methods. You can use MagicMock without having to configure the magic methods yourself.
-
MagicMock的功能完全cover Mock，比如：
from mock import MagicMock thing = ProductionClass() thing.method = MagicMock(return_value=3) thing.method(3, 4, 5, key='value') # return 3 thing.method.assert_called_with(3, 4, 5, key='value') -
MagicMock相對于Mock的優勢：
>>> from mock import MagicMock >>> mock = MagicMock() >>> mock.__str__.return_value = 'foobarbaz' >>> str(mock) 'foobarbaz' >>> mock.__str__.assert_called_once_with()原來需要：
>>> from mock import Mock >>> mock = Mock() >>> mock.__str__ = Mock(return_value = 'wheeeeee') >>> str(mock) 'wheeeeee'
-
-
create_autospec: 使Mock對象擁有和原對象相同的字段和方法，對于方法對象，則擁有相同的簽名
>>> from mock import create_autospec >>> def function(a, b, c): ... pass ... >>> mock_function = create_autospec(function, return_value='fishy') >>> mock_function(1, 2, 3) 'fishy' >>> mock_function.assert_called_once_with(1, 2, 3) >>> mock_function('wrong arguments') Traceback (most recent call last): ... TypeError: <lambda>() takes exactly 3 arguments (1 given) >>> from mock import create_autospec >>> >>> mockStr = create_autospec(str) >>> print mockStr.__add__("d", "e") <MagicMock name='mock.__add__()' id='4537473552'>
Unittest2
Unittest2致力于將Python2.7及以后版本上unittest框架的新特性移植（backport）到Python2.4~Python2.6平臺中。
Backport是將一個軟件補丁應用到比該補丁所對應的版本更老的版本的行為。
你知道這些就可以了，基本上你不會用到它。
unittest2py3k is the Python 3 compatible version of unittest2
py.test
pytest是另一種固件測試框架，它的API設計非常簡潔優雅，完全脫離了XUnit的窠臼（unittest是XUnit在Python中的實現）。但這也正是它的缺點，unittest是標準庫的一部分，用者甚眾，與之大異難免曲高和寡。
py.test功能完備，并且可擴展，但是它語法很簡單。創建一個測試組件和寫一個帶有諸多函數的模塊一樣容易，來看一個例子
# content of test_sample.py
def func(x):
return x + 1
def test_answer():
assert func(3) == 5
運行一下：
$ py.test
============= test session starts =============
platform darwin -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /Users/wuwenxiang/Documents/workspace/testPyDev, inifile:
collected 1 items
some_test.py F
================== FAILURES ===================
_________________ test_answer _________________
def test_answer():
> assert func(3) == 5
E assert 4 == 5
E + where 4 = func(3)
some_test.py:6: AssertionError
========== 1 failed in 0.01 seconds ===========
官方文檔中入門的例子在這里，pytest也給出了unittest Style的兼容寫法示例，然并X，看完之后你會發現：圈子不同，不必強融，這句話還真TM有道理。
py.test的setup/teardown語法與unittest的兼容性不高，實現方式也不直觀。
我們來看一下setup/teardown的例子：
# some_test.py
import pytest
@pytest.fixture(scope='function')
def setup_function(request):
def teardown_function():
print("teardown_function called.")
request.addfinalizer(teardown_function)
print('setup_function called.')
@pytest.fixture(scope='module')
def setup_module(request):
def teardown_module():
print("teardown_module called.")
request.addfinalizer(teardown_module)
print('setup_module called.')
def test_1(setup_function):
print('Test_1 called.')
def test_2(setup_module):
print('Test_2 called.')
def test_3(setup_module):
print('Test_3 called.')
pytest創建固件測試環境（fixture）的方式如上例所示，通過顯式指定scope=''參數來選擇需要使用的pytest.fixture裝飾器。即一個fixture函數的類型從你定義它的時候就確定了，這與使用@nose.with_setup()不同。對于scope='function'的fixture函數，它就是會在測試用例的前后分別調用setup/teardown。測試用例的參數如def test_1(setup_function)只負責引用具體的對象，它并不關心對方的作用域是函數級的還是模塊級的。
有效的 scope 參數限于：function, module, class, session，默認為function。
運行上例：$ py.test some_test.py -s。-s用于顯示print()函數
執行效果：
$ py.test -s some_test.py
============= test session starts =============
platform darwin -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /Users/wuwenxiang/Documents/workspace/testPyDev, inifile:
collected 3 items
some_test.py setup_function called.
Test_1 called.
.teardown_function called.
setup_module called.
Test_2 called.
.Test_3 called.
.teardown_module called.
========== 3 passed in 0.01 seconds ===========
這里需要注意的地方是：setup_module被調用的位置。
Nose
nose廣為流傳，它主要用于配置和運行各種框架下的測試用例，有更簡潔友好的測試用例發現功能。nose的自動發現策略是會遍歷文件夾，搜索特征文件（默認是搜索文件名中帶test的文件）
$ nosetests
F.
======================================================================
FAIL: some_test.test_answer
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/Users/wuwenxiang/Documents/workspace/testPyDev/some_test.py", line 6, in test_answer
assert func(3) == 5
AssertionError
----------------------------------------------------------------------
Ran 2 tests in 0.004s
FAILED (failures=1)
很可惜，官網說：Nose has been in maintenance mode for the past several years and will likely cease without a new person/team to take over maintainership. New projects should consider using Nose2, py.test, or just plain unittest/unittest2.
Nose2是Nose的原班人馬開發。nose2 is being developed by the same people who maintain nose.
Nose2是基于unittest2 plugins分支開發的，但并不支持python2.6之前的版本。Nose2致力于做更好的Nose，它的Plugin API并不兼容之前Nose的API，所以如果你migration from Nose，必須重寫這些plugin。nose2 implements a new plugin API based on the work done by Michael Foord in unittest2’s plugins branch. This API is greatly superior to the one in nose, especially in how it allows plugins to interact with each other. But it is different enough from the API in nose that supporting nose plugins in nose2 will not be practical: plugins must be rewritten to work with nose2.
然而……
Nose2的更新……也很有限……
其作者Jason Pellerin先生坦言他目前(2014年)并沒有多余的時間進行personal projects的開發，每周對nose與nose2的實際開發時間大概只有30分鐘，在這種情況下，nose與nose2都將很難再有大的改版與修正。
Green
不同與nose/nose2，green是單純為了強化unittest中test runner功能而出現的工具。green所提供的只有一個功能強大、使用方便、測試報告美觀的test runner。如果你的項目中的測試都是以傳統unittest module撰寫而成的話，green會是一個很好的test runner選擇。
使用green執行測試：
pip install green
cd path/to/project
green
Doctest
Doctest的標準文檔在這里：
Doctest看起來像是在交互式運行環境中的輸出，事實上也確實如此 :)
def square(x):
"""Squares x.
>>> square(2)
4
>>> square(-2)
4
"""
return x * x
if __name__ == '__main__':
import doctest
doctest.testmod()
Doctest的作用是作為函數/類/模塊等單元的解釋和表述性文檔。所以它們有如下特點：
- 只有期望對外公開的單元會提供doctest
- 這些doctest通常不是很細致
編寫doctest測試基本不需要學習新技能點，在交互式環境里運行一下，然后把輸出結果檢查一下貼過來就可以了。
doctest還有一些高級用法，但基本上用不到，用到的時候再去查標準文檔好了。 :)
Mox
Mox是Java EasyMock框架在Python中的實現。它一個過時的，很像mock的類庫。從現在開始，你應該放棄學習Mox，在任何情況下都用Mock就對了。
參考 Mox的官方文檔：
Mox is a mock object framework for Python based on the Java mock object framework EasyMock.
New uses of this library are discouraged.
People are encouraged to use https://pypi.python.org/pypi/mock instead which matches the unittest.mock library available in Python 3.
Mox3 是一個非官方的類庫，是mox的Python3兼容版本
Mox3 is an unofficial port of the Google mox framework (http://code.google.com/p/pymox/) to Python 3.
It was meant to be as compatible with mox as possible, but small enhancements have been made.
The library was tested on Python version 3.2, 2.7 and 2.6.
Use at your own risk ;)
其它
-
tox(官方文檔): 一個自動化測試框架
- checking your package installs correctly with different Python versions and interpreters
- running your tests in each of the environments, configuring your test tool of choice
- acting as a frontend to Continuous Integration servers, greatly reducing boilerplate and merging CI and shell-based testing.
Basic example:
# content of: tox.ini , put in same dir as setup.py [tox] envlist = py26,py27 [testenv] deps=pytest # install pytest in the venvs commands=py.test # or 'nosetests' or ...You can also try generating a tox.ini file automatically, by running tox-quickstart and then answering a few simple questions.
To sdist-package, install and test your project against Python2.6 and Python2.7, just type:tox testr(官方文檔): 是一個test runner。
-
Django的Unittest(官方文檔)
官方文檔推薦用Unittest：The preferred way to write tests in Django is using the unittest module built in to the Python standard library.
-
django.test.TestCase繼承了unittest.TestCase
Here is an example which subclasses from django.test.TestCase, which is a subclass of unittest.TestCase that runs each test inside a transaction to provide isolation:
from django.test import TestCase from myapp.models import Animal class AnimalTestCase(TestCase): def setUp(self): Animal.objects.create(name="lion", sound="roar") Animal.objects.create(name="cat", sound="meow") def test_animals_can_speak(self): """Animals that can speak are correctly identified""" lion = Animal.objects.get(name="lion") cat = Animal.objects.get(name="cat") self.assertEqual(lion.speak(), 'The lion says "roar"') self.assertEqual(cat.speak(), 'The cat says "meow"')
-
Flask的Unittest
-
官方文檔中介紹：Flask provides a way to test your application by exposing the Werkzeug test Client and handling the context locals for you. You can then use that with your favourite testing solution. In this documentation we will use the unittest package that comes pre-installed with Python.
app.test_client()
app = flask.Flask(__name__) with app.test_client() as c: rv = c.get('/?tequila=42') assert request.args['tequila'] == '42'Unittest with Flask
class FlaskrTestCase(unittest.TestCase): def setUp(self): self.db_fd, flaskr.app.config['DATABASE'] = tempfile.mkstemp() self.app = flaskr.app.test_client() flaskr.init_db() def tearDown(self): os.close(self.db_fd) os.unlink(flaskr.app.config['DATABASE']) def test_empty_db(self): rv = self.app.get('/') assert b'No entries here so far' in rv.data -
@app.route("/ajax/") def some_json(): return jsonify(success=True) class TestViews(TestCase): def test_some_json(self): response = self.client.get("/ajax/") self.assertEquals(response.json, dict(success=True))
-
建議和總結
-
在項目中盡量不要mix多種功能類似的框架。
你可以選unittest + green，或者nose/nose2(依使用Python版本和項目的歷史遺留而定) ，或者pytest，但是盡量不要混合使用。
-
關于Unittest
- 如果沒有特別的原因，新項目應該用unittest。
- Unittest中要用Assert，不要用FailUnless
- Django和Flask中都應該用unittest框架，他們也都提供了一個unittest.TestCase的子類以便于做與WebServer想關的測試
-
關于Mock
- 如果要Mock一個對象，用MagicMock
- 如果要在函數或者with-statment中Mock一個對象，用patch
- 如果要在函數或者with-statement中Mock一個對象的屬性，用patch.object
- 如果要在函數或者with-statement中Mock一個字典（增加或重建一個鍵值對），用patch.dict
- 如果要在函數或者with-statement中一次Patch多個Mock對象，用patch.multiple
- 如果希望Mock對象擁有和原對象相同的字段和方法（對于方法對象，則擁有相同的簽名），用create_autospec。
