Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
212 changes: 149 additions & 63 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,54 +1,79 @@
testresources: extensions to python unittest to allow declarative use
of resources by test cases.
testresources
=============

Copyright (C) 2005-2013 Robert Collins <robertc@robertcollins.net>
testresources extends ``unittest`` with a clean and simple API to provide test
optimisation where expensive common resources are needed for test cases - for
example sample working trees for VCS systems, reference databases for
enterprise applications, or web servers ... let your imagination run wild.

Licensed under either the Apache License, Version 2.0 or the BSD 3-clause
license at the users choice. A copy of both licenses are available in the
project source as Apache-2.0 and BSD. You may not use this file except in
compliance with one of these two licences.
Usage
-----

Unless required by applicable law or agreed to in writing, software
distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
license you chose for the specific language governing permissions and
limitations under that license.
Here is a minimal example demonstrating how to use testresources in your
project. It's not very useful - temporary directories are *not* the kind of
resource that testresources are most useful for and you would be better off
using something like fixtures for this - but it does demonstrate some of the
key concepts and classes, which we will discuss in more detail below. Firstly,
we have our "resource manager":

See the COPYING file for full details on the licensing of Testresources.
.. code-block:: python

import shutil
import tempfile
import testresources

Testresources
+++++++++++++

testresources extends unittest with a clean and simple api to provide test
optimisation where expensive common resources are needed for test cases - for
example sample working trees for VCS systems, reference databases for
enterprise applications, or web servers ... let imagination run wild.
class TemporaryDirectoryResource(testresources.TestResourceManager):

def make(self, dependency_resources):
return tempfile.mkdtemp()

def clean(self, resource):
shutil.rmtree(resource)

def isDirty(self, resource):
# Assume the directory is always modified after use.
return True

Dependencies to build/selftest
==============================
With the resource manager in place, we can then declare the resource in a test
and access it via the assigned attribute:

* Python 3.9+
* docutils
* testtools (http://pypi.python.org/pypi/testtools/)
* fixtures (http://pypi.python.org/pypi/fixtures)
.. code-block:: python

Dependencies to use testresources
=================================
import os
import unittest

* Python 3.9+

For older versions of Python, testresources <= 1.0.0 supported 2.4, 2.5 and
3.2.
class TestMyCode(unittest.TestCase, testresources.ResourcedTestCase):

How testresources Works
=======================
resources = [('workdir', TemporaryDirectoryResource())]

def test_create_file(self):
# self.workdir is automatically set up before this test runs
# and torn down (or reused) afterwards.
path = os.path.join(self.workdir, 'output.txt')
with open(path, 'w') as f:
f.write('hello')
self.assertTrue(os.path.exists(path))

Finally, we need to add a ``load_tests`` hook to the test module so that we cna
use the ``OptimisingTestSuite``. This ensures our test runner will reorder
tests to minimise the number of times the resource is set up and torn down:

.. code-block:: python

def load_tests(loader, tests, pattern):
return testresources.OptimisingTestSuite(tests)

How it works
------------

The basic idea of testresources is:

* Tests declare the resources they need in a ``resources`` attribute.
* When the test is run, the required resource objects are allocated (either
newly constructed, or reused), and assigned to attributes of the TestCase.
newly constructed, or reused), and assigned to attributes of the
``TestCase``.

testresources distinguishes a 'resource manager' (a subclass of
``TestResourceManager``) which acts as a kind of factory, and a 'resource'
Expand All @@ -64,11 +89,11 @@ when an OptimisingTestSuite is wrapped around a test suite using those
features, the result will be flattened for optimisation and those setup's will
not run at all.

Main Classes
============
Main classes
------------

testresources.ResourcedTestCase
-------------------------------
``testresources.ResourcedTestCase``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

By extending or mixing-in this class, tests can have necessary resources
automatically allocated and disposed or recycled.
Expand Down Expand Up @@ -99,8 +124,8 @@ For example::
def test_log(self):
show_log(self.branch, ...)

testresources.TestResourceManager
---------------------------------
``testresources.TestResourceManager``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

A TestResourceManager is an object that tests can use to create resources. It
can be overridden to manage different types of resources. Normally test code
Expand Down Expand Up @@ -175,20 +200,24 @@ See pydoc testresources.TestResourceManager for details.

.. _sample: doc/example.py

testresources.GenericResource
-----------------------------
``testresources.GenericResource``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Glue to adapt testresources to an existing resource-like class.

testresources.FixtureResource
-----------------------------
.. _fixtureresource:

``testresources.FixtureResource``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Glue to adapt testresources to the simpler fixtures.Fixture API. Long
term testresources is likely to consolidate on that simpler API as the
recommended method of writing resources.
Glue to adapt testresources to the simpler ``fixtures.Fixture`` API. Long term
testresources is likely to consolidate on that simpler API as the recommended
method of writing resources.

testresources.OptimisingTestSuite
---------------------------------
This is discussed in further detail in `testresources vs. fixtures`_.

``testresources.OptimisingTestSuite``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This TestSuite will introspect all the test cases it holds directly and if
they declare needed resources, will run the tests in an order that attempts to
Expand All @@ -208,24 +237,23 @@ OptimisingTestSuite. You could add everything to a single OptimisingTestSuite,
getting global optimisation or you could use several smaller
OptimisingTestSuites.


testresources.TestLoader
------------------------
``testresources.TestLoader``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This is a trivial TestLoader that creates OptimisingTestSuites by default.

unittest.TestResult
-------------------
``unittest.TestResult``
~~~~~~~~~~~~~~~~~~~~~~~

testresources will log activity about resource creation and destruction to the
result object tests are run with. 6 extension methods are looked for:
``startCleanResource``, ``stopCleanResource``, ``startMakeResource``,
``stopMakeResource``, ``startResetResource`` and finally ``stopResetResource``.
``testresources.tests.ResultWithResourceExtensions`` is
an example of a ``TestResult`` with these methods present.
``testresources.tests.ResultWithResourceExtensions`` is an example of a
``TestResult`` with these methods present.

Controlling Resource Reuse
==========================
Controlling resource reuse
--------------------------

When or how do I mark the resource dirtied?

Expand All @@ -241,8 +269,55 @@ see if it is safe to reuse.
Finally, you can arrange for the returned resource to always call back to
``TestResourceManager.dirtied`` on the first operation that mutates it.

testresources vs. fixtures
--------------------------

The `fixtures <https://pypi.org/project/fixtures/>`_ library solves a similar
problem: managing test dependencies that need to be set up and torn down.
However, testresources and fixtures differ in the scope of the test
dependencies they manage.

testresources is designed for resources that are expensive to create and can be
safely shared across multiple tests. The ``OptimisingTestSuite`` reorders
tests at the suite level so that tests sharing the same expensive resource run
consecutively, minimising the total number of setup and teardown cycles. This
makes sense when the cost of constructing the resource is meaningfully large
relative to the cost of running the tests themselves. Examples of areas where
testresources makes sense would be provisioning database backends that are
shared across tests, or loading large, static test assets from disk.

By comparison, fixtures is designed for per-test setup and teardown. A fixture
is created fresh (or at least reset) for each test, and tests interact with it
via ``useFixture()``. fixtures is therefore far better suited for things like
mock patches, temporary directories, fake loggers, environment variables, or
fake HTTP sessions. In all these cases, the overhead of managing the resources
is low enough that recreating them per test is perfectly acceptable.

Finally, there may be cases where you wish to use the framework provided by
``fixtures.Fixture`` but avoid recreating it for every test in a module. To
this end, the ``FixtureResource`` class is what you want. As discussed
`previously <fixtureresource>`, this is a glue class that wraps any
``fixtures.Fixture`` so it can participate in testresources' suite-level
optimisation. If you already have a well-written fixture but want to avoid
recreating it for every test in a module, wrapping it in a ``FixtureResource``
and adding the ``load_tests`` hook is all that is needed. For example:

.. code-block:: python

# Defined once at module scope so that all tests share the same instance.
MY_RESOURCE = testresources.FixtureResource(MyExpensiveFixture())

class MyTest(unittest.TestCase, testresources.ResourcedTestCase):
resources = [('data', MY_RESOURCE)]

def test_something(self):
self.data.some_attribute # provided by MyExpensiveFixture

def load_tests(loader, tests, pattern):
return testresources.OptimisingTestSuite(tests)

FAQ
===
---

* Can I dynamically request resources inside a test method?

Expand All @@ -264,9 +339,20 @@ FAQ
I guess you should arrange for a single instance to be held in an appropriate
module scope, then referenced by the test classes that want to share it.

Releasing
=========
License
-------

1. Add a section to NEWS (after In Development).
2. git tag -s
3. python setup.py sdist bdist_wheel upload -s
Copyright (C) 2005-2013 Robert Collins <robertc@robertcollins.net>

Licensed under either the Apache License, Version 2.0 or the BSD 3-clause
license at the users choice. A copy of both licenses are available in the
project source as Apache-2.0 and BSD. You may not use this file except in
compliance with one of these two licences.

Unless required by applicable law or agreed to in writing, software
distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
license you chose for the specific language governing permissions and
limitations under that license.

See the COPYING file for full details on the licensing of Testresources.
Loading