Add a gate-specific tox env for functional tests

The functional gate jobs (*-neutron-dsvm-functional) need special
configuration (e.g. OS_SUDO_TESTING, OS_ROOTWRAP_CMD) to run tests.
This patch adds a new gate-specific tox env (tox -e dsvm-functional)
that sets the appropriate values and allows easy duplication of the
results of the upstream job on a host that has successfully run
devstack with neutron enabled.

A new configuration option, OS_FAIL_ON_MISSING_DEPS, is also set by
the dsvm-functional tox env to ensure that dependency failures will
result in failures rather than skips when run in the gate.

Change-Id: Idbfef1ba72ed129a3021509c7969e2685b8a6d09
Partial-bug: #1336172

TESTING.rst

@@ -4,16 +4,16 @@ Testing Neutron
 Overview
 --------
 
-The unit tests are meant to cover as much code as possible and should
-be executed without the service running. They are designed to test
-the various pieces of the neutron tree to make sure any new changes
-don't break existing functionality.
+The unit tests (neutron/test/unit/) are meant to cover as much code as
+possible and should be executed without the service running. They are
+designed to test the various pieces of the neutron tree to make sure
+any new changes don't break existing functionality.
 
-The functional tests are intended to validate actual system
-interaction.  Mocks should be used sparingly, if at all.  Care
-should be taken to ensure that existing system resources are not
-modified and that resources created in tests are properly cleaned
-up.
+The functional tests (neutron/tests/functional/) are intended to
+validate actual system interaction.  Mocks should be used sparingly,
+if at all.  Care should be taken to ensure that existing system
+resources are not modified and that resources created in tests are
+properly cleaned up.
 
 Development process
 -------------------
@@ -100,6 +100,16 @@ Neutron source code::
 
     tox
 
+To run functional tests that do not require sudo privileges or
+specific-system dependencies::
+
+    tox -e functional
+
+To run all the functional tests in an environment that has been configured
+by devstack to support sudo and system-specific dependencies::
+
+    tox -e dsvm-functional
+
 For more information on the standard Tox-based test infrastructure used by
 OpenStack and how to do some common test/debugging procedures with Testr,
 see this wiki page:

neutron/tests/functional/agent/linux/base.py

@@ -29,7 +29,7 @@ def check_command(self, cmd, error_text, skip_msg, root_helper=None):
         try:
             utils.execute(cmd, root_helper=root_helper)
         except RuntimeError as e:
-            if error_text in str(e):
+            if error_text in str(e) and not self.fail_on_missing_deps:
                 self.skipTest(skip_msg)
             raise
 

neutron/tests/functional/agent/linux/test_ovsdb_monitor.py

@@ -47,8 +47,6 @@ def setUp(self):
 
     def _check_test_requirements(self):
         self.check_sudo_enabled()
-        self.check_command(['which', 'ovsdb-client'],
-                           'Exit code: 1', 'ovsdb-client is not installed')
         self.check_command(['ovsdb-client', 'list-dbs'],
                            'Exit code: 1',
                            'password-less sudo not granted for ovsdb-client',

neutron/tests/functional/base.py

@@ -46,8 +46,12 @@ class BaseSudoTestCase(base.BaseTestCase):
 
     def setUp(self):
         super(BaseSudoTestCase, self).setUp()
-        self.root_helper = os.environ.get('OS_ROOTWRAP_CMD', SUDO_CMD)
+        env = os.environ
+        self.sudo_enabled = env.get('OS_SUDO_TESTING') in base.TRUE_STRING
+        self.root_helper = env.get('OS_ROOTWRAP_CMD', SUDO_CMD)
+        self.fail_on_missing_deps = (
+            env.get('OS_FAIL_ON_MISSING_DEPS') in base.TRUE_STRING)
 
     def check_sudo_enabled(self):
-        if os.environ.get('OS_SUDO_TESTING') not in base.TRUE_STRING:
+        if not self.sudo_enabled:
             self.skipTest('testing with sudo is not enabled')

tox.ini

@@ -18,6 +18,14 @@ setenv = OS_TEST_PATH=./neutron/tests/functional
 commands =
   python setup.py testr --slowest --testr-args='{posargs}'
 
+[testenv:dsvm-functional]
+setenv = OS_TEST_PATH=./neutron/tests/functional
+         OS_SUDO_TESTING=1
+         OS_ROOTWRAP_CMD=sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
+         OS_FAIL_ON_MISSING_DEPS=1
+commands =
+  python setup.py testr --slowest --testr-args='{posargs}'
+
 [tox:jenkins]
 sitepackages = True
 downloadcache = ~/cache/pip