Package csb :: Package test
[frames] | no frames]

Source Code for Package csb.test

  1  """ 
  2  This is a top level package, hosting the entire CSB test framework. It is divided 
  3  into several major parts: 
  4   
  5      - test cases, located under csb.test.cases 
  6      - test data, in C{/csb/test/data} (not a package) 
  7      - test console, in C{/csb/test/app.py} 
  8   
  9  This module, csb.test, contains all the glue-code functions, classes and  
 10  decorators you would need in order to write tests for CSB.     
 11   
 12      1. Configuration and Tree 
 13       
 14         L{Config<csb.test.Config>} is a common config object shared between CSB 
 15         tests. Each config instance contains properties like: 
 16               
 17              - data: the data folder, automatically discovered and loaded in 
 18                csb.test.Config.DATA at module import time 
 19              - temp: a default temp folder, which test cases can use 
 20           
 21         Each L{Config<csb.test.Config>} provides a convenient way to retrieve 
 22         files from C{/csb/test/data}. Be sure to check out L{Config.getTestFile} 
 23         and L{Config.getPickle}. In case you need a temp file, use 
 24         L{Config.getTempStream} or have a look at L{csb.io.TempFile} and 
 25         L{csb.io.TempFolder}.  
 26           
 27         All test data files should be placed in the C{data} folder. All test 
 28         modules must be placed in the root package: csb.test.cases. There is 
 29         a strict naming convention for test modules: the name of a test module 
 30         should be the same as the name of the CSB API package it tests. For  
 31         example, if you are writing tests for C{csb/bio/io/__init__.py}, the 
 32         test module must be C{csb/test/cases/bio/io/__init__.py}. C{csb.test.cases} 
 33         is the root package of all test modules in CSB. 
 34       
 35      2. Writing Tests 
 36       
 37         Writing a test is easy. All you need is to import csb.test and then 
 38         create your own test cases, derived from L{csb.test.Case}: 
 39          
 40             >>> import csb.test 
 41             >>> @csb.test.unit 
 42                 class TestSomeClass(csb.test.Case): 
 43                     def setUp(self): 
 44                         super(TestSomeClass, self).setUp() 
 45                         # do something with self.config here... 
 46          
 47         In this way your test case instance is automatically equipped with a  
 48         reference to the test config, so your test method can be: 
 49   
 50             >>> @csb.test.unit 
 51                 class TestSomeClass(csb.test.Case): 
 52                     def testSomeMethod(self): 
 53                         myDataFile = self.config.getTestFile('some.file') 
 54                         self.assert... 
 55           
 56         The "unit" decorator marks a test case as a collection of unit tests. 
 57         All possibilities are: L{csb.test.unit}, L{csb.test.functional}, L{csb.test.custom}, 
 58         and L{csb.test.regression}. 
 59                      
 60         Writing custom (a.k.a. "data", "slow", "dynamic") tests is a little bit 
 61         more work. Custom tests must be functions, not classes. Basically a 
 62         custom test is a function, which builds a unittest.TestSuite instance  
 63         and then returns it when called without arguments. 
 64          
 65         Regression tests are usually created in response to reported bugs. Therefore,  
 66         the best practice is to mark each test method with its relevant bug ID: 
 67          
 68             >>> @csb.test.regression 
 69                 class SomeClassRegressions(csb.test.Case) 
 70                     def testSomeFeature(self) 
 71                     \""" 
 72                     @see: [CSB 000XXXX]  
 73                     \""" 
 74                     # regression test body... 
 75              
 76      3. Style Guide: 
 77       
 78         - name test case packages as already described 
 79         - group tests in csb.test.Case-s and name them properly 
 80         - prefix test methods with "test", like "testParser" - very important 
 81         - use camelCase for methods and variables. This applies to all the 
 82           code under csb.test (including test) and does not apply to the rest 
 83           of the library! 
 84         - for functional tests it's okay to define just one test method: runTest 
 85         - for unit tests you should create more specific test names, for example:  
 86           "testParseFile" - a unit test for some method called "parse_file" 
 87         - use csb.test decorators to mark tests as unit, functional, regression, etc. 
 88         - make every test module executable:: 
 89          
 90             if __name__ == '__main__': 
 91                 csb.test.Console()   # Discovers and runs all test cases in the module 
 92       
 93      4. Test Execution 
 94       
 95         Test discovery is handled by C{test builders} and a test runner 
 96         C{app}. Test builders are subclasses of L{AbstractTestBuilder}.   
 97         For every test type (unit, functional, regression, custom) there is a 
 98         corresponding test builder. L{AnyTestBuilder} is a special builder which 
 99         scans for unit, regression and functional tests at the same time. 
100   
101         Test builder classes inherit the following test discovery methods: 
102       
103             - C{loadTests} - load tests from a test namespace. Wildcard 
104               namespaces are handled by C{loadAllTests} 
105             - C{loadAllTests} - load tests from the given namespace, and 
106               from all sub-packages (recursive) 
107             - C{loadFromFile} - load tests from an absolute file name 
108             - C{loadMultipleTests} - calls C{loadTests} for a list of  
109               namespaces and combines all loaded tests in a single suite 
110                
111         Each of those return test suite objects, which can be directly executed 
112         with python's unittest runner. 
113          
114         Much simpler way to execute a test suite is to use our test app  
115         (C{csb/test/app.py}), which is simply an instance of L{csb.test.Console}:: 
116          
117             $ python csb/test/app.py --help 
118          
119         The app has two main arguments:  
120       
121             - test type - tells the app which TestBuilder to use for test dicsovery 
122               ("any" triggers L{AnyTestBuilder}, "unit" - L{UnitTestBuilder}, etc.)  
123             - test namespaces - a list of "dotted" test modules, for example:: 
124       
125                  csb.test.cases.bio.io.*   # io and sub-packages 
126                  csb.test.cases.bio.utils  # only utils 
127                  .                         # current module 
128       
129         In addition to running the app from the command line, you can run it 
130         also programmatically by instantiating L{csb.test.Console}. You can 
131         construct a test console object by passing a list of test namespace(s) 
132         and a test builder class to the Console's constructor. 
133   
134       
135      5. Commit Policies 
136       
137         Follow these guidelines when making changes to the repository: 
138       
139             - B{no bugs in "trunk"}: after fixing a bug or implementing a new 
140               feature, make sure at least the default test set passes by running 
141               the test console without any arguments. This is equivalent to: 
142               app.py -t any "csb.test.cases.*". (If no test case from this set covers 
143               the affected code, create a test case first, as described in the other 
144               policies) 
145       
146             - B{no recurrent issues}: when a bug is found, first write a regression 
147               test with a proper "@see: BugID" tag in the docstring. Run the test 
148               to make sure it fails. After fixing the bug, run the test again before 
149               you commit, as required by the previous policy 
150                
151             - B{test all new features}: there should be a test case for every new feature 
152               we implement. One possible approach is to write a test case first and 
153               make sure it fails; when the new feature is ready, run the test again 
154               to make sure it passes 
155   
156  @warning: for compatibility reasons do NOT import and use the unittest module 
157            directly. Always import unittest from csb.test, which is guaranteed 
158            to be python 2.7+ compatible. The standard unittest under python 2.6 
159            is missing some features, that's why csb.test will take care of 
160            replacing it with unittest2 instead.  
161  """ 
162  from __future__ import print_function 
163   
164  import os 
165  import sys 
166  import imp 
167  import types 
168  import time 
169  import getopt 
170  import tempfile 
171  import traceback 
172   
173  import csb.io 
174  import csb.core 
175   
176  try: 
177      from unittest import skip, skipIf 
178      import unittest 
179  except ImportError: 
180      import unittest2 as unittest 
181   
182  from abc import ABCMeta, abstractproperty 
183 184 185 -class Attributes(object):
186 187 UNIT = '__CSBUnitTest__' 188 CUSTOM = '__CSBCustomTest__' 189 FUNCTIONAL = '__CSBFunctionalTest__' 190 REGRESSION = '__CSBRegressionTest__'
191
192 -class Config(object):
193 """ 194 General CSB Test Config. Config instances contain the following properties: 195 196 - data - path to the CSB Test Data directory. Default is L{Config.DATA} 197 - temp - path to the system's temp directory. Default is L{Config.TEMP} 198 - config - the L{Config} class 199 """ 200 201 DATA = os.path.join(os.path.abspath(os.path.dirname(__file__)), 'data') 202 """ 203 @cvar: path to the default test data directory: <install dir>/csb/test/data 204 """ 205 TEMP = os.path.abspath(tempfile.gettempdir()) 206 """ 207 @cvar: path to the default system's temp directory 208 """ 209
210 - def __init__(self):
211 212 self.__config = Config 213 self.__data = Config.DATA 214 self.__temp = Config.TEMP
215 216 @staticmethod
217 - def setDefaultDataRoot(path):
218 """ 219 Override the default L{Config.DATA} with a new data root directory. 220 221 @param path: full directory path 222 @type path: str 223 """ 224 if not os.path.isdir(path): 225 raise IOError('Path not found: {0}'.format(path)) 226 227 Config.DATA = os.path.abspath(path)
228 229 @property
230 - def data(self):
231 """ 232 Test data directory 233 @rtype: str 234 """ 235 return self.__data
236 237 @property
238 - def temp(self):
239 """ 240 Test temp directory 241 @rtype: str 242 """ 243 return self.__temp
244
245 - def getTestFile(self, fileName, subDir=''):
246 """ 247 Search for C{fileName} in the L{Config.DATA} directory. 248 249 @param fileName: the name of a test file to retrieve 250 @type fileName: str 251 @param subDir: scan a sub-directory of L{Config.DATA} 252 @type subDir: str 253 254 @return: full path to C{fileName} 255 @rtype: str 256 257 @raise IOError: if no such file is found 258 """ 259 file = os.path.join(self.data, subDir, fileName) 260 if not os.path.isfile(file): 261 raise IOError('Test file not found: {0}'.format(file)) 262 return file
263
264 - def getPickle(self, fileName, subDir=''):
265 """ 266 Same as C{self.getTestFile}, but try to unpickle the data in the file. 267 268 @param fileName: the name of a test file to retrieve 269 @type fileName: str 270 @param subDir: scan a sub-directory of L{Config.DATA} 271 @type subDir: str 272 """ 273 file = self.getTestFile(fileName, subDir) 274 return csb.io.Pickle.load(open(file, 'rb'))
275
276 - def getContent(self, fileName, subDir=''):
277 """ 278 Same as C{self.getTestFile}, but also read and return the contents of 279 the file. 280 281 @param fileName: the name of a test file to retrieve 282 @type fileName: str 283 @param subDir: scan a sub-directory of L{Config.DATA} 284 @type subDir: str 285 """ 286 with open(self.getTestFile(fileName, subDir)) as f: 287 return f.read()
288
289 - def getTempStream(self, mode='t'):
290 """ 291 Return a temporary file stream:: 292 293 with self.getTempStream() as tmp: 294 tmp.write(something) 295 tmp.flush() 296 file_name = tmp.name 297 298 @param mode: file open mode (text, binary), default=t 299 @type mode: str 300 @rtype: file stream 301 """ 302 return csb.io.TempFile(mode=mode)
303
304 - def ensureDataConsistency(self):
305 """ 306 Try to deserialize some pickled data files. Call L{Config.updateDataFiles} 307 if the pickles appeared incompatible with the current interpreter. 308 """ 309 try: 310 self.getPickle('1nz9.model1.pickle') 311 except: 312 self.updateDataFiles()
313
314 - def updateDataFiles(self):
315 """ 316 Refresh the pickled structures in csb/test/data. This might be needed when 317 the internal representation of some classes has changed. 318 """ 319 from csb.io import Pickle 320 from csb.bio.io.wwpdb import get 321 from csb.bio.structure import Ensemble, ChemElements 322 323 model1 = get('1nz9', model=1) 324 model2 = get('1nz9', model=2) 325 326 ensemble = Ensemble() 327 ensemble.models.append(model1) 328 ensemble.models.append(model2) 329 Pickle.dump(ensemble, open(os.path.join(self.data, '1nz9.full.pickle'), 'wb')) 330 331 mse = model1.chains['A'].find(164) 332 mse._pdb_name = 'MSE' 333 mse.atoms['SD']._element = ChemElements.Se 334 mse.atoms['SD']._full_name = 'SE ' 335 Pickle.dump(model1, open(os.path.join(self.data, '1nz9.model1.pickle'), 'wb'))
336
337 -class Case(unittest.TestCase):
338 """ 339 Base class, defining a CSB Test Case. Provides a default implementation 340 of C{unittest.TestCase.setUp} which grabs a reference to a L{Config}. 341 """ 342 343 @property
344 - def config(self):
345 """ 346 Test config instance 347 @rtype: L{Config} 348 """ 349 return self.__config
350
351 - def setUp(self):
352 """ 353 Provide a reference to the CSB Test Config in the C{self.config} property. 354 """ 355 self.__config = Config() 356 assert hasattr(self.config, 'data'), 'The CSB Test Config must contain the data directory' 357 assert self.config.data, 'The CSB Test Config must contain the data directory'
358
359 - def reRaise(self, addArgs=()):
360 """ 361 Re-raise the last exception with its full traceback, but modify the 362 argument list with C{addArgs} and the original stack trace. 363 364 @param addArgs: additional arguments to append to the exception 365 @type addArgs: tuple 366 """ 367 klass, ex, _tb = sys.exc_info() 368 ex.args = list(ex.args) + list(addArgs) + [''.join(traceback.format_exc())] 369 370 raise klass(ex.args)
371
372 - def assertFasterThan(self, duration, callable, *args, **kargs):
373 """ 374 Fail if it took more than C{duration} seconds to invoke C{callable}. 375 376 @param duration: maximum amount of seconds allowed 377 @type duration: float 378 """ 379 380 start = time.time() 381 callable(*args, **kargs) 382 execution = time.time() - start 383 384 if execution > duration: 385 self.fail('{0}s is slower than {1}s)'.format(execution, duration))
386
387 - def assertWithinDelta(self, value, expected, delta=1e-1):
388 """ 389 Fail if the difference is larger than delta 390 391 @param value: input value 392 @type value: float 393 394 @param expected: expected value 395 @type expected: float 396 397 @param delta: allowed deviation 398 @type delta: float 399 """ 400 401 if abs(value - expected) > delta: 402 self.fail('|{0} - {1}| > {2})'.format(value, expected, delta))
403 404 @classmethod
405 - def execute(cls):
406 """ 407 Run this test case. 408 """ 409 suite = unittest.TestLoader().loadTestsFromTestCase(cls) 410 runner = unittest.TextTestRunner() 411 412 return runner.run(suite)
413
414 -class InvalidNamespaceError(NameError, ImportError):
415 pass
416
417 -class AbstractTestBuilder(object):
418 """ 419 This is a base class, defining a test loader which exposes the C{loadTests} 420 method. 421 422 Subclasses must override the C{labels} abstract property, which controls 423 what kind of test cases are loaded by the test builder. 424 """ 425 426 __metaclass__ = ABCMeta 427 428 @abstractproperty
429 - def labels(self):
430 pass
431
432 - def loadFromFile(self, file):
433 """ 434 Load L{csb.test.Case}s from a module file. 435 436 @param file: test module file name 437 @type file: str 438 439 @return: a C{unittest.TestSuite} ready for the test runner 440 @rtype: C{unittest.TestSuite} 441 """ 442 mod = self._loadSource(file) 443 suite = unittest.TestLoader().loadTestsFromModule(mod) 444 return unittest.TestSuite(self._filter(suite))
445
446 - def loadTests(self, namespace):
447 """ 448 Load L{csb.test.Case}s from the given CSB C{namespace}. If the namespace 449 ends with a wildcard, tests from sub-packages will be loaded as well. 450 If the namespace is '__main__' or '.', tests are loaded from __main__. 451 452 @param namespace: test module namespace, e.g. 'csb.test.cases.bio' will 453 load tests from '/csb/test/cases/bio/__init__.py' 454 @type namespace: str 455 456 @return: a C{unittest.TestSuite} ready for the test runner 457 @rtype: C{unittest.TestSuite} 458 """ 459 if namespace.strip() == '.*': 460 namespace = '__main__.*' 461 elif namespace.strip() == '.': 462 namespace = '__main__' 463 464 if namespace.endswith('.*'): 465 return self.loadAllTests(namespace[:-2]) 466 else: 467 loader = unittest.TestLoader() 468 tests = loader.loadTestsFromName(namespace) 469 return unittest.TestSuite(self._filter(tests))
470
471 - def loadMultipleTests(self, namespaces):
472 """ 473 Load L{csb.test.Case}s from a list of given CSB C{namespaces}. 474 475 @param namespaces: a list of test module namespaces, e.g. 476 ('csb.test.cases.bio', 'csb.test.cases.bio.io') will 477 load tests from '/csb/test/cases/bio.py' and 478 '/csb/test/cases/bio/io.py' 479 @type namespaces: tuple of str 480 481 @return: a C{unittest.TestSuite} ready for the test runner 482 @rtype: C{unittest.TestSuite} 483 """ 484 if not csb.core.iterable(namespaces): 485 raise TypeError(namespaces) 486 487 return unittest.TestSuite(self.loadTests(n) for n in namespaces)
488
489 - def loadAllTests(self, namespace, extension='.py'):
490 """ 491 Load L{csb.test.Case}s recursively from the given CSB C{namespace} and 492 all of its sub-packages. Same as:: 493 494 builder.loadTests('namespace.*') 495 496 @param namespace: test module namespace, e.g. 'csb.test.cases.bio' will 497 load tests from /csb/test/cases/bio/*' 498 @type namespace: str 499 500 @return: a C{unittest.TestSuite} ready for the test runner 501 @rtype: C{unittest.TestSuite} 502 """ 503 suites = [] 504 505 try: 506 base = __import__(namespace, level=0, fromlist=['']).__file__ 507 except ImportError: 508 raise InvalidNamespaceError('Namespapce {0} is not importable'.format(namespace)) 509 510 if os.path.splitext(os.path.basename(base))[0] != '__init__': 511 suites.append(self.loadTests(namespace)) 512 513 else: 514 515 for entry in os.walk(os.path.dirname(base)): 516 517 for item in entry[2]: 518 file = os.path.join(entry[0], item) 519 if extension and item.endswith(extension): 520 suites.append(self.loadFromFile(file)) 521 522 return unittest.TestSuite(suites)
523
524 - def _loadSource(self, path):
525 """ 526 Import and return the Python module identified by C{path}. 527 528 @note: Module objects behave as singletons. If you import two different 529 modules and give them the same name in imp.load_source(mn), this 530 counts for a redefinition of the module originally named mn, which 531 is basically the same as reload(mn). Therefore, you need to ensure 532 that for every call to imp.load_source(mn, src.py) the mn parameter 533 is a string that uniquely identifies the source file src.py. 534 """ 535 name = os.path.splitext(os.path.abspath(path))[0] 536 name = name.replace('.', '-').rstrip('__init__').strip(os.path.sep) 537 538 return imp.load_source(name, path)
539
540 - def _recurse(self, obj):
541 """ 542 Extract test cases recursively from a test C{obj} container. 543 """ 544 cases = [] 545 if isinstance(obj, unittest.TestSuite) or csb.core.iterable(obj): 546 for item in obj: 547 cases.extend(self._recurse(item)) 548 else: 549 cases.append(obj) 550 return cases
551
552 - def _filter(self, tests):
553 """ 554 Filter a list of objects using C{self.labels}. 555 """ 556 filtered = [] 557 558 for test in self._recurse(tests): 559 for label in self.labels: 560 if hasattr(test, label) and getattr(test, label) is True: 561 filtered.append(test) 562 563 return filtered
564
565 -class AnyTestBuilder(AbstractTestBuilder):
566 """ 567 Build a test suite of cases, marked as either unit, functional or regression 568 tests. For detailed documentation see L{AbstractTestBuilder}. 569 """ 570 @property
571 - def labels(self):
573
574 -class UnitTestBuilder(AbstractTestBuilder):
575 """ 576 Build a test suite of cases, marked as unit tests. 577 For detailed documentation see L{AbstractTestBuilder}. 578 """ 579 @property
580 - def labels(self):
581 return [Attributes.UNIT]
582
583 -class FunctionalTestBuilder(AbstractTestBuilder):
584 """ 585 Build a test suite of cases, marked as functional tests. 586 For detailed documentation see L{AbstractTestBuilder}. 587 """ 588 @property
589 - def labels(self):
590 return [Attributes.FUNCTIONAL]
591
592 -class RegressionTestBuilder(AbstractTestBuilder):
593 """ 594 Build a test suite of cases, marked as regression tests. 595 For detailed documentation see L{AbstractTestBuilder}. 596 """ 597 @property
598 - def labels(self):
599 return [Attributes.REGRESSION]
600
601 -class CustomTestBuilder(AbstractTestBuilder):
602 """ 603 Build a test suite of cases, marked as custom tests. CustomTestBuilder will 604 search for functions, marked with the 'custom' test decorator, which return 605 a dynamically built C{unittest.TestSuite} object when called without 606 parameters. This is convenient when doing data-related tests, e.g. 607 instantiating a single type of a test case many times iteratively, for 608 each entry in a database. 609 610 For detailed documentation see L{AbstractTestBuilder}. 611 """ 612 @property
613 - def labels(self):
614 return [Attributes.CUSTOM]
615
616 - def loadFromFile(self, file):
617 618 mod = self._loadSource(file) 619 suites = self._inspect(mod) 620 621 return unittest.TestSuite(suites)
622
623 - def loadTests(self, namespace):
624 625 if namespace.strip() == '.*': 626 namespace = '__main__.*' 627 elif namespace.strip() == '.': 628 namespace = '__main__' 629 630 if namespace.endswith('.*'): 631 return self.loadAllTests(namespace[:-2]) 632 else: 633 try: 634 mod = __import__(namespace, fromlist=['']) 635 except ImportError: 636 raise InvalidNamespaceError('Namespace {0} is not importable'.format(namespace)) 637 suites = self._inspect(mod) 638 return unittest.TestSuite(suites)
639
640 - def _inspect(self, module):
641 642 objects = map(lambda n: getattr(module, n), dir(module)) 643 return self._filter(objects)
644
645 - def _filter(self, factories):
646 """ 647 Filter a list of objects using C{self.labels}. 648 """ 649 filtered = [] 650 651 for obj in factories: 652 for label in self.labels: 653 if hasattr(obj, label) and getattr(obj, label) is True: 654 suite = obj() 655 if not isinstance(suite, unittest.TestSuite): 656 raise ValueError('Custom test function {0} must return a ' 657 'unittest.TestSuite, not {1}'.format(obj.__name__, type(suite))) 658 filtered.append(suite) 659 660 return filtered
661
662 -def unit(klass):
663 """ 664 A class decorator, used to label unit test cases. 665 666 @param klass: a C{unittest.TestCase} class type 667 @type klass: type 668 """ 669 if not isinstance(klass, type): 670 raise TypeError("Can't apply class decorator on {0}".format(type(klass))) 671 672 setattr(klass, Attributes.UNIT, True) 673 return klass
674
675 -def functional(klass):
676 """ 677 A class decorator, used to label functional test cases. 678 679 @param klass: a C{unittest.TestCase} class type 680 @type klass: type 681 """ 682 if not isinstance(klass, type): 683 raise TypeError("Can't apply class decorator on {0}".format(type(klass))) 684 685 setattr(klass, Attributes.FUNCTIONAL, True) 686 return klass
687
688 -def regression(klass):
689 """ 690 A class decorator, used to label regression test cases. 691 692 @param klass: a C{unittest.TestCase} class type 693 @type klass: type 694 """ 695 if not isinstance(klass, type): 696 raise TypeError("Can't apply class decorator on {0}".format(type(klass))) 697 698 setattr(klass, Attributes.REGRESSION, True) 699 return klass
700
701 -def custom(function):
702 """ 703 A function decorator, used to mark functions which build custom (dynamic) 704 test suites when called. 705 706 @param function: a callable object, which returns a dynamically compiled 707 C{unittest.TestSuite} 708 @type function: callable 709 """ 710 if isinstance(function, type): 711 raise TypeError("Can't apply function decorator on a class") 712 elif not hasattr(function, '__call__'): 713 raise TypeError("Can't apply function decorator on non-callable {0}".format(type(function))) 714 715 setattr(function, Attributes.CUSTOM, True) 716 return function
717
718 -def skip(reason, condition=None):
719 """ 720 Mark a test case or method for skipping. 721 722 @param reason: message 723 @type reason: str 724 @param condition: skip only if the specified condition is True 725 @type condition: bool/expression 726 """ 727 if isinstance(reason, types.FunctionType): 728 raise TypeError('skip: no reason specified') 729 730 if condition is None: 731 return unittest.skip(reason) 732 else: 733 return unittest.skipIf(condition, reason)
734
735 -class Console(object):
736 """ 737 Build and run all tests of the specified namespace and kind. 738 739 @param namespace: a dotted name, which specifies the test module 740 (see L{csb.test.AbstractTestBuilder.loadTests}) 741 @type namespace: str 742 @param builder: test builder to use 743 @type builder: any L{csb.test.AbstractTestBuilder} subclass 744 @param verbosity: verbosity level for C{unittest.TestRunner} 745 @type verbosity: int 746 @param update: if True, refresh all pickles in csb/test/data 747 @type update: bool 748 """ 749 750 BUILDERS = {'unit': UnitTestBuilder, 'functional': FunctionalTestBuilder, 751 'custom': CustomTestBuilder, 'any': AnyTestBuilder, 752 'regression': RegressionTestBuilder} 753 754 USAGE = r""" 755 CSB Test Runner Console. Usage: 756 757 python {0.program} [-u] [-t type] [-v verbosity] namespace(s) 758 759 Options: 760 namespace(s) A list of CSB test dotted namespaces, from which to 761 load tests. '__main__' and '.' are interpreted as the 762 current module. If a namespace ends with an asterisk 763 '.*', all sub-packages will be scanned as well. 764 765 Examples: 766 "csb.test.cases.bio.*" 767 "csb.test.cases.bio.io" "csb.test.cases.bio.utils" 768 "." 769 770 -t type Type of tests to load from each namespace. Possible 771 values are: 772 {0.builders} 773 774 -v verbosity Verbosity level passed to unittest.TextTestRunner. 775 776 -u update-files Force update of the test pickles in csb/test/data. 777 """ 778
779 - def __init__(self, namespace=('__main__',), builder=AnyTestBuilder, verbosity=1, 780 update=False, argv=None):
781 782 if not argv: 783 argv = sys.argv 784 785 self._namespace = None 786 self._builder = None 787 self._verbosity = 1 788 self._update = False 789 self._program = os.path.basename(argv[0]) 790 791 self.namespace = namespace 792 self.builder = builder 793 self.verbosity = verbosity 794 self.update = update 795 796 self.parseArguments(argv[1:]) 797 self.run()
798 799 @property
800 - def namespace(self):
801 return self._namespace
802 @namespace.setter
803 - def namespace(self, value):
804 if csb.core.iterable(value): 805 self._namespace = list(value) 806 else: 807 self._namespace = [value]
808 809 @property
810 - def builder(self):
811 return self._builder
812 @builder.setter
813 - def builder(self, value):
814 self._builder = value
815 816 @property
817 - def verbosity(self):
818 return self._verbosity
819 @verbosity.setter
820 - def verbosity(self, value):
821 self._verbosity = value
822 823 @property
824 - def builders(self):
825 return ', '.join(Console.BUILDERS)
826 827 @property
828 - def program(self):
829 return self._program
830 831 @property
832 - def update(self):
833 return self._update
834 @update.setter
835 - def update(self, value):
836 self._update = bool(value)
837
838 - def run(self):
839 840 if self.update: 841 Config().updateDataFiles() 842 else: 843 Config().ensureDataConsistency() 844 845 builder = self.builder() 846 suite = builder.loadMultipleTests(self.namespace) 847 848 runner = unittest.TextTestRunner(verbosity=self.verbosity) 849 runner.run(suite)
850
851 - def exit(self, message=None, code=0, usage=True):
852 853 if message: 854 print(message) 855 if usage: 856 print(Console.USAGE.format(self)) 857 858 sys.exit(code)
859
860 - def parseArguments(self, argv):
861 862 try: 863 864 options, args = getopt.getopt(argv, 'hut:v:', ['help', 'update-files', 'type=', 'verbosity=']) 865 866 for option, value in options: 867 if option in('-h', '--help'): 868 self.exit(message=None, code=0) 869 if option in('-t', '--type'): 870 try: 871 self.builder = Console.BUILDERS[value] 872 except KeyError: 873 self.exit(message='E: Invalid test type "{0}".'.format(value), code=2) 874 if option in('-v', '--verbosity'): 875 try: 876 self.verbosity = int(value) 877 except ValueError: 878 self.exit(message='E: Verbosity must be an integer.', code=3) 879 if option in('-u', '--update-files'): 880 self.update = True 881 882 if len(args) > 0: 883 self.namespace = list(args) 884 885 except getopt.GetoptError as oe: 886 self.exit(message='E: ' + str(oe), code=1)
887 888 889 if __name__ == '__main__': 890 891 Console() 892