WRITING A TEST ============== This test framework is intended to test low-level Lintian checks and behavior with manually constructed source packages. It provides a mechanism to manually construct pathological source packages and run lintian on them. It should only be used when this degree of low-level control is necessary. For other, more conventional cases, use the tests framework. A test in this framework is a directory containing (at least) a tags file and a Makefile. The directory may contain other files as needed for that test. The tags file ------------- The expected output of Lintian when run on the package, including info and experimental tags. The Lintian output will be lexicographically sorted before comparing it with tags. This file may be empty if the test case should produce no Lintian output. The Makefile ------------ Makefile must be a GNU makefile which, when invoked via make (using the default target), creates a .dsc file in the current directory and any other files needed to make up a source package. Any .dsc file found in the directory will be checked. must match the directory name. This makefile may assume that the current working directory when it is invoked is the directory in which it's located (or a copy of it). It can take any action needed to generate the deb file, such as running dpkg-source. The Makefile does not have to clean up after itself. It can assume that the directory is deleted and recreated between test runs. The desc file ------------- Please refer to t/tests/README for the full description of the desc file. Skipping tests -------------- To mark a test to be skipped, create a file named "skip" in the test directory containing the reason for why this test should be skipped. This should be used for tests for work that hasn't been finished or tests that must be temporarily skipped due to bugs in other packages. TEST NAMING CONVENTIONS ======================= Each test name should begin with the name of the part tested, e.g. -... -... lintian-... lintian-info-... Use generic- as a prefix for test cases that don't cover a specific portion of Lintian but instead test Lintian's behavior on a useful special case of package (such as a minimal package). RUNNING THE TEST SUITE ====================== The complete test suite will be run with debian/rules runtests, but this can take quite a lot of time. Normally this is only necessary after significant structural changes or before a release as a final check. To run a specific test case, run: debian/rules runtests onlyrun= It's often more useful to run every test that is relevant to a particular tag. To do that, run: debian/rules check-tag tag= This will run all tests that list that tag in the tags file. TEST WRITING TIPS ================= Please keep each test case focused. One of the problems that developed with the old test suite is that each test was serving many separate purposes and testing large swaths of Lintian, which made it difficult to know what could be changed and what would destroy some other useful test. Test cases should only test a set of closely related tags and new tests should be added for new issues that aren't part of that closely-related set. Test cases should be as Lintian-clean as possible except for the tags that they're testing for. The template is intended to help with this. It generates a Lintian-clean basic package for you to start with. You should override only the minimal required to trigger your test, and try to fix any unrelated problems. Sometimes this won't be possible and the only way to trigger a tag is to also trigger another tag, and that's fine, but it shouldn't be the normal case.