Test Results
After you run the tool, there are two locations where you can view your test results.
The quickest way to look at the test results is to use the HTML button to pop up a browser with the results.
- Click the HTML button or pull down the View menu and select Display Test Report.
The results appear in a new tab in your default browser.
As the tests executing, the tool adds lines to the conformance_results.csv file. You can easily view this file in a spreadsheet application.
- In your file explorer, navigate to the folder that contains the CCT files.
- In the logs/conformancelog folder, open the identifier-datestamp-conformance_results.csv file where identifier is the CHUID's FASC-N or GUID, and datestamp is the start and stop timestamps of the test run. The results will open in your computer's default spreadsheet software, like Microsoft Excel.
Sort the results on the 5th column to group the failures together at the top of the sheet.
Note: If you stop the tool before all of the tests have completed, partial results will not be viewable. Instead, the HTML output from the previous run will be rendered.
CCT uses slf4j which is a simple logging facade for logging systems such as Apache log4j. The log files are configurable using the XML configuration file, user_log_config.xml. It shouldn't be necessary to modify this file. If necessary, refer to the Log4j configuration syntax guide for details.
When the the Verify PIN and Execute Tests button is clicked, the tool notes the date and time. This becomes the start time. When the last test case finishes, the time is again noted. The card identifier, the start and stop time comprise the unique portion of every log file associated with that test run. After the last test case is run, the currently open log files are closed and renamed with the identifier and time stamps. New log files with no time stamps are then created and prepared for the next test sequence.
- Debug - combined output stream from appenders
- Container - contains hexadecimal dump of each tag in each container found
- Conformance - CSV file with test cases and results
- Test progress - start and stop event fired by each test case
- APDU transmitter - bytes sent to the card
The debug log merges the output from all appenders so that almost anything that happened during a test run has been captured in this file. The problem is, the file has multiple types of output, and it becomes difficult to sift through the output. Other logs capture specific output in a specific format described below.
The conformance log is actually a .csv file with a header followed by one row per test case. The result of the test is in the Actual results column.
The tags found in each container are logged to the corresponding container log. Each container is given separate log file, using identifer, start time, stop time, and container name.
Opening up a typical container log shows the data in that container, expressed as a raw byte buffer, then broken into individual tags found in that container. The file is initialized with the expected tags in the order that they are documented in SP 800-73-4.
]
A hexidecimal dump of the values for each tag in each non-empty container can be found in the logs/containers directory.