Testing: Integration Testing Server APIs

Integration Testing

Integration testing has many advantages:

1. It safeguards against regressions in API behavior
2. It provides information on application performance
3. It documents APIs used throughout the application

bhima uses mocha and chai-http for integration testing. The tests are found in server/test/api/.

Running Integration Tests

To run integration tests properly, the database must be rebuilt, the server started, and then a call to mocha must be made with the path to the API tests. There is a bash shell script to automate this procedure.

# rebuild the database, build and start the server, and run mocha `/server/test/api/`
./sh/integration-tests.sh

Integration testing takes less than a few minutes. Performance varies by machine, but the bottleneck is typically building the test database.

Integration tests are designed to ensure that all APIs have uniform behavior, and should test every possible HTTP endpoint. Complex endpoints (such as a /search?paramA=valueA&paramB=valueB) should have a variety of tests that demonstrate the endpoint can handle complex queries and fail gracefully. Because integration tests are lightweight, more tests are generally better than fewer tests.

Writing Integration Tests

In order to create uniform behavior between APIs, bhima has helper utilities to facilitate writing new tests. As always, see the source for full documentation.

var chai = require('chai');
var helpers = require('./helpers');

// configure the test environment
helpers.configure(chai);

Testing Secure Routes

Most API tests are against protect routes which require a session. There is a method login() to facilitate these types of tests.

// get a chai-http instance to use
var agent = chai.request.agent(helpers.baseUrl);

// before the test, make sure we are signed in
before(helpers.login(agent));

Testing CRUD APIs

The most frequently created APIs are CRUD APIs that perform specific database actions via HTTP verbs and reply with specific HTTP status codes to indicate the result of the action. Since these tests are so common, there are some utilities to help evaluate the status of a request.

/**
 * asserts that the server sent a JSON-encoded response with status code 201.  The
 * response should contain either an ID (integer) or UUID (36 character length string).
 */
helpers.api.created(res);


/**
 * asserts that the server sent a JSON-encoded response with status code 204.  The
 * response should have an empty body.
 */
helpers.api.deleted(res);


/**
 * asserts that the server sent a JSON-encoded response with status code {code}.  The
 * response should contain the properties "code" (translatable error code) and "reason"
 * (human readable error message). 
 */
helpers.api.errored(res, code);


/**
 * asserts that the server sent a JSON-encoded response with status code 200.  The
 * response should be an array of length {length}.
 */
helpers.api.listed(res, length);

What should be tested?

Every HTTP endpoint should have at least one test. For more complex endpoints, or endpoints that may return different results based on server state, there should be multiple tests.

API Test Quick Checklist

Since that majority of bhima's integration tests concern APIs, below is a quick checklist of routes and conditions to check when creating API tests.

• GET /resources should return an empty list of no resources present in the database.
• GET /resources/unknown should return a 404 NOT FOUND error (unknown id)
• GET /resources/:id should return a 200 OK HTTP status header for an existing id
• GET /resources?param=value should transform output based on the query string. For example, ?detailed=1 should be tested by making sure all the properties of the resource are returned.
• POST /resources should succeed for a valid object, with a specified id/uuid.
• POST /resources should succeed for a valid object, without a specified id/uuid.
• POST /resources should fail (400 BAD REQUEST) for an invalid object (missing keys, failed unique constraints, etc).
• PUT /resources/:id should succeed for a valid update, and return the full changed object.
• PUT /resources/unknown should fail (404 NOT FOUND) for an unknown id.
• PUT /resources/:id should fail (400 BAD REQUEST) for an invalid update.
• DELETE /resources/:id should succeed for a resource that can be deleted.
• DELETE /resources/unknown should fail (404 NOT FOUND) for an unknown id.
• DELETE /resources/:id should fail (400 BAD REQUEST) if the resource has a constraint preventing it from being deleted. For example, a reference constraint.