Functional requirements

Tools

Test data

Tools

Libraries

Testdata

Framework

Path Coverage [To achieve 100% path coverage, at least one request must address each path of the API. Four HTTP requests are needed, one per path, to reach 100% path coverage.]

Operaton Coverage [To achieve 100% operation coverage, every path must be sent one request per allowed HTTP verb (GET, POST, PUT or DELETE).]

Parameter Coverage [To achieve 100% parameter coverage, all input parameters of every operation must be used at least once. Exercising different combinations of parameters is desirable, but not strictly necessary to achieve 100% of coverage under this criterion.]

Parameter Value Coverage [The coverage is computed as the number of different values that parameters are given divided by the total number of possible values that all parameters can take. To achieve 100% parameter value coverage, every boolean and enum parameter must take all possible values. Nevertheless, it is suggested to test multiple values with other types of parameters such asstrings and integers.]

Content-Type Coverage [To achieve 100% input content-type coverage, for every operation that accepts a request body, all data formats (e.g. JSON and XHTML) must be tested.]

Operation flow coverage [for every resource that can be created, at most four operation flows must be tested, namely those related to its reading (one or several), updating and deletion after its creation.]

Status code class coverage [the coverage of a test suite according to its ability to produce both correct and erroneous responses in the API under test. These responses are typically identified by 2XX and 4XX status codes respectively]

Status code coverage [This criterion extends the previous one by considering status codes instead of simply classes of status codes. Therefore, to achieve 100% coverage, all status codes of all operations must be obtained.]

Response body properties coverage [The coverage of this criterion is computed as the number of properties obtained divided by the total number of all properties from all objects that can be obtained in API responses. To achieve 100% coverage of this criterion, all properties from all response objects must be obtained]

Content-Type Coverage [This criterion has the same meaning as the input content-type criterion, but in this case the coverage is measured on the output data formats obtained in API responses.]

Basic Positive Tests

Extended Positive testing with optional parameters

with valid input

with invalid input

deeper form of negative testing to break API to check robustness

Negative testing with huge payload in an attempt to overflow the system

ensure API responds to correct authorization via all agreed auth methods – Bearer token, cookies, digest, etc. – as defined in spec

ensure API refuses all unauthorized calls

ensure that specific endpoints are exposed to user based on role. API should refuse calls to endpoints which are not permitted for user’s role

check HTTP/HTTPS according to spec

ensure that internal data representations that are desired to stay internal do not leak outside to the public API in the response payloads

given()

when()

then()

extract()

Main objective

Security and Authorization [Check that the API is designed according to correct security principles: deny-by-default, fail securely, least privilege principle, reject all illegal inputs, etc.]

Performance [Check API response time, latency, TTFB/TTLB in various scenarios (in isolation and under load)]

Load/Stress [Find capacity limit points and ensure the system performs as expected under load, and fails gracefully under stress] Load (positive) and Stress(negative)

Usability testing [For public APIs: a manual “Product”-level test going through the entire developer journey from documentation, login, authentication, code examples, etc. to ensure the usability of the API for users without prior knowledge of our system.]