This version of the Ed-Fi ODS / API is no longer supported. See the Ed-Fi Technology Version Index for a link to the latest version.
How To: Use the ODS / API Smoke Test Utility
Chris Moffatt (Deactivated)
Julie Remde
The ODS / API Smoke Test Utility is a console application that will run each dynamically generated endpoint of the ODS / API. Its purpose is to verify an installation and operation of an ODS / API, and is useful as part of a continuous build or continuous deployment environment. Although it does not perform complex scenario tests, the Smoke Test tool does methodically perform operations in scenario-like orders.
The EdFi.SmokeTest.Console project is located in the LoadTools solution that is distributed with the Ed-Fi ODS / API. The LoadTools solution may be built using Visual Studio 2015. Because the Smoke Test Utility uses Swagger metadata and an externally built SDK, it does not need to be recompiled when the ODS changes.
The Smoke Test tool has three modes of operation:
- Non-Destructive API Tests. Use Swagger metadata to identify and invoke the non-destructive API methods of the ODS / API.
- Non-Destructive SDK Tests. Use a pre-built SDK library to invoke the non-destructive API methods of the ODS / API.
- Destructive SDK Tests. Use a pre-built SDK library to invoke all the API methods of the ODS / API.
Destructive SDK Tests alter the target database and may not completely clean up after themselves. These tests are best used against a QA or test installation of the ODS, not production data.
The two non-destructive tests are identical in functionality in that they both: test for general connectivity; retrieve a session token; and invoke variations of GET scenarios on each API endpoint. However, they differ in that the Non-Destructive SDK tests use a customer-supplied SDK to access the API while the Non-Destructive API tests use a native HTTP Client to invoke dynamically the API endpoints advertised via the Swagger endpoint.
The Destructive SDK tests extend upon the functionality of the non-destructive tests by testing POST, PUT, and DELETE endpoints. Extensions as well as core entities and attributes are tested.
Profiles and composites are not tested by the Smoke Test Utility at this time.
| Test | Non-Destructive API | Non-Destructive SDK | Destructive SDK |
|---|---|---|---|
| HTTP Library | HTTP Client | Rest Sharp | Rest Sharp |
| Get Static Page | Yes | Yes | Yes |
| Get Swagger Metadata | Yes | No | No |
| Get Session Token | Yes | Yes | Yes |
| Get All | Yes | Yes | Yes |
| Get All (skip and limit) | Yes | Yes | No |
| Get by Id | Yes | Yes | No |
| Get by Key | Yes | Yes | No |
| Get by Example | Yes | Yes | No |
| Post | No | No | Yes |
| Put | No | No | Yes |
| Delete | No | No | Yes |
A failure condition for earlier tests will prevent subsequent tests from being run. For example, if Get All fails or returns no results, then Get by Id can not run because there are no resources retrieved that may be used to retrieve a valid Id. Likewise, if Post fails, then there is no entity for Put to update or Delete to remove. This situation will log a "skipped" comment for the remaining operations in an API.
Test Execution Order
For destructive tests, the Smoke Test Tool generates an execution order using the dependencies exposed in the SDK library. This order, while correct and comprehensive, may be somewhat non-deterministic when moving between SDK libraries. Where the results of smoke tests are to be compared, it is recommended that the results be sorted.
Logging
The Smoke Test Tool uses log4net to capture all relevant test information. Out of the box, the tool logs to the console and to a rolling log file that is created in the same directory as the console application. These settings may be changed as necessary by altering the application .config file. Consult the log4net documentation for detailed configuration settings and documentation.
Command-Line Parameters
Built-in help documentation is available for the Smoke Test tool by running the tool with no parameters.
| Parameter | Description | Non-Destructive API | Non-Destructive SDK | Destructive SDK |
|---|---|---|---|---|
a, apiurl | The web API url (e.g., http://server/api/v1.0) | Required | Required | Required |
k, key | The web API OAuth key | Required | Required | Required |
l, library | The complete path to a compiled Ed-Fi SDK library | N/A | Required | Required |
m, metadataurl | The OpenAPI (Swagger) metadata url (e.g., http://server/swagger) | Required | N/A | N/A |
n, namespace | Override the URI to use when generating namespace values (e.g., http://edfi.org) | Optional | Optional | Optional |
o, oauthurl | The OAuth url (e.g., http://server/oauth) | Required | Required | Required |
s, secret | The web API OAuth secret | Required | Required | Required |
t, testset | The test set to run | NonDestructiveApi | NonDestructiveSdk | DestructiveSdk |
?, Help | Show command line help | Optional | Optional | Optional |
A Word About Security
Because the Smoke Test Utility operates under the security constraints of the provided key/secret, some operations may be invalid depending upon the security configuration and the state of other entities in the database. These characteristics are not available for the utility to examine and determine. Intermittent warnings and errors may possibly be logged due to the inaccessibility of resources for security reasons.
Security may be disabled for all keys/secrets by running the following query against the EdFi_Security database:
update [EdFi_Security].[dbo].[ResourceClaimAuthorizationMetadatas] set AuthorizationStrategy_AuthorizationStrategyId = 1
This statement replaces the authorization strategy for all resource types to be "no further authorization required".
Deleting the EdFi_Security database will cause it to be rebuilt with the default claims the next time the websites are run.