This version of the Ed-Fi ODS / API is no longer supported. See the Ed-Fi Technology Version Index for a link to the latest version.

 

What's New - Previous v3.x Releases

Owned by Vinaya Mayya
Dec 18, 2018

This documentation provides a summary overview of the major improvements, updates, fixes, and changes made in previous releases of the Ed-Fi ODS / API v3.x suite and provides links to additional documentation. This information is likely of interest to those upgrading from the Ed-Fi ODS / API v2.x. 

Contents:

Detail follows.

ODS / API v3.0

This section provides an overview of the new features included in the v3.0 release of the Ed-Fi ODS / API.  

New & Updated Standards

Several changes were made to the ODS / API v3.0 to adhere to standards.

Alignment With the Upcoming Ed-Fi Unifying Data Model v3.0

Every generation of Ed-Fi technology is aligned with the Ed-Fi Unifying Data Model. The ODS / API v3.0 implements the Unifying Data Model v3.0, which contains improvements and additions based on feedback from implementers.

Upgrade to OpenAPI 2.0 Specification

This change upgrades the Ed-Fi ODS / API to use the Swagger / OpenAPI specification 2.0.

The previous Ed-Fi ODS / API v2.x produced Swagger 1.2 metadata. This specification is out-of-date, and tooling support is lacking going forward — which means that the Swagger UI could no longer be updated without an upgrade to the metadata.

Overview

The upgrade to OpenAPI 2.0 brings the following benefits:

  • Moves to an actively supported and maintained version of Swagger UI.
  • Adopts Swagger community tools for generating client SDKs for the ODS / API. This provides API SDK support for languages beyond C# and Java, as well as enables the Alliance to deprecate costly support and maintenance for the previous Client API SDK. 
  • Standardizes the return type for HTTP GETs (discussed further, below).
  • Enables significantly improved runtime performance of the latest Swagger UI.

Code Generation

The Swagger Codegen tooling is a template-driven engine that can generate documentation, API clients, and server stubs in different languages by parsing an OpenAPI specification. Client languages include ActionScript, Bash, C#, C++, Erlang, Go, Groovy, Haskell, Java, Node.js, Objective-C, Perl, PHP, PowerShell, Python, R, Ruby, Scala, Swift, and others.

The free, open-source Swagger Codegen project is maintained in GitHub at https://github.com/swagger-api/swagger-codegen. Any version of Swagger Codegen compatible with the OpenAPI 2.0 specification can generate client code for the Ed-Fi ODS / API v3.0.

Design Details

The metadata exposed by the Ed-Fi ODS / API is used for two purposes. One is to allow interactive exploration of the API using Swagger UI. The other is to generate SDK client code. With these use cases in mind, the API metadata is being reorganized around a set of distinct metadata endpoints. 

  • As with the current implementation, there will be API sections for Resources, Descriptors, Bulk, Identity, and one for each Profiles and Composites.  
  • ODS / API v3.0 introduces new metadata endpoints specifically targeted for code generation.

The table below summarizes these endpoints.

NameDescriptionPrimary Target Audience
ResourcesIncludes all the data collection resources including Ed-Fi Extensions (but excludes Ed-Fi Descriptors).API consumers and application vendors using Swagger UI.
DescriptorsIncludes only Ed-Fi Descriptors (including Descriptors added via Extensions).API consumers and application vendors using Swagger UI.
BulkContains all operations related to bulk data loads.API consumers and application vendors who are operating against an API that supports bulk loads.
IdentityContains all operations supporting integration with an external UniqueId system.API consumers and application vendors working with APIs that have an integration with an external Unique ID system.
Profile-SpecificIncludes the relevant subset of the Standard API Resources (including Extensions and any referenced Descriptors) with appropriately constrained surface areas.Specialty vendors, for data collection and/or integration.
Composite-SpecificIncludes read-only resources defined as an Ed-Fi Composite. Includes any referenced Descriptors.API consumers and application vendors focused on data integration activities in a specific subject area.
Data Collection APIIncludes all Descriptors and Resources, including all Extensions.SDK generators.

Standardized Use of OAuth 2.0 Client Credentials Grant Type

The ODS / API v2.x authentication process used a customized approach that looked like "2.5 legged OAuth," meaning that it leveraged idioms from both 2-legged and 3-legged OAuth, but followed neither specification exactly. Since the API authenticates systems (not users) and treats those systems as the "resource owner," no "authorization code" is necessary, but was nevertheless required in the v2.x authorization flow implementation.

Rather than continue this custom implementation, the ODS / API v3.0 implements the OAuth 2.0 Client Credentials Grant Flow. This enables the ODS / API v3.0 to work "out-of-the-box" with standard tools and SDKs.

The API authentication process no longer uses "authorization codes," and thus all technical artifacts related to the concept have been removed. This necessitated changes to the Admin database by removing a table (dbo.ClientAuthorizationCodes), updating stored procedures (e.g., AccessTokenIsValid), as well as various C# classes (e.g., the OAuthController and its associated artifacts). New tests have been written to cover the new functionality, and obsolete tests have been removed.

Additionally, all behavior related to the rolling expiration functionality of the existing tokens has been removed to comply with the OAuth 2.0 standards.  

New & Improved Features

This section summarizes the major new features of the ODS / API v3.0. 

Decoupling of Ed-Fi Core and Extensions in the API

The goal of this feature is to completely decouple the API artifacts for Ed-Fi Core from those related to Ed-Fi Extensions. This change allows implementations to support Ed-Fi Core data elements and add s based on a given configuration. It is now also possible to apply multiple Extensions simultaneously, such as host-specific Extensions (e.g., as implemented by many state education agencies), along with domain-specific Extensions (e.g., talent management, transportation).

Support for Extensions in JSON Resource Models

Instead of a seamless blending of extended elements into the JSON payload (previous implementation), extended elements of an existing entity are held in an extension JSON object. ODS / API v2.x and ODS / API v3.0 examples follow.

ODS / API v2.x

ODS / API v3.0

{
    "studentUniqueId": "397589871",
    "firstName": "John",
    "lastSurname": "Ortiz",
    "tribalAffiliation": "Pascua Yaqui"

    [etc...]

}

{
    "studentUniqueId": "397589871",
    "firstName" : "John",
    "lastSurname:" "Ortiz",
    
 [etc...]
 
    “_ext” : {
        “SomeState” : {
            “tribalAffiliation” : “Pascua Yaqui”,
            [etc...]
        }
    }
}

The Improvements to Routes section that follows includes an overview on how these changes are reflected in the URI routes. 

MetaEd IDE Extension Support

The Ed-Fi ODS / API can easily be extended using the MetaEd IDE. MetaEd v1.2 and above provides support for the ODS / API v3.0. See MetaEd IDE - What's New for details.

XML Shredder Support for Decoupled Extensions

XML Shredder support has been enhanced for decoupled extensions. The ODS / API 3.x supports multiple extensions, though support for bulk load is limited to one extension at a time. Multiple extensions can be loaded by configuring and running bulk load for different extensions in succession. 

Standardization of Return Types for HTTP GET Operations

Previous versions of the Ed-Fi ODS / API implemented variable GET semantics to reflect the natural key model of the Ed-Fi Data Standards. For example, "Get By Key" required natural key fields and return a single JSON object whereas "Get By Example" used supplied fields as search values, but returned a JSON collection.

In this previous model, developers could inadvertently trip over the difference by not catching situations in which a natural key values are submitted. Further, these types of variable semantics are not supported in OpenAPI v2.0 (and it's unclear if they will return in OpenAPI v3.0).

The ODS / API v3.0 moved to standardized return types for GET operations. In v3.0, the supported GET operations are:

  • A standard GET operation that returns resources using a search of entity properties. The values of properties of the resource that are specified will be used to return all matching results, if any.
  • A GET operation that retrieves a resource by the specified resource identifier.

See ODS-894 for further details.

Improvements to Routes

The ODS / API v3.0 includes improvements to the URI routes that clients use to connect to the platform.

Improvement Summary

The following changes will be made to the URI paths, both to support the new Extensions model, and provide improved support for API versioning as shown.

DescriptionODS / API v2.x (Previous)ODS / API v3.0
Base URI ../api/v2.0/2018 

/{api_area}/{version}/{year}*

Examples:
/data/v3
/composites/v1
/identity/v2
/bulk/v1

Basic Relative URL Format

Data Management Resources:
…/{org}/{resource}

API Composites:
.../{org}/{category}/{resource}

Examples
Resources (Ed-Fi)/api/v2.0/2018/schools//data/v3/ed-fi/schools
Descriptors (Ed-Fi)/api/v2.0/2018/termDescriptors/data/v3/ed-fi/termDescriptors
Resources (Host Extensions)/api/v2.0/2018/leavers/data/v3/TX/leavers
Resources (Domain Extensions)/api/v2.0/2018/applicants/data/v3/talentMgmt/applicants
Composites (Ed-Fi)

/api/v2.0/2018/assessment/studentAssessments

/composites/v1/ed-fi/assessment/studentAssessments
Composites (Host Organization)Custom/composites/v1/MI/enrollment/Schools
Composites Based on Domain or Host ExtensionsN/AUnsupported
Bulkapi/v2.0/2018/bulkOperations/bulk/v1/bulkOperations
Identity

api/identity/v1/identities
api/identity/v2/identities

/identity/v2/identities

Notes:

* School Year is now optional in the route
** Composites are treated as a reserved word
*** “Org” name registration will be managed by the Ed-Fi Alliance (or self-managed by community through conventions for SEAs, LEAs)

Separate Bulk and Identity Endpoints

In addition to the general improvements noted above, the endpoints for bulk and identity features are now clearly separated. This allows separate versioning for these features, and also better supports implementations that are not using the optional identity and bulk features.

Examples include:

  • /data/v3
  • /identity/v2
  • /bulk/v1

Configurable School Year

Some implementers rely on including the school year in the API root, while others do not. The presence of the school year value in the route is now configurable, and off by default.

School year can be configured in the API route through the following steps:

  • Update owin:appStartup key in the Web.config of the EdFi.Ods.WebApi project to:  <add key="owin:appStartup" value="YearSpecific" />
  • Update swagger.webApiMetadataUrl key in the Web.config of the EdFi.Ods.SwaggerUI project to: <add key="swagger.webApiMetadataUrl" value="http://localhost:54746/metadata/2017/"/>

Support for Multiple Operational Contexts

The ODS / API v3.0 sets the stage to support multiple operational contexts.

Ed-Fi Descriptor References Require Fully Qualified Namespaces

The Ed-Fi Descriptor pattern allows controlled vocabularies to be customized for the needs of a particular context. Many commonly used option lists such as Academic Subject, Grading Period, Language, Term, and so forth are defined by Descriptors.

All Descriptor references now require namespaces, and do not rely on the concept of a default operational context. This change ensures that the semantics of all messages are clear, even in transit. This change also communicates information about who governs the values being sent for the controlled vocabularies that make up a particular Descriptor.

The Descriptor references must be formatted as follows:

uri://[organization indicator]/[name of descriptor]#[code value]

The following are example from the Ed-Fi Alliance namespace, which includes the default values in the as-shipped ODS / API:

uri://ed-fi.org/AcademicSubjectDescriptor#Chemistry
uri://ed-fi.org/GradeLevelDescriptor#Fifth grade 

Namespaces are assigned by platform hosts. Generally speaking, namespaces identify the originating entity in the [organization indicator] segment. In addition, the namespace adheres to the following patterns:

  • Valid namespace format is: [scheme]://[organization indicator]/[descriptor name]
  • Valid scheme is always 'uri'
  • Valid organization indicator can only contain alphanumeric and $-_.+!*'(),
  • Valid descriptor name can only contain alphanumeric
  • Valid code value can contain anything except '#'

The ODS / API v3.0 accepts the new format and provides validation errors for data in the incorrect format. 

Code Refactoring

The ODS / API v3.0 includes a refactored code base for easier understanding and maintenance.

  • The source code reorganization addresses complexity and establishes a foundation for more manageable, composable releases. For example, the refactoring will support distribution of the ODS / API core via NuGet packages in a future release.
  • A reengineering of the API code generation templates addresses technical debt, and prepares for API code generation support using the Ed-Fi MetaEd v1.x software.

Note that these code refactoring changes for ODS / API v3.0 were also applied to the ODS / API v2.3 release.

Version 3.0 Update Impact & Guidance 

This section is primarily of interest to current platform hosts and client implementers of the Ed-Fi ODS / API. It contains general guidance about updating from previous versions, and a summary of the impact that the major changes have on existing implementations. The Feature & Change Summary contains detail about each change. 

The ODS / API v3.0 was designed to minimize breaking changes and impact on current technology implementers. However, some improvements and additions with a non-zero impact were deemed necessary by the Ed-Fi Community.

ChangeImpact on Platform HostsImpact on Client Developers
Data Standard Support
  • Review any current Ed-Fi Extensions to see if added elements are included in the new Data Standard. 
  • Apply and test Extensions, coordinate roll-out and support with API consumers though data governance, sandbox testing, and so forth.
  • Review client application API interactions for breaking changes in JSON payloads due to data model changes.
  • Adapt client code to any data model changes.
Ed-Fi Extension Decoupling
  • Review new Extension capabilities.
  • Client system developers will need to accommodate any platform changes or enhancements in API integration code.
Swagger / Open API Updates
  • Review new documentation and sandbox capability.
Standardization of Return Types for HTTP GET Operations
  • Review and understand new, simplified GET patterns.
  • Review and understand new, simplified GET patterns.
  • Adapt client code for GET operations to new specifications.
Standard OAuth 2.0 Alignment
  • Review and understand standard OAuth 2.0 Client Credentials Grant Flow.
  • Adapt client code to request a new access token upon token expiration.
  • This requirement will likely be familiar to developers using OAuth 2.0 in other implementations. The ODS / API v3.0 removes the previous non-standard OAuth implementation, which included a rolling expiration for tokens.
Descriptor References Require Namespaces
  • Review and consume the updated code.
  • Review all Ed-Fi Descriptors, ensure any localized Descriptors (i.e., those with values other than the as-shipped Ed-Fi Descriptor sets) adhere to the fully qualified namespace pattern. An overview of the feature and the namespace naming requirements is available in the New & Improved Features section of the Feature & Change Summary documentation.
  • Publish updated documentation, notify client systems.
  • Adapt client code to new specifications.
Code Refactoring
  • Review and consume the refactored code.
  • If applicable, re-apply customizations.
  • No impact.

* MetaEd v1.2 and above provides support for  ODS / API v3.0. See /wiki/spaces/METAED/pages/18941343 for details.