Overview

Data Standard v3.0a is the proposed next major version of the Ed-Fi Data Standard.

This new version includes a number of revisions to the Ed-Fi Unifying Data Model (UDM). Those changes are reflected in the REST API and XML bindings for the standards. Potential adopters should read about changes made in Data Standard v2.1, as changes in Data Standard v3.0a are additive to those in v2.1. Note that v2.1 was never operationalized in Ed-Fi technology (i.e., no releases of the Ed-Fi ODS / API are aligned with Data Standard v2.1).

In general, Data Standard v3.0a proposes a set of changes to the Unifying Data Model which continue the work of addressing issues and gaps discovered in field work. By themselves, these "iterative" changes might not be sufficient to merit a major release. The major release is principally designed to signal a few larger changes, covered below in the Structural Changes section. 

Structural Changes from v2.1

This section outlines the major changes between the proposed changes in v3.0a and v2.1.

New Extensibility Pattern

The patterns used to extend implementations based on the Ed-Fi Unifying Data Model have been revised substantially. The principal change is that the v3.0 extensibility model requires namespaces for all extensions to the model.

In API and XML bindings for the UDM, the standard also defines sub-entities to hold extension elements, and, for the API binding, a specific path to identify clearly an API resource as an extension.

{
    "studentUniqueId": "397589871",
    "firstName": "John",
    "lastSurname": "Ortiz",
    "tribalAffiliation": "Pascua Yaqui"
}
  
  
  
  
  

{
    "studentUniqueId": "397589871",
    "firstName" : "John",
    "lastSurname:" "Ortiz",
    “_ext” : {
        “SomeState” : {
            “tribalAffiliation” : “Pascua Yaqui”
        }
    }
}

Listing 1. Comparison of data standard extensibility in REST API resource binding

In v2.x versions of the Data Standard, by contrast, extension elements were mingled alongside core elements of the Ed-Fi data model. This resulted in two issues:

1. It generated confusion as to which elements were core to the Ed-Fi model and which were extensions. This was a serious issue for API client developers and API implementers looking to leverage standardized data exchanges. The proposed segmentation and namespacing clarifies which elements are core and which are not, and also provides indications of the provenance of extended elements.
2. If an API used more than one set of extensions, name collisions from those sets were possible. By introducing namespaces, the risk of name collisions across extensions is eliminated.

Removal of Types (DATASTD-1088)

Data Standard v3.0 changes how standardized enumeration sets (i.e., controlled vocabularies) are supported in the data model.

The entity class of types (e.g., RaceType) is removed from the UDM, and with it the presence of two different classes of enumeration. Where in v2.x there were both Ed-Fi descriptors and types, in v3.0a there are only descriptors (e.g., RaceDescriptor).

Instead of type sets, standardized enumeration sets provided as part of the Ed-Fi Data Standard will be surfaced as descriptor sets in the "ed-fi.org" namespace. It is possible for the Ed-Fi Data Standard to endorse an externally governed enumeration set (e.g., ISO country or language codes), and if so, the appropriate namespace can be employed if needed.

Mappings from descriptors to types in the v2.x UDM have been removed, as they introduced confusion as well. For example, if a mapping exists, which value is a client supposed to employ and in what context(s)? Instead, the Alliance envisions that implementations (including those provided by the Alliance and its community via the Ed-Fi ODS / API) will provide for mappings between different descriptor sets, allowing for support of multiple "operational contexts" (contexts in which controlled vocabularies are different for the same underlying concept). The more systematic use of namespaces will also serve to clarify operational context.

Major Changes from v2.1

Addressing Volatility of GradingPeriod Key (DATASTD-1090)

The GradingPeriod key contains a date (StartDate). Dates have generally proven in field work to be volatile (subject to change and entry error). Since GradingPeriod cascades extensively in the model, a new key was proposed to address this volatility.

This proposed change essentially continues work from Data Standard v2.1, which removed volatility in composite key fields. With changes in Section key (from DataStandard v2.1) and GradingPeriod (v3.0a) now present in the model, implementers are advised to consider impacts to existing exchanges.

Introduction of Period Entity and Pattern to Capture Entity History (DATASTD-1085, DATASTD-1092)

Over time, the ability of the model to capture multi-year data has grown in importance in the community. Implementers have sought to leverage the standard to exchange historical data, and have also extended the model to make the data "multi-year" aware. This work has been difficult. Since identity in UDM bindings such as the ODS / API is driven by natural keys, dates are often needed to distinguish repeated events and to capture history (e.g., StudentSchoolEnrollments, where BeginDate is part of the key and allows implementations to capture a history of re-enrollment).

While forcing data providers to supply dates may work in areas where dates are strongly governed (e.g., enrollment), in other situations where dates are not strongly governed (e.g., a student changes their name) this practice may force on data providers a requirement that cannot be fulfilled (e.g., the data provider system may not record dates for the name change event).

Data Standard v3.0a proposes a pattern for capturing history via a Period entity, employed as an optional collection. This model allows for entities to capture and retain dates reflecting periods in which the data was valid.

      "addresses": [
      {
        "addressTypeDescriptor": "Home",
        "stateAbbreviationDescriptor": "TX",
        "streetNumberName": "89 South Milton St",
        "city": "Grand Bend",
        "postalCode": "78834",
        "periods": [
          {
            "beginDate": "2017-08-12",
            "endDate": "2017-11-20"
          },
         {
            "beginDate": "2018-02-01",
            "endDate": "2017-06-06"
          }
        ]
      }

Listing 2. Example of Period entity to capture entity history

The principal concern with the pattern is that in places where multiple clients write/exchange the same entity, it seems possible that a client without the historical context could overwrite the data in the Period entity.

Changes to Reduce Overlap in Capture of Student Characteristics (DATASTD-1083, DATASTD-1084)

Over time, the Ed-Fi Unifying Data Model has acquired multiple, often overlapping ways to capture concepts around (broadly speaking) "student characteristics." In some cases, the semantics of these entities has also become overly broad, further inviting overlap. A few changes were made to improve this situation.

All Individual Changes