The Ed-Fi Resources API, based on the Ed-Fi Data Standard, provides fine-grain access to educational data, modeled largely on the common denominators of the source systems that provide the data. For applications that consume data from an Ed-Fi API, this can result in a very “chatty” application integration: the consumer must make large numbers of calls over the network to retrieve the required data.

Furthermore, the authorization model in an Ed-Fi API is designed for client-server interactions, not for user interactions. Thus, the Ed-Fi API should not be used directly from a user interface.

In this article, we will explore design patterns and implementation concerns for building backend applications that address these problems.

Data Access Patterns

In the data access patterns below, user applications or other backend services call another service that sit between them and the Ed-Fi service host. The Ed-Fi client application could be providing data through many different access patterns. The most common access patterns are described below under the heading of Frontend API Design Patterns.

Direct Database Interaction

How it works

A dedicated backend application interacts directly with the Ed-FI database.

When to use

The Ed-Fi Alliance strongly discourages this pattern for the following reasons:

1. It bypasses the authorization security in the Ed-Fi API, potentially affecting both read and write operations.

2. This approach may put too much strain on the primary data storage, causing resource contention for other Ed-Fi client applications that are using the Ed-Fi API.

3. The Ed-Fi database is not a standard. Thus, different implementations or even different versions of the same implementation could have unstated breaking changes at the database layer. For example, the Ed-Fi ODS/API Platform and the Ed-Fi Data Management Service (unreleased at the time of writing) have very different backend database structures. An integration built on the ODS/API’s EdFi_ODS database would not be compatible with the Data Management Service’s database.

Implementation Notes

Although no longer considered advisable, the performance benefits make this a tempting option. Many applications have been built on this model in the past. In such cases, it is advisable to limit the direct database interaction to read operations only, and to run from a read-only database copy. The copy could be a snapshot or replica. Using a read-only copy mitigates the resource contention concern on the primary database. Limiting to read operations eliminates half of the authorization security concern.

Also see Row-Level Authorization below.

Real-time API Interaction

How it works

A dedicated backend application interacts directly with the Ed-FI API, translating the incoming coarse-grained request into many fine-grained requests that utilize the Ed-Fi Resources API or other API specifications implemented in the Ed-Fi service application.

When to use

Implementation Notes

Batch and Save

How it works

The backend application retrieves

When to use

Implementation Notes

Streaming Data

How it works

When to use

Implementation Notes

How it works

When to use

Implementation Notes

Frontend API Design Patterns

Backend-for-frontend

How it works

This pattern starts from the needs of a specific user interface, creating a finely-tuned API specification that optimizes data transfer for that application.

The backend-for-frontend service (BFF) then handles translation of the custom specification into requests

When to use

There is only a single front-end application that needs access to the Ed-Fi resources.

Implementation Notes

Central Aggregating Gateway

How it works

When to use

There are multiple front-end applications with different use cases.

Implementation Notes

GraphQL

How it works

When to use

Implementation Notes

Cross-cutting Concerns

Row-Level Authorization

Caching

Ed-Fi API Client Credentials

Define the problem:

• Course-grained access to fine-grained resources (“chattiness”)

  • Network latency

• Getting rid of composites

• Alternate security protocols

Solution: create a specialized backend API, which sits “close” to the Ed-Fi API (for low network latency).

• Central Aggregating Gateway: more generic design than a BFF, can support multiple user interfaces

• “A central-purpose aggregating gateway sits between external user interfaces and downstream microservices and performs call filtering and aggregation for all user interfaces. Without aggregation, a user interface may have to make multiple calls to fetch required information, often throwing away data that was retrieved but not needed.”

• BFF: designed for a single front-end

• “The main distinction between a BFF and a central aggregating gateway is that a BFF is single purpose in nature—it is developed for a specific user interface.”

• https://learning.oreilly.com/library/view/building-microservices-2nd/9781492034018/ch14.html by Sam Newman, ch 14, O’Reilly Media, Inc.

• Solves the impedance mismatch between systems.

• Similar to the Facade pattern in OO systems

Alternatives

• GraphQL - why not? Well-defined need is easier to express and reason about. Create a simple single definition of an API.

Design

• BEF app needs a key and secret to an ODS/API. Maybe to many of them.

• Analyze your authorization strategy and needs.

• Cache local data for higher performance.

  • Possible implication: ETL

• Change Queries (ETL!)

  • Show basic interaction

  • Definitely(?) implies caching.

• Storing multiple tenants' data? Decide on a multi-tenancy pattern for segregating the data in the caching layer.

• Evolutionary diagrams:

  • FE to BFF to BEF to ODS/API

  • Add caching layer and ETL

• Gateway Aggregation pattern

  • Resiliency patterns and more