Vision

What is it?

A research and development effort to explore potential for use of new technologies, including managed cloud services, for starting up an Ed-Fi compatible API.

It is not…

A replacement for the Ed-Fi ODS/API. Portions of it could someday become a replacement, but we are not there.

Research Questions

Some of the questions to be explored in this project include:

• Can we build an API application that supports multiple data standards without requiring substantial coding work for each revision of the standard?
• Can an Ed-Fi API promote events to a first-class concept, supporting notifications, subscriptions, and real-time data transformations?
• How much might a fully cloud-native architecture cost to operate?
• Is it feasible to build a system that is both cloud-native and fully operational on-premises?
• What are the most important features and security models for unlocking widespread deployment across the education sector?

End State Architecture

Taken to its logical conclusion, the end state architecture for Meadowlark would be more of a framework than a monolithic "product", with many different, competing, components that could be substituted into the system to perform designated functions. For example, the Ed-Fi API is an HTTP-based service with many possible implementations. The initial implementation uses AWS Lambda Functions. Alternate implementations could use a stand-alone NodeJs web server - such as Express or Fastify - or could implement the HTTP services in the functional framework for Azure, Google Cloud, etc.

The following diagram deliberately mixes-and-matches generic icon and icons from Amazon Web Services, Google Cloud Platform, Microsoft Azure - thus representing that the desired architecture is meant to be dynamic, pluggable, and platform-agnostic.

General Principles

1. Prefer open source components or protocols.
2. Code with strong separation of concerns in mind, enabling common business logic to be interact with multiple front-end (HTTP) and back-end (data store) components.
3. Provide enough testing to prove viability, but not so much as would be required for a production-ready product.
4. Evolution toward an (database-first) Event-Driven Architecture.

Milestone 0.1.0

DONE

https://xkcd.com/2054/
Some rights reserved:  https://xkcd.com/license.html

Implemented	Not Implemented
​Data Model 3.1 and 3.3b ... and easy to support others Ownership-based authorization Validates authentication token Authorizes client to access data that were written by the client ("ownership") Optional foreign key enforcement With HTTP header, can disable foreign key validation Separate data stores for transaction processing (DynamoDB) and querying (OpenSearch)	​Authentication oauth endpoint is fake, returning hard-coded tokens for demo purposes Cascading deletes Extensions Composites Profiles

Milestone 0.2.0

IN PROGRESS

https://xkcd.com/974

Implemented	Removed or Broken
Implement on alternate data stores: PostgreSQL  MongoDB Full access authorization mode Pure Node.js front end in fastify	AWS deployment (temporarily broken) Data Model 3.1 (temporary; Meadowlark - Multiple Data Standards) Early experiment with ed-org authorization DynamoDB

Milestone 0.3.0 

PLANNED

Low-frills installation for API support without breaking vendor integrations, suitable for pilot testing on Azure.

Looking Beyond Milestone 0.3.0

The development team plans to explore these topics as we look beyond the work necessary to have a (pilot) testable system:

• Meadowlark - Durable Change Data Capture
• Meadowlark - File System Storage
• Meadowlark - Materialized Views