...
The ability to retain a JSON document opens many possibilities for downstream processing and eventing. As a document posted to the API represents a "one logical event" in the operations of an school district (e.g., "student X was marked absent on day Y"), the pre-packaging of that data opens up the possibility for other data consumers to consume it as a documents (e.g., the document could be posted to a log of attendance events to which other systems subscribe). Meadowlark itself uses this mechanism to index the documents in a search engine for query support.
The relational format of the ODS data storage delivers other benefits, such as the ability to perform complex validations based on SQL, so it is is not a case of one storage format is better than the other, but that there are use case benefits to each. Indeed, there are also certainly ways where both technologies could be mixed.
Analytics Modules
The Meadowlark team experimented with downstream analytics processing using the above eventing mechanism. API documents were made accessible to AWS Athena, which allows for interactive queries with large-scale data sets. The team made simple visualizations from the API data in Athena with AWS QuickSight, the cloud-native BI tool.
In addition to QuickSight, tools like Power BI Desktop also include support for creating reports and dashboards driven by Athena. It would be interesting to create real use-case driven analytics modules that work with a Meadowlark framework designed for community extensibility.
Reuse of Meadowlark Technology
Meadowlark makes use of MetaEd to generate API document schema validations and to locate natural key and foreign key references in API documents. Some of this is done in a "pre-processing" step that mirrors the behavior of a MetaEd plugin, while the rest is done at API invocation time. This could be moved entirely into MetaEd plugins that generate standard JSON Schema and JSONPath API data from a MetaEd model. This information could be used by the ODS/API platform, for example, to support its own schema validation.
This could also be part of a broader modularization of Meadowlark to enable extensions of Meadowlark created by the Ed-Fi community. With a clean separation of Meadowlark document validation and reference extraction from a web framework, alternatives like Azure Functions or even simple on-premise web application servers become possible. Similarly, separation of Meadowlark's back-end storage, querying and reference validation could allow for community-contributed alternatives like Azure Cosmos DB or local MongoDB instances.