TAG Meeting 2022-06-16

Participants

Agenda

1. Update on breaking changes / API evolution (including communication of data model changes)
2. AMT direction (including API-first strategy)

Materials

• Ed-Fi API Evolution Strategy - for TAG.pptx
• AMT and API-First - for TAG.pptx

Notes

Topic 1: API Evolution

There was a general agreement that some plan was needed and that something that resembled this was a good answer to those needs.

• Would LEAs update on every 2 years - seems overly optimistic. Maybe this need to be longer.
• Could result in managed providers supporting 2 production versions and possibly 1-2 dev versions.  The number of prod versions could be longer if updates were not consumed and agencies stayed on older versions.
• States commonly need to maintain 3 versions, 1 current and 2 historical
• The discovery/metadata URL likely needs to be standardized as well, particularly given this change and breaking changes.
• API fragmentation is not are not a new problem, but introducing breaking changes could make it worse by providing breaks in the core standard.
• Introducing versions of subsets of the API could be helpful.
• Decoupling API code and ODS versions could be helpful for providers by allowing them to maintain the same ODS features while evolving the API.

Topic 2: API-first and AMT discussion

General strong agreement that API-first should be the model for analytics, and not direct SQL access. Some reasons were re-cited, but the meeting did not attempt to re-list all of them or review this general direction. Most comments focused on other aspects of this work:

• Is the expectation to have bulk end-point with JSON from other APIs? Goal is not to change the current API; this is just a tool which may leverage existing tools and move data from the API to fill the data lake
• Discussion of how extensions should work in this model.
• Discussion of adding event-driven processes for GETs: Streaming data out of the ODS/API into Kafka or another message queue as events that can be handled in realtime, instead of needing to extract data from either the API or the database.
• Some discussion of who would use this architecture and if it is therefore worth it to spend Ed-Fi development resources to build it. Several points
  • Seen as too complex for most local education agencies: difficulty for agencies to access the skill set for some of this. There were some counter-examples of smaller districts developing capabilities cited.
  • Such an architectural reference point might be useful for newer providers or large urbans, and that generally capability with data management and analysis was increasing for agencies.
  • A fully deployable architecture might be expensive: are there other aspects of the Ed-Fi roadmap that could use the focus instead of this.
  • EA mentioned that they are releasing a version of this under an open license as well that could play the role of organizing community activity.
  • The Alliance is looking to better document the core use cases that drive Ed-Fi, and that a reference architecture provides a way to do that in code.
  • Some discussion of the possible technical candidates for the various parts of the proposed reference architecture and project.