Meeting 1 - 2020-07-22
Participants
- Britto Augustine, Arizona DOE
- Stephen Fuqua, Ed-Fi Alliance
- Jean-Francois Guertin, EdWire
- Eric Jansson, Ed-Fi Alliance
- Erik Joranlie, EdAnalytics
- Vinaya Mayya, Ed-Fi Alliance
- Jim McKay, Certica
- Laura Michaels, Palm Beach County School District
- Patrick Yoho, InnovateEDU
Agenda
- Introductions (5 mins)
- Review origins and goals of SIG (5 mins)
- Discussion on previously-identified concerns and questions (40 minutes):
- Is there any call for using Windows containers in addition to (but not instead of) Linux containers?
- Corollary: is there demand for Windows-supported container setup for ODS/API for recent previously-released 3.x code?
- Do we need to build in more fault tolerance / circuit breakers to handle the more ephemeral nature of container uptime?
- Should the standard architecture include an external caching service that can be shared across API container instances?
- Should the standard architecture include tools for aggregating and visualizing log data, including custom applications, HTTP, and database?
- Brief description of planned community use cases (20 minutes).
- Next steps (5 minutes).
Meetings Notes
Any missing containers in proposed architecture? Is this a good starting point?
- Consider a blue-green deployment, staging environment that is simpler. → Patrick Yoho particularly concerned.
- Maybe just a single environment for all Ed-Fi platform tools? Start with an MVP?
- Putting together a HELM chart? Maybe that comes later, as likely not needed for an MVP.
- Brief mention of Apache Camel for message-oriented facade in front of the API, no detailed conversation.
- Conclusions
- Go simplest route with Docker Compose first, then look at Kubernetes.
- Make a list of use cases and then prioritize these - that will define an MVP.
Operating system?
- Proposal: to go Linux first.
- Look into musl-based distributions - these have performance advantages https://www.musl-libc.org/.
- Could use Alpine-based distribution
- Any demand for Windows-based?
- Unclear, but probably low - look to community to validate this over time.
- Might be useful for ODS/API 3.3, 3.4, and 5.0.0, which will not work on Linux.
- Conclusions:
- Start with Linux.
- Will consider Windows down the road as there is demand.
Fault tolerance?
- Can be costly.
- Likely won't go into production use initially - may start with ancillary use cases (e.g.vendor integration)
- Mostly this is handed off to coordination tools.
- Conclusion: not needed initially / low investment
External caching service?
- Current ODS architecture not suitable to a shared cache architecture.
- Current cache not heavily used (e.g. descriptors) and would migrate to containers OK.
- Counterpoint: Student IDs are in that cache, so it can get large for large implementations.
- As this cache is in memory, it could get duplicated many times over.
- Current caching implementation does not sync well with updates in the database.
- But a shared / distributed cache should solve that, since all API instances would update the shared cache when updating the database.
- Conclusions:
- There is some need, as larger implementations will see issues and/or need workarounds (e.g. SEA).
- Possible action: estimate externalizing the cache(s) in the ODS/API.
Logging?
- Is this a service needed, or implementation-specific?
- Conclusion: No - logging is likely implementation specific
Focus on ease of injecting the configurations the implementation wants
Use Cases?
- Identified scenarios
- Single-tenant with shared instance.
- Single-tenant with year-specific instances.
- Multi-tenant via new district-specific instances.
- State-based shared instance
- State-based year-specific instance
- Extension support
- With the year-specific and district-specific settings, prefer to have one set of running containers for the installation, rather than a set of separate containers for every district (think about a SaaS provider).
Is Admin App container needed?
- Conclusion: helpful, but not required, platform containers useful without Admin App.
Parking Lot
- Comment: I think one of the most important things for docker-compose or helm solutions is balancing configuration vs simpleness and making sure that documentation is really strong. +1
- Easily switch between internal or external PostgreSQL.
- Probably multiple sample docker-compose files.
- Does anyone need XML support, thus need/want a client-side bulk loader image?
Next Steps
- Update the technical vision with more use case information and map out iterative MVP's for 2020 and beyond. Share with SIG members.
- Further requirements definition will be handled via ad hoc communication, but we will reconvene the SIG if the Alliance or community members see value