Vision

The Fix-it-Friday (FIF) project is designed to be a rapid response to the changing educational conditions due to COVID-19. The primary product offering is an end-user application for teachers and administrators to review student rosters and drill into contact information, student survey responses, and potentially other data sourced from educational data systems.

Architectural Requirements

The project is loosely constrained, compared to traditional Ed-Fi software development. For the solution architecture, there are only a few must have requirement:

1. This is an open source project, and the architectural choices must reflect that by being attractive to software developers beyond the Ed-Fi staff and contractors.
2. The project must support student data privacy as defined by FERPA by limiting user access to the appropriate scope of information for the user's role in the education organization.

The architecture should:

1. Be flexible with respect to source systems of record.
2. Use open source components and technologies for low operational costs.
3. Be composed of loosely-coupled systems to allow for rapid evolution.
4. Support a location-agnostic deployment model (e.g. on-premises or on any major cloud provider).
5. Provide a user experience that works on screen sizes ranging from phone to full-size monitor or projector.
6. Provide scripted configuration options for rapid deployment by individual schools, local education agencies, or charter management organizations.

Design Patterns and Technology Choices

In service to the vision and requirements described above, the Fix-it-Friday development team is proceeding with the following design patterns and technology choices.

Deployment Model

• Microservices: compose overall application using independent components
  • User interface for teachers and administrators
    • Same application for configuration? ETL? To be determined
  • One or more HTTP-based API applications
  • One or more ETL services for loading source system data into a dedicated database.

User Interface

• React: The user interface application will utilize the React framework and be written in Typescript, with a Node.js web server
• TypeScript: The API application(s) will be written in Typescript using a Node.js web server, with Node modules used as needed to provide the required functionality.
• Device and Browser Independence
  • Bootstrap will provide scaling to different viewport sizes including Phone, Tablet, or full-size monitor. 
  • The interface will target "modern" browsers, e.g. reasonably up-to-date versions of Chrome, Firefox, Edge, or Safari
  • Development effort will rely heavily on interoperable standards that should work, but the Ed-Fi Alliance does not have the capacity for thorough testing on all of these browsers and the matrix of platforms and devices. Will make heavy use of browser-based tools for simulating viewport sizes.

Web-based API

• GraphQL: The primary API supporting the user interface will provision data through a GraphQL interface rather than a REST interface, optimizing for both lower network traffic between client and server, and greater agility in the API interface and data modeling processes.
• Security: While many security details are yet to be worked out, initially the team plans to support OAuth2 and claim-based authorization using JSON Web Tokens (JWT). Integration with third party OAuth2 providers will be pursued. Other authentication techniques have not been completely ruled out (e.g. SAML).

Data Storage

• PostgreSQL: Data storage will likely rely on a hybrid relational / document model using PostgreSQL, with rapid development and query optimization more important than normalization and disk usage.
• Row-level Security: It is assumed that row-level data access will be provisioned based on the user's allowed scope of access, e.g. a teacher allowed to access data for students in their classroom, a principal allowed to access data for students in their school, etc. The data storage model must provide sufficient support so that queries only return the required data to the API application(s).

Data Loading

• ETL To-be-determined: distinct ETL services can be built for different scenarios, e.g. pull from the ED-Fi ODS/API (via Analytics Middle Tier?), source data from CSV files extracted from source systems, etc. Tech-stack to be used is yet to be determined, and it could rely on dedicated ETL to tools or custom code written in Typescript, Python, or another language.
• Asychronous Job Scheduling: use case discovery will determine if unattended ETL is needed. Application support will likely need to support manual management, e.g. a school administrator manually uploading CSV files. In either case, asynchronous job-scheduling service will likely be needed for queuing and processing work. Will investigate best-of-breed open solutions, such as Celery. For these needs would prefer a system that uses PostgreSQL as the queuing database rather than needing to install something like Redis or RabbitMQ.
• Patterns: the development team will create create clear standards, instructions, and/or hooks to help other teams develop specific ETL solutions.

Diagrams

While PostgreSQL is depicted as a container below, the system could also be deployed using a managed PostgreSQL service such as Amazon RDS. Ideally, the authentication process will be seamless for a user who has already authenticated with the Identity Provider before visiting the application. An additional step may be required for the browser application to generate a JSON Web Token with necessary claim information in it once the user has been authenticated.

The source system(s) could be the system of record, the Ed-Fi ODS, or any system that can make data available.

Deployment Considerations

Project Buzz is currently deployed to Ed-Fi's demo site and to early piloteers using Windows Server.  The static UI is deployed through IIS and the API and other services are Windows Services.  This deployment model reduced the complexity for early installations.  The software artifacts should support a dockerized deployment method if you want.

• Docker Containers will be the ultimate goal for delivery and deployment, with Kubernetes orchestration. Short term will use PowerShell scripts for Windows on-premises deployment.