SEA Modernization Setup Guide

Use this Setup Guide to install and configure your SEA Modernization solution. At the end of this Setup guide you'll have a Production ready environment in which you can begin to load Student Information System data into your Ed-Fi Operational Data Store. 

Summary

Solution Overview

In this section you'll get a general understanding of the components of your SEA Modernization solution to provide context for completing the remainder of the Setup Guide. 

The diagram below depicts the general architecture of your SEA Modernization solution. 

• LEA Student Information Systems connect and transmit data into the SEA ODS using the Ed-Fi ODS/API.
• Those connections are managed and secured via the ODS/API Admin App. 
• With data unified in the ODS in Ed-Fi format, multiple avenues for surfacing actionable data become available including the use of BI reporting platforms and data warehousing.  

Once you complete this Setup Guide:

• Data specifications will be published
• Any necessary extensions to the Ed-Fi Model will be created.
• Ed-Fi Solution will be deployed on stage/production server.
• Credentials and connections between SIS systems and the ODS / API will be managed using the ODS / API Admin App.
• Agency (Seed) Data will be populated in your Operational Data Store (ODS).
• Data from your Student Information System will flow into your Operational Data Store (ODS) via the ODS / API.
• Validation reporting will be configured.

The core components involved in this solution are:

• Student Information Systems (SIS).
• The Ed-Fi ODS / API & Tools, which include Ed-Fi ODS / API, Ed-Fi Swagger Documentation, Ed-Fi Admin App and MetaEd.
• Validation Reports 
• Membership Count and Special Education Student Count Reports

The rest of this guide will instruct you on:

• Developing data specifications.
• Developing necessary extensions.
• Installing the starter kit components. 
• Connecting district SIS's to the Ed-Fi ODS via the ODS / API.
• Validating the data collected in the Ed-Fi ODS.
• Generating Membership Count and Special Education Student Count Reports.

Step 1: Develop Data Specifications 

To use the Ed-Fi Data Standard, states must re-express, in Ed-Fi language, the data specifications they have provided to their technology providers and school districts.

The data specifications comprise elements that your state needs to collect in order to meet your reporting needs (using the Ed-Fi Operational Data Store (ODS)). 

The elements needed in your specifications depend on your data collection need for reports you intend to service. Defining a data specification generally means mapping between the state elements and those elements in the Ed-Fi Data Model. This mapping exercise will identify where the elements are 1:1 aligned, where they are not 1:1 aligned, what elements need to be omitted, and what elements do not exist in Ed-Fi and need to be included as Ed-Fi Extensions, more on this below. 

The data mapping exercise can be done using our published MappingEdu tool or an Excel spreadsheet. Many agencies are most comfortable with a spreadsheet and want to start there.

The MappingEdu tool has the advantage of allowing for greater collaboration, as data specifications are hosted online and it is a web-based application. Ed-Fi Licensees have access to download and to use the tool. However, to be productive, business analysts first need to learn to use the tool.

Step 2: Develop Model Extensions (If Required)

When considering the use cases your agency is pursuing with the Ed-Fi solution, some gaps between the data you need to realize your use case and the data elements that are in Ed-Fi may become apparent. To resolve this, the Ed-Fi Data Standard can be "extended" to accommodate these missing entities. The data elements are added to the Ed-Fi Operational Data Store by a business analyst or a developer using the MetaEd tool.

It can be very tempting to ask for aggregate data when extending the Ed-Fi APIs. As an example, if a state needs to know the number of days a student was in attendance for a grading period, why not just extend the model to ask for this aggregate number?

The Ed-Fi model is built on the notion that granular data is inherently more useful and informative, and also on the vision of establishing industry-wide data exchange models where all systems benefit from standardization. Aggregate data makes both more difficult. In the above example, different states may have different ways of counting absences; does a medical release or home-bound day constitute an absence? How about a student early release?

Aggregate fields often require specific business rules that result in complex, state-specific logic falling on the Student Information System providers, and this reduces the reusability of their application code across state boundaries. As a result, the ecosystem does not enjoy the benefits of standardized data exchange, and this means lower data quality for your agency. 

If you think a missing element represents a common community need (as opposed to only or overly local conditions), please open a ticket on the Ed-Fi Tracker and provide detailed information.

Step 3: Install the Ed-Fi Solution

Install the Ed-Fi Solution

In this step you will install all of the components of the Ed-Fi solution using the recommended Binary Installation method. The basic list of Infrastructure items that you will need to be successful at this guide are provided below. It is always recommended to follow your institutions best practices for maintenance and backups.

At its most basic level, the Ed-Fi ODS / API platform consists of two logical servers: a database server (for the ODS and related databases) a web server (for the ODS / API and Admin App). Following diagram shows a typical configuration:

Hardware Recommendations

Note: Due to the disk-intensive nature of the Ed-Fi ODS, the ODS Database server is typically much more capable, in terms of memory, disk, and CPU than the ODS / API.

Software Requirements

For more details on setting up required components and software, refer to Ed-Fi ODS/API Installation Instructions.  

Install the ODS/API and Admin App by following the instructions in the Shared Instance Installation Steps. Most states choose YearSpecific database partitioning strategy, instead of SharedInstance. See Platform Dev Guide - Extensibility & Customization for more on database partitioning strategies. 

Step 4: Setup API Security

In this step you will configure the Admin App to create your first application and generate a Key and Secret for use in securely connecting to your Student Information System. 

Key and Secret

Once you have successfully installed the Admin App following the instructions provided above you can continue to connect new LEA SIS systems to your Ed-Fi platform. The process is outlined in the "Connecting and Securing SIS systems in the Admin App" section below.  In order to set up a connection with a SIS or other vendor system, you must generate a key and secret with the correct claim sets for the data that the source system will be providing. You should review the domains that your source system is sending and verify that the key and secret have the correct permissions. Ed-Fi ODS platform ships with 'SIS Vendor' claim set which is sufficient for most SIS integrations, however Admin App comes with /wiki/spaces/ADMIN/pages/25238340 for fine-grained control over permissions if default claim sets that ships with Ed-Fi ODS platform are not sufficient for you needs. 

Connecting and Securing SIS systems in the Admin App

1. Follow the Admin App - Getting Started instructions to complete its initial configuration steps.
2. Follow the Admin App - Next Steps to configure your first Vendor and Application entry for your Student Information System. 
3. Once you create an Application in the Admin App, you will store and use the generated "Key" and "Secret" in the next section.
   1. Note: Treat the Key and Secret as secure information for your organization (because it will control student data exchange from SIS systems).
   2. Store it according to your local IT Security policy. Often this will be in a password keeper app, or on a secure file server, or similar method (check with your IT department).
4. Once you create "Key" and "Secret", verify API connectivity by authenticating with Key/Secret pair as described in Authentication. Once you have verified the URL and the Key/Secret pair, it is important to check with your vendor to see if they have a specific process and/or utilities for verifying connectivity to vendor application.
5. Note that other applications you may add over time, for example Assessment applications used in other starter kits, will each have their own "key" and "secret".

Additional useful tools for testing API Connectivity and Functionality

• PostMan
• Swagger
• SQL: Tooling for Verifying API Claimsets 
• API: /oauth/token_info endpoint to review namespace prefix and education organization associations 

Step 5: Load Agency Data

In this step you will load Agency or "Seed" data into your Ed-FI ODS to prepare for the loading of student data from technology partners. 

Seed Data

Before you can receive student information data from your district SIS systems, an initial set of data needs to be loaded  into the Ed-Fi ODS (i.e., "seed data") . This includes Education Organizations (schools, LEAs, SEAs) and Courses. Some states load program information as well. Always follow the data dependency order when you are loading this data into the Ed-Fi ODS. The recommended approaches to load seed data is to use either SQL scripts or the Ed-Fi bulk load utility to process using XML files containing the the education organization and course information.

Understand Dependencies When Loading Data

Before you can load your production or test data, your Ed-Fi ODS / API requires that each data item, whether submitted via API from a SIS system or directly loaded in bulk to the ODS database, must be structurally and relationally consistent and syntactically valid according to the Ed-Fi Data Standard v3.x

Put simply, some data items must be loaded successfully before others can be loaded, and each data item must follow the rules of the standardized vocabulary to load correctly.

You do not, however, need to load everything in order for your SIS systems to get started. Furthermore, acceptable defaults for some needed data are already loaded in your Ed-Fi ODS / API to ease your start. If your SIS is Ed-Fi-certified, it will already have tackled these load order dependencies and other requirements, and you won't have to worry about these details now. However, it is important to know the proper loading order for your data to minimize the frustration of tracking down potential errors during the loading process. 

Provided below is a dependency graph for loading data into the ODS / API. The vertical path represents dependencies that must be satisfied as you move downward, while items in the same horizontal position can be loaded independently or simultaneously.  Please note that although all Ed-Fi data domains are represented in the graph, only Descriptors and Education Organization data is required to "seed" the ODS for further data loading. 

Configure Your Student Information System

Each SIS provider has its own vendor-specific configuration tool and steps to connect to the Ed-Fi API, perform basic data mapping, provide descriptors, and to enable you to begin loading data into your ODS API.

The SIS administrator will connect their SIS to state ODS API using the key and secret you generated in the Admin App. 

• Verify that the SIS supports Ed-Fi data interoperability, and is certified to use the ODS API: Registry of Ed-Fi Certified Products - Student Information Systems API v3 Certification. 
• Verify that the SIS being used is the version that has been certified by the Ed-Fi Alliance or state certification  program.
• If the current version of the SIS is not the certified version, plan an update/upgrade to the certified version of the SIS.
• If the SIS is not listed as an Ed-Fi certified SIS, use the Vendor Request Form to get our help in making the request and conveying your requirements to the vendor.
• Request Ed-Fi enablement and configuration from the SIS vendor's product support team.
• Log into the SIS vendor's customer support portal or knowledge base to find the most current help documentation to guide you in the SIS configuration steps.
• Follow the SIS vendor's configuration guide to configure the product to connect to your ODS API and prepare for loading your data.

See the General Reference page for a list of publicly known vendor configuration guides.

Map SIS Codes to the ODS API

Once the SIS is configured to connect with the Ed-Fi API, the next step to getting data to the ODS API is to map "code sets", or Descriptors, as they are known in the Ed-Fi data standard, from the data elements and codes used in the SIS to those you will use in the Ed-Fi ODS API. To do this: 

• Refer to the SIS vendor configuration guide you used above to locate and use the data mapping tools in the SIS application.
• Proceed with the SIS vendor's data mapping instructions, referring to their configuration guide and the dependency chart provided above to work through the data mapping and loading in the right sequence.

Step 6: Staging and Promotion to Production

Staging

A staging (or UAT) environment is an important piece of your solution. It provides an end-user testing area that can be populated with Test data (often a backup of production data to provide the most realistic testing), to allow stakeholders and QA personnel the chance to go beyond automated tests in checking for reliability and expected results on your applications.

Production Deployment

Promotion to the production environment is the final step in your Ed-Fi ODS/API implementation. It can also be complex and sometimes daunting due to the many permutations and configurations that are available. The Production Deployment section of the ODS/API documentation describes the factors that go into planning an Ed-Fi ODS/API deployment, along with guidance on creating a secure deployment.  The production deployment guidelines are also applicable for staging areas. 

Step 7: Running Validation Reports

Placeholder

Validation report process details here