Page Comparison

Use this Setup Guide to install and configure your SEA Modernization solution. At the end of this Setup guide you'll have a Pilot ready environment in which you can begin to load Student Information System data into your Ed-Fi Operational Data Store. 

Summary

Solution Overview

In this section you'll get a general understanding of the components of your SEA Modernization solution to provide context for completing the remainder of the Setup Guide. 

The diagram below depicts the general architecture of your SEA Modernization solution. 

• LEA Student Information Systems connect and transmit data into the SEA ODS using the Ed-Fi ODS/API.
• Those connections are managed and secured via the ODS/API Admin App. 
• With data unified in the ODS in Ed-Fi format, multiple avenues for surfacing actionable data become available including the use of BI reporting platforms and data warehousing.  

Once you complete this Setup Guide:

• Data specifications will be published
• Any necessary extensions to the Ed-Fi Model will be created.
• Ed-Fi Solution will be deployed on stage/pilot server.
• Credentials and connections between SIS systems and the ODS / API will be managed using the ODS / API Admin App.
• Agency (Seed) Data will be populated in your Operational Data Store (ODS).
• Data from your Student Information System will flow into your Operational Data Store (ODS) via the ODS / API.
• Validation reporting will be configured.

The core components involved in this solution are:

• Student Information Systems (SIS).
• The Ed-Fi ODS / API & Tools, which include Ed-Fi ODS / API, Ed-Fi Swagger Documentation, Ed-Fi Admin App and MetaEd.
• Validation Reports 
• Membership Count and Special Education Student Count Reports

The rest of this guide will instruct you on:

• Developing data specifications.
• Developing necessary extensions.
• Installing the starter kit components. 
• Connecting district SIS's to the Ed-Fi ODS via the ODS / API.
• Validating the data collected in the Ed-Fi ODS.
• Generating Membership Count and Special Education Student Count Reports.

Step 1: Develop Data Specifications 

To use the Ed-Fi Data Standard, states must re-express, in Ed-Fi language, the data specifications they have provided to their technology providers and school districts.

The data specifications comprise elements that your state needs to collect in order to meet your reporting needs (using the Ed-Fi Operational Data Store (ODS)). 

The elements needed in your specifications depend on your data collection need for reports you intend to service. Defining a data specification generally means mapping between the state elements and those elements in the Ed-Fi Data Model. This mapping exercise will identify where the elements are 1:1 aligned, where they are not 1:1 aligned, what elements need to be omitted, and what elements do not exist in Ed-Fi and need to be included as Ed-Fi Extensions, more on this below. 

The data mapping exercise can be done using our published MappingEdu tool or an Excel spreadsheet. Many agencies are most comfortable with a spreadsheet and want to start there.

The MappingEdu tool has the advantage of allowing for greater collaboration, as data specifications are hosted online and it is a web-based application. Ed-Fi Licensees have access to download and to use the tool. However, to be productive, business analysts first need to learn to use the tool.

State-Specific Code Values

As part of data mapping exercise, you also need to express code sets in Ed-Fi. Ed-Fi code sets are called "Descriptors." These values need to be imported in the Ed-Fi ODS so that technology providers submitting data can reference these values as they submit their data. Note that the ODS / API will reject values if Descriptors are not in the ODS; this is an important safeguard to improve data quality.

The Ed-Fi Data Standard ships with a set of default Descriptors. You are invited to use these values. States' usage of Ed-Fi defined Descriptors has varied.  Some implementations take an adopt-and-extend approach—i.e., the SEA keeps the default Ed-Fi values but adds any additional descriptor values that are missing from the Ed-Fi set. If there are any Ed-Fi values that should not be used, these are excluded by external documentation and downstream validations. In this approach, no Ed-Fi default values are removed. In some implementations, states have replaced the Ed-Fi Descriptors' namespace with their state namespace to have all the Descriptors in their State namespace. We recommend the adopt-and-extend approach. Review Descriptor Guidance for more details. 

Note: When values are added, they must always be added in the SEA namespace to provide a technical means of identifying new values and should be given a definition. 

Step 2: Develop Model Extensions (If Required)

When considering the use cases your agency is pursuing with the Ed-Fi solution, some gaps between the data you need to realize your use case and the data elements that are in Ed-Fi may become apparent during the mapping exercise. To resolve this, the Ed-Fi Data Standard can be "extended" to accommodate these missing entities. The data elements are added to the Ed-Fi Operational Data Store by a business analyst or a developer using the MetaEd tool.

It can be very tempting to ask for aggregate data when extending the Ed-Fi APIs. As an example, if a state needs to know the number of days a student was in attendance for a grading period, why not just extend the model to ask for this aggregate number?

The Ed-Fi model is built on the notion that granular data is inherently more useful and informative, and also on the vision of establishing industry-wide data exchange models where all systems benefit from standardization. Aggregate data makes both more difficult. In the above example, different states may have different ways of counting absences; does a medical release or home-bound day constitute an absence? How about a student early release?

Aggregate fields often require specific business rules that result in complex, state-specific logic falling on the Student Information System providers, and this reduces the reusability of their application code across state boundaries. As a result, the ecosystem does not enjoy the benefits of standardized data exchange, and this means lower data quality for your agency. 

State Specific Extensions - Do's/Don'ts

If you think a missing element represents a common community need (as opposed to only or overly local conditions), please open a ticket on the Ed-Fi Tracker and provide detailed information.

Step 3: Install the Ed-Fi Solution

In this step you will install all of the components of the Ed-Fi solution using the recommended Binary Installation method. The basic list of Infrastructure items that you will need to be successful at this guide are provided below. It is always recommended to follow your institutions best practices for maintenance and backups.

At its most basic level, the Ed-Fi ODS / API platform consists of two logical servers: a database server (for the ODS and related databases) a web server (for the ODS / API and Admin App). Following diagram shows a typical configuration:

Hardware Recommendations

Note: Due to the disk-intensive nature of the Ed-Fi ODS, the ODS Database server is typically much more capable, in terms of memory, disk, and CPU than the ODS / API.

Software Requirements

For more details on setting up required components and software, refer to Ed-Fi ODS/API Installation Instructions.  

Install the ODS/API and Admin App by following the instructions in the Year Specific Installation Steps. Most states choose YearSpecific database partitioning strategy. See Platform Dev Guide - Extensibility & Customization for more on database partitioning strategies. 

Step 4: Setup API Security

In this step you will configure the Admin App to create your first application and generate a Key and Secret for use in securely connecting to your Student Information System. 

Connecting and Securing SIS systems in the Admin App

Now that the Ed-Fi Solution is installed, it is time to complete initial configuration steps in the Admin App so that you can connect and secure your district Student Information Systems. When you start the Admin App you will land on a login page. Since it is the first time in the app, you will need to register an Admin user for the app. Click on the "Register as a new user" button in the bottom right. 

The registration is a simple form with email and password fields. Fill the form and click on "Register."

The Admin App will then inform you that some additional setup steps are required. These are automated steps, so just click "Continue" to proceed.

Once the automated additional process is complete, the Admin App will ask if you'd like to Enable Product Improvement by allowing telemetry data to be sent to Ed-Fi to help with ongoing improvements. If you agree, leave the "Enable Product Improvement" box checked, otherwise deselect it. Click on "Proceed" to continue. 

Setup will require a restart of the ODS / API to have configuration settings take effect. Restart the ODS / API from the IIS. 

Click on the "Proceed" button to continue. 

The Admin App will guide you to register an instance of the ODS. Enter the school year and a description for your ODS as shown below. Click on "Save" to register it. 

After the ODS instance is registered, you will see the instance details as shown below.  The Admin App is now set up and ready for use. Click on the "Home" link at the top right to get started adding your first vendor and application.

Now you'll configure your first Vendor and Application for your first Student Information System.  At the home page of the Admin App you will see the options below.  Select "Global" to proceed to adding a vendor.
 

The Global page provides an overview of the process for adding and securing a vendor in the Admin App. Click on the "Add Vendor" button in the bottom right corner. 

Fill out your vendor information in the Add Vendor page. The Namespace prefix should be a URL which includes the domain of the Student Information System (SIS) you are using. 
Image Modified

Once you hit "Save Changes" on your Vendor details, you should see the new Vendor in the vendor list as shown below. Now that the vendor is created, the application needs to be added.  Click on the "Define Applications" button to enter the Vendor and Application list.

You will see the option to add an application under each Vendor that you've created.  Click on the "Add Application" button to initiate the process. 

The application form sets how a vendor will be used.  Below is an example of setting up a vendor application with a SIS claim set for an LEA. Note: "SIS Vendor" is a preconfigured claim set designed to allow vendors to load all the SIS related information to the ODS / API. Click on "Add Application" to save the application. 

Image Modified

Next, you will be provided a key and secret to be used with importing data. Note: Treat the Key and Secret as secure information for your organization (because it will control student data exchange from SIS systems). Store it according to your local IT Security policy. Often this will be in a password keeper app, on a secure file server, or a similar method. Click on "I have copied the Key and Secret to proceed. 

Image Modified

You will now see the newly created application listed under the vendor as confirmation that you have successfully completed the process. 

Image Modified

Once you create "Key" and "Secret", verify API connectivity by authenticating with Key/Secret pair as described in Authentication. Once you have verified the URL and the Key/Secret pair, it is important to check with your vendor to see if they have a specific process and/or utilities for verifying connectivity to vendor application.

Note that you may add other SIS vendors and other applications to this vendor over time, each have their own "key" and "secret".

Additional useful tools for testing API Connectivity and Functionality

• PostMan
• Swagger
• SQL: Tooling for Verifying API Claimsets 
• API: /oauth/token_info endpoint to review namespace prefix and education organization associations 

Step 5: Load Agency Data

In this step you will load Agency or "Seed" data into your Ed-Fi ODS to prepare for the loading of student data from technology partners.

Seed Data

Before you can receive student information data from your district SIS systems, an initial set of data needs to be loaded  into the Ed-Fi ODS (i.e., "seed data") . This includes Education Organizations (schools, LEAs, SEAs) and Courses. Some states load program information as well. Always follow the data dependency order when you are loading this data into the Ed-Fi ODS. The recommended approaches to load seed data is to use either SQL scripts or the Ed-Fi bulk load utility to process using XML files containing the the education organization and course information.

Understand Dependencies When Loading Data

Before you can load your data, your Ed-Fi ODS / API requires that each data item, whether submitted via API from a SIS system or directly loaded in bulk to the ODS database, must be structurally and relationally consistent and syntactically valid according to the Ed-Fi Data Standard v3.x

Put simply, some data items must be loaded successfully before others can be loaded, and each data item must follow the rules of the standardized vocabulary to load correctly.

You do not, however, need to load everything in order for your SIS systems to get started. Furthermore, acceptable defaults for some needed data are already loaded in your Ed-Fi ODS / API to ease your start. If your SIS is Ed-Fi-certified, it will already have tackled these load order dependencies and other requirements, and you won't have to worry about these details now. However, it is important to know the proper loading order for your data to minimize the frustration of tracking down potential errors during the loading process. 

Provided below is a dependency graph for loading data into the ODS / API. The vertical path represents dependencies that must be satisfied as you move downward, while items in the same horizontal position can be loaded independently or simultaneously.  Please note that although all Ed-Fi data domains are represented in the graph, only Descriptors and Education Organization data is required to "seed" the ODS for further data loading. 

Configure Your Student Information System

Each SIS provider has its own vendor-specific configuration tool and steps to connect to the Ed-Fi API, perform basic data mapping, provide descriptors, and to enable you to begin loading data into your ODS API.

The SIS administrator will connect their SIS to state ODS API using the key and secret you generated in the Admin App. 

• Verify that the SIS supports Ed-Fi data interoperability, and is certified to use the ODS API: Registry of Ed-Fi Certified Products - Student Information Systems API v3 Certification. 
• Verify that the SIS being used is the version that has been certified by the Ed-Fi Alliance or state certification  program.
• If the current version of the SIS is not the certified version, plan an update/upgrade to the certified version of the SIS.
• If the SIS is not listed as an Ed-Fi certified SIS, use the Vendor Request Form to get our help in making the request and conveying your requirements to the vendor.
• Request Ed-Fi enablement and configuration from the SIS vendor's product support team.
• Log into the SIS vendor's customer support portal or knowledge base to find the most current help documentation to guide you in the SIS configuration steps.
• Follow the SIS vendor's configuration guide to configure the product to connect to your ODS API and prepare for loading your data.

See the General Reference page for a list of publicly known vendor configuration guides.

Map SIS Codes to the ODS API

Once the SIS is configured to connect with the Ed-Fi API, the next step to getting data to the ODS API is to map "code sets", or Descriptors, as they are known in the Ed-Fi data standard, from the data elements and codes used in the SIS to those you will use in the Ed-Fi ODS API. To do this: 

• Refer to the SIS vendor configuration guide you used above to locate and use the data mapping tools in the SIS application.
• Proceed with the SIS vendor's data mapping instructions, referring to their configuration guide and the dependency chart provided above to work through the data mapping and loading in the right sequence.

Step 6: Staging and Promotion to Pilot

In this step you will review staging and pilot strategies for your newly created Ed-Fi SEA solution. 

Staging

A staging (or UAT) environment is an important piece of your solution. It provides an end-user testing area that can be populated with Test data (often a backup of production data to provide the most realistic testing), to allow stakeholders and QA personnel the chance to go beyond automated tests in checking for reliability and expected results on your applications.

Pilot Deployment

Promotion to the pilot environment is the final step in your Ed-Fi ODS/API implementation. It can also be complex and sometimes daunting due to the many permutations and configurations that are available. The Production Deployment section of the ODS/API documentation describes the factors that go into planning an Ed-Fi ODS/API deployment, along with guidance on creating a secure deployment.  The production deployment guidelines are also applicable for staging and pilot areas. 

Step 7: Running Validations

In this step you will setup validation process for your Ed-Fi SEA solution. 

L1 & L2 Validations

ODS / API applies certain validations like schema validations (or e.g., referential integrity validations, required field validations, and data type validations) when the data is submitted and rejects invalid data along with validation message before it can be stored in the ODS. These are called Level 1 (L1) validations. Certain other validations cannot be run by the API as they need additional context not available for the API processing. So, you might want to analyze the data your LEA SIS’ submit for accuracy through real-time/nightly/on demand validation cycles where data is processed through your custom business rules and errors are logged. The second round of validations that happen during validation cycles are called Level 2 (L2) validations. Depending on the quality of the data, validation messages can be generated to give your LEA feedback (typically via a state error portal) on data that needs review and/or correction. The LEA then fixes the errors, the data is re-transmitted to the API (generally in near-real-time) and the data quality improves.

Typically, SEAs periodically move data from the Ed-Fi ODS to an Ed-Fi ODS that is multiyear. This environment generally has some additional columns to allow for multiple years of data to be compared. In this context, you can run validation rules that apply to multiple years of data and report those validation errors back to LEA as well. e.g. Student In Grades K-8 Without Enrollment Record Exit Code Who Do Not Appear In Following School Year