Use this Setup Guide to install and configure your Program Diversity and Persistence solution using test data. This guide will walk you through setting up a test environment that closely mimics a staging or production environment. At the end of this Setup Guide you'll have a test environment with your own test data loaded, but to cover all required elements (credentials, candidates, and financial aid/grants) for the Program Diversity and Persistence solution additional steps will be required as indicated at the end of Step 3.
This test environment allows your tech team, administrators, and stakeholders to test configuration changes being considered for the production environment without an impact on end-users. It is important to note that this test environment is separate and distinct from the sample environment you accessed during the Quick Start.
This Setup Guide assumes that you have access to a the following data (in CSV format), which provides the data to power the dashboard:
- Student Information System data
- Candidates that have received credentials from the state or other licensing body
- Data on Candidates that have received grants
The following steps to get this test environment up and running are very similar to the steps for setting up a production or staging environment.
Summary
- Step 1. Install the Ed-Fi Solution
- Step 2. Configure your Education Organizations and Applications
- Step 3. Configure Data Import and Load Data
- Step 4. Install and Test the Program Diversity and Persistence Dashboard
- Step 5. Publish the Program Diversity and Persistence Dashboard and Plan Your Rollout
More information
This Setup Guide builds on core skills and concepts in the Program Diversity and Persistence Quick Start. We recommend running that Quick Start before moving on to this more detailed Setup Guide.
Step 1. Install the Ed-Fi Solution
In this step, you will install the components of the Ed-Fi solution needed for the starter kit. These components are:
- ODS / API for Suite 3 v6.1
- Analytics Middle Tier v3.1
- Admin App for Suite 3 v3.1
- Data Import v2.0
Basic Infrastructure and Prerequisites
The list of infrastructure items you need to be successful is provided below.
- Database Server (Windows Server 2019) with the following components installed:
PowerShell 5.0.
.NET Core 6.0 SDK
Microsoft SQL Server 2019 Standard (or higher)
- Web Server (Windows Server 2019) with the following components installed:
.NET Core 6.0 Hosting Bundle. The .NET Core 3.1 Hosting Bundle is required for running the API on IIS.
Internet Information Services. IIS is the web server that will run the ODS / API.
For more details on setting up required components and software, refer to Getting Started - Binary Installation. It is always recommended to follow your institution's best practices for maintenance and backups.
Install Ed-Fi ODS/API
Install the ODS / API for Suite 3 v6.1 by following instructions in the Getting Started - Binary Installation guide
Install Analytics Middle Tier
Install Analytics Middle Tier by following the instructions in the AMT Deployment Guide to install the 'EPP' and 'RLS' collections
Install Admin App
Install Instructions for Admin App can be found in the Admin App v3.1 guide.
Install Data Import
Install Instructions for Data Import can be found in the PowerShell Installation for Data Import using NuGet Packages guide. Data Import configuration is also described below.
Time to complete
Installing the Ed-Fi Solution usually takes a few hours to a day, assuming you have access to the hardware and software prerequisites, and are comfortable working with Windows Server, IIS, and SQL Server.
The time to configure and coordinate data flowing from a certified SIS system typically takes anywhere from a couple of hours to a week or two of calendar time, depending on the type of SIS.
Step 2. Configure your Education Organizations and Applications
In this step, you will load data from your SIS, credential, and financial aid providers to the Ed-Fi ODS / API for it to be available on the Diversity and Persistence dashboard. This step requires you to have access to data in CSV format for Data Import to load into the Ed-Fi ODS / API. If you would like to see sample data import templates and CSV data it can be found on our GitHub here.
Add Your Education Organizations in the Admin App
Once you have successfully installed the Admin App following the instructions provided above, you can continue to configure Admin App to add your Education Organizations. This configuration will allow your data to be loaded properly as much of it requires education organizations. You can also find instructions for AdminApp configuration at Admin App - Next Steps.
The first time you open Admin App will ask you to register a new user in order to get access to the utility. The user you set up can be used for any future access to the tool. You will also have the opportunity to create additional users later.
In order to register a new user you will need to provide an email address and create a new password.
After this initial one-time set up you will see a screen like the following on your subsequent visits to Admin App. You can start by clicking on the "Settings" button within Admin App to configure your Education Organizations.
After you fill out the information click 'Add'. You will be taken back to the Education Organizations tab and should see the Post-Secondary Institution you just added. Now click on the 'Add School' button on the lower right under your Post-Secondary Institution.
A new popup will show that looks very similar to the one you used to create the Post-Secondary Institution. Fill in this form with a School ID that is different from the Post-Secondary ID from that last step. At least one grade level from the drop down. As well as the Name of your EPP and Address information for your EPP. Click the 'Add' button when your done.
You will be returned to the Education Organizations tab, you should see two education organizations, if you do click the 'Home' button at the top right side of the page
Click on the "Global" button within Admin App to configure the vendor and Application that will provide API keys and access for Data Import to load your data.
Click on the "Add Vendor" button within the Vendors tab to add the vendor you will be using with Data Import.
The Namespace prefix should be a URL which includes your institution's web domain (e.g., uri://www.demo.edu)
After you fill out the information and click on "Save Changes", you should see the new entry at the bottom of the screen.
Now that the vendor is created, the application needs to be added. Click on the "Define Applications" button to enter the Application menu. Click on Add Application
The application form will describe how the vendor will be used. Here is an example of how to fill out the Education Organization and Claim Set information.
Note: For test environments using the Ed-Fi Sandbox claim set is appropriate, bur if this is a production environment we suggest using a claim set tailored specifically to this starter kit.
The next screen will provide you a key and secret to be used with importing data. These values will be required in your data import configuration. Make sure to save this information off somewhere for safekeeping.
Finally, the information of the added application is displayed.
- Once you create an Application in the Admin App, you will store and use the generated "Key" and "Secret" in the next section.
- Note: Treat the Key and Secret as secure information for your organization (because it will control API access to data in the Ed-Fi ODS).
- Store it according to your local IT Security policy. Often this will be in a password keeper app, on a secure file server, or a similar method.
Need help?
Having difficulty with your Program Diversity and Persistence Starter Kit? Reach out to an Ed-Fi Alliance representative for assistance.
- Ed-Fi Slack channel #starter-kits
- Open a ticket in Ed-Fi Tracker
Step 3. Configure Data Import and Load Data
The Data Import tool is used for loading export files (typically in CSV format) generated manually, by the EPP, or one of their automated services. There are 5 steps to load data to the ODS:
- Configure Data Import for First Time Use
- Create your map
- Create an Import Agent
- Upload the file
- Execute the Data Import
The initial setup of data import will only need to happen once per server that Data Import is installed on.
The remaining steps will need to be repeated for each CSV you plan to upload. We have provided an example below using program data.
Configure Data Import for First Time Use
The first time you log in to you Data Import application you will see a log in screen. You will need to create a user for this site by clicking on the register link.
Enter an email address and password to get access to data import.
Now you can log in to the Data Import application. Under the Admin tab, select the "Configuration" menu item. This is where you will configure Data Import to access the ODS / API.
Click on the "Ed-Fi API Connections" button to configure the connection to the API.
The next step is to add an API connection which you can do by clicking on the "Add API Connection" button.
Enter the API data and then the key and secret generated in the previous Admin App configuration step. Test the connection and confirm appropriate credentials. Click on Save Changes. The API Connection Name should match the name used in Admin App.
The new connection should appear in the API Connections list.
Go back to the Configuration menu item in the Admin dropdown to configure Admin App for template sharing service. Template sharing is a way to re-use existing assessment vendor templates. In the configuration page, click on Authorize to add the API key and secret. You can request a key and secret here if you do not already have one.
Enter the provided key and secret and confirm by clicking on "Test Template Sharing Service Connection".
Finally, click on the 'Update Configuration' button to save your changes.
Data Import is now configured and you are ready to create or import a template to load data.
Create or Import a Mapping Template
If you already have a import map created select the 'Import / Export' option from the Admin menu.
On the Import / Export Templates page, select 'Choose File', this will open a file picker dialog where you can choose the json template to import.
To create a new map for importing data select 'Maps' from the configuration menu.
Click 'Add Data Map'
Fill in the Map Name, for this example we will be using program data, so we will name the map 'EPP Programs'. Select '5.2' from the API Version drop down. Next choose the API resource that you would like data import to import into. Since we're importing program data, we'll select Programs.
Finally, click on the 'Choose File' button and select your CSV program data.
Click 'Upload'
Data Import will read the file looking at the first row for the column names, it will then show you those column names and the first five rows of data in the CSV.
You can now map columns in the CSV to fields in the Programs API resource.
Once you have completed the mapping, click 'Save Map'
This will bring you back to the Maps page. You should now see your new map in the map list.
Create an Import Agent
With the map created, Data Import needs an agent to execute the mapping against data. Select 'Agents' from the Configure menu.
Click 'Add Agent'
Give the agent a name, this should be something that represents the data being imported, for this example we're using program, so select your API connection created previously. Data Import had a number of different ways to import data, including over FTP or SFTP as well as from the file system.
We're choosing to use a manual template. Make sure your agent is enabled and choose the map created in the previous step. Make sure to click 'Add Map' after you've selected your map from the dropdown.
Confirm that the map is now listed above the drop down as in the screenshot below.
Click 'Save Agent.' Your new agent will show in the list of agents.
Upload a file
Click on the upload icon to the right of the agent
Click 'Choose a File'
Select the program data file and click 'Upload
You will be taken to the activity page that will show your agent and if the file was able to be uploaded and read. The status should be Uploaded. You can now tell Data Import to import your data into the Ed-Fi ODS.
Execute Data Import
When the configuration is completed and the data files are loaded, import the data in: c:\inetpub\wwwroot\Ed-Fi\Data Import\DataImport.Server.TransformLoad
Run the DataImport.Server.TransformLoad.exe executable
A successful import will look similar to this, lots of scrolling text, but no red errors
To import the data necessary for the Program Diversity and Persistence starter kit, you will need to upload data for people, candidates, credentials, programs, and financial aid. This will require repeating Section 3, beginning with Create or Import a Mapping Template for each of the remaining data sources until all of the required data is loaded.
Deep dive
For detailed information on the data requirements to be able to power the Program Diversity and Persistence Dashboard, see the data loading requirements linked below.
Step 4. Install and Test the Program Diversity and Persistence Dashboard
In this step, you'll install Power BI prerequisites, as well as download and run the Program Diversity and Persistence Dashboards.
Prerequisites
- At this point, you should have an operational Ed-Fi ODS / API with data loaded through Data Import.
- Additionally, you should have installed the AMT views on your ODS database.
- You will also need a machine that has access to the ODS database and has Power BI Desktop installed.
Install Power BI Desktop
The Program Diversity and Persistence Dashboard was built with Microsoft's Power BI Desktop application.
To run the dashboard you will need a machine with:
- Access to the database server where you installed the Ed-Fi ODS database
- Power BI Desktop that can be downloaded at this link: https://powerbi.microsoft.com/en-us/downloads/
Download and Configure the Program Diversity and Persistence Dashboard
The latest version of the Program Diversity and Persistence Power BI Dashboard file is located in a GitHub repository.
Use the machine that has Power BI and create the following folder structure: C:\Ed-Fi\QuickStarts\EPP_Diversity\.
Save this file onto that location by clicking this link: Diversity and Persistence Power Bi Report .
Once the download completes, navigate to the folder where you downloaded the file and double-click it. This should open Power BI and load the Program Diversity and Persistence Dashboard as depicted below:
Update Groups Used in the Dashboard
The current Diversity and Persistence Dashboard groups Programs into grade levels and Financial Aid into Aid Type. The program groups are All Level, Elementary, Middle School, and High School. The financial aid groups are Grants, Loans, Work Study, and Other. In this step, you'll update both groups to better fit your organization.
Update Program Groups
- In the report view in Power BI Desktop, expand the Candidate table in the Data pane.
- Click on the ellipsis (...) next to the field ProgramName (groups) in the Candidate table and select Edit Groups.
- A dialogue box appears that allows editing groups.
- Expand the groups on the right to see the programs in each group. You can edit the groupings by selecting or multi-selecting (CRTL+Select) programs and then clicking Group or Ungroup. You can also edit the names of groups by double-clicking on group names.
- Once you are done editing groups, click the OK button to save your settings.
Update Financial Aid Groups
- In the report view in Power BI Desktop, expand the Financial Aid table in the Data pane.
- Click on the ellipsis (...) next to the field AidType (groups) in the Financial Aid table and select Edit Groups.
- A dialogue box appears that allows editing groups.
- Expand the groups on the right to see the Aid Type in each group. You can edit the groupings by selecting or multi-selecting (CRTL+Select) Aid Type and then clicking Group or Ungroup. You can also edit the names of groups by double-clicking on group names.
- Once you are done editing groups, click the OK button to save your settings.
Connect your Power BI Dashboard to your Ed-Fi ODS
The data that is pre-populated is a set of data that mimics real world data in a educator preparation program.
To connect to your Ed-Fi ODS, use the top menu and navigate to Home and then Transform Data. Click the dropdown arrow and select "Data source settings".
When the modal opens click on the "Change Source..." button.
A second modal will popup with inputs to configure your SQL Server server connection. Use the IP address or the hostname of the server that contains the database. Then input the name of your Ed-Fi ODS database. Usually the name of your database will be EdFi_ODS.
Depending on your organization's security configuration, you may use integrated or SQL server security.
Once you click ‘OK’, you will see SQL statements. Click ‘OK or ‘Run’
Additionally, it might show you the following screen:
This is where you can switch from Windows Integrated Security to MsSQL security if you desire.
Proceed to click on connect and accept the defaults.
You will see a screen that will load data from your ODS like the one below. Note that it might take a minute or two to finish loading. This depends on the amount of data and students you have in your Ed-Fi ODS.
Once it loads, the Program Diversity and Persistence Dashboard will display your data.
If you haven't already, visit the Program Diversity and Persistence Dashboard Walkthrough to learn more about how to use the Program Diversity and Persistence Dashboard.
Step 5. Publish the Program Diversity and Persistence Dashboard and Plan Your Rollout
The final step is to publish the Program Diversity and Persistence Dashboards to Power BI Online. This release is targeted to your internal test users, allowing them to log in, view, use, and test the data visualizations. At the end of this step, you will be able to plan your rollout and deployment.
Basic Power BI Online Setup
We will guide you through some of the basics around Power BI Online.
For this dashboard to work you will need:
- A subscription to Power BI Online
- A few users created in Office 365
- A Workspace to be able to publish the dashboard
Creating Users in Power BI Online
If you already have users created in your Power BI account you can skip to the next section.
To create users follow these simple steps:
- Login to Power BI Online
- Click on the
menu on the top left of the screen and select 
as depicted below.
- This will open the 365 admin center. From here click on
button indicated on the image below.
- This will open a modal window that will allow you to add a user. Follow the instruction on that modal to create a user.
- It is important to note that the user's email address in Power BI should be the same as the ones in you SIS and in the Ed-Fi ODS. This is how we match the user that is logged in with a role in the Ed-Fi ODS.
- You can repeat this process to add all the users that will be accessing the Program Diversity and Persistence Dashboard on Power BI
Creating a Workspace to Hold the Dashboard
Back on the main screen of Power BI Online, click on the menu icon and then on the "Workspaces" subsection, which expands a side modal. At the bottom, click on the "Create a workspace" button to toggle a slide-out form on the right. This slide out form allows you to create and name the workspace. For this demo we named it "Ed-Fi EPP Starter Kits".
At this point you are done with this step.
Publishing the Dashboard
Open Power BI Desktop and Sign in with the account that has access to publish to Power BI online.
Proceed to locate and click on the "Home → Publish" menu entry as depicted below.
This will open a modal popup window. Select the workspace you created in the section above and click on the "Select" button.
In a few seconds you will see the modal popup indicating that it is publishing. When complete you will see a "Success!" message.
To confirm, we recommend that you login to Power BI Online and ensure you can see the Dashboard.
To do this, open your favorite browser and navigate to https://powerbi.microsoft.com/ and login.
Click on your workspace and you should see the "Program Diversity and Persistence Dashboard" in the list.
Enabling Security on the Report
The report comes configured with two security roles:
- Supervisor - Has full access to all data
- Standard - This role is meant to restrict access to Personally Identifiable Information and does not have access to the details page of the starter kit.
To enable it in Power BI Online, follow these steps:
Open your browser and navigate to Power BI Online. Then proceed to open the workspace where you published the "Program Diversity and Persistence Dashboard" report. Ensure that the "All" submenu has been selected, and click on the context menu for the row showing the "Dataset". In the context menu select the security option.
In the Row-Level Security page, ensure you click either 'Standard' or 'Supervisor'. Then add either a role or an individual user who will be able to view the report.
Click on the "Save" button once you are done adding groups or users to the members section .
To test the security, hover over the "Standard" or 'Supervisor' role and click on the context menu then select "Test as role."
This will load the report and you should see the widgets show blank data as the user you are using will most likely not be part of the security.
On the top blue menu, click on "Now viewing as: " and enter the email of a previously configured user. In this case, we are going to use Sandra Atkins, who is a Faculty member. This is our sample data set. The report will show data specific to that user profile.
Explore by impersonating other users.
Here is a screenshot impersonating MaryAnn Flores who is a Dean.
When an actual user logs in, the data will be filtered specifically to that user’s profile.
Additional Recommended Reading and Live Dataset Connectivity
We recommend reading the following articles. They explain how to setup a live database connection from Power BI Online to your Ed-Fi ODS SQL Server database.
Note
Regarding identity mapping, it is important to note that a user's email address in Power BI Online should match the email in your SIS and in the Ed-Fi ODS. This is how the Dashboard matches the user that is logged in to a role in the Ed-Fi ODS.