Introduction
This document describes the customer success team’s experiences installing the Ed-Fi platform in Azure. Based on what we have seen community members being most interested in, these instructions detail an implementation that uses the virtualized Microsoft Sql Server databases and virtualized web applications running as App Services on a Microsoft platform. These are NOT necessarily the recommendations of the Ed-Fi Alliance, this is just the path we have tested.
...
This guide does not get into automated implementation strategies or advanced security configuration. Users should expect to work with a qualified cloud architect to figure out what sort of strategies make sense for their organization, their skill set, their budget, and their objectives.
Prerequisites
The following prerequisites should be addressed before starting the Azure deployment.
...
Expand |
---|
- Log into the Azure portal and search for the SQL Servers option
- Select Create
- Select a previously created Subscription and Resource Group
- Set the name for the Server and its location
- Set Authentication method to either "Use SQL authentication" or "Use both SQL and Azure AD authentication"
- NOTE: SQL Server Authentication is the only authentication method needed, but Azure AD authentication provides a more secure way to externally connecting to the databases.
- Set the SQL Server admin login and Password
- Go to Next: Networking
- Set Allow Azure services and resources to access this server to Yes
- Click Next: Additional settings
- Set Enable Microsoft Defender for SQL to Not now
- Click Next: Tags
- Set any desired Tag name and its values
- Review the generated configuration
- Click on Create
- You should get a notification regarding your ongoing resource deployment, as well as a redirect to the Deployment Status page
- After the deployment is successful, select "Go to resource" or navigate manually to the created SQL server
|
Step 2. Deploy local databases to Azure SQL
In this section we will be deploying the local databases up to the Azure SQL Server we just set up.
...
Expand |
---|
We have two options for deploying databases, either the databases can be pushed to an Azure storage account from SSMS or they can be published directly from SSMS Deck |
---|
id | deploy local databases options |
---|
|
Card |
---|
id | Deploy via Storage accounts |
---|
label | Deploy via Storage accounts |
---|
|
- Under the server object > "Databases", right-click any of the local databases. Select "Tasks" > "Export Data-Tier Application"
- Click on "Next" on "Introduction"
- On "Export Settings", under "Settings", click on "Save to Microsoft Azure"
- Click on "Connect", and "Sign in..."
- Sign in to your Azure account
- Select the appropriate Storage account and Blob Container to store the backup file. Click "OK"
- After returning to the export wizard, corroborate the information provided and click "Next"
- Corroborate the information and click "Finish"
- On your Azure portal page, navigate to the SQL Server created in Step 1
- On the "Overview" section of the resource page, click on "+ Import Database"
- Fill the requirements with the corresponding information
- Database name: type the same database name it had from the backup or leave blank
- Collation: leave as is
- Authentication type: SQL Server
- Server admin login: The name of the administrator account of the SQL server
- Password: The password of the administrator account of the SQL server
- For "Storage", click on "Select backup"
- Navigate to the storage account containing the backup file
- Navigate to the container containing the backup file
- Select the corresponding backup file and click "Select"
- You should get a notification with the database import request on your Azure portal notifications tab
- You can monitor the process of the import status on the SQL Server resource page, under "Data management" > "Import/Export History"
- Once the process is finished, status under "Import/Export history" should be "Completed" and the database should appear under "Settings" > "SQL Databases"
|
Card |
---|
id | Deploy direct from Sql Server Management Studio (SSMS) |
---|
label | Deploy direct from Sql Server Management Studio (SSMS) |
---|
|
- Select your local DB server name and choose Windows Authentication
- Click on Connect
- There are three databases that we need to import into Azure. By default, these are called: EdFi _Admin, EdFi_ODS, EdFI_Security. If this is a yearspecific instance, there could be more than one EdFi_ODS for each year. After successfully connecting, locate the databases by expanding the Databases folder
- Choose the database that you wish to import (they can be imported in any desired order)
- Right click on that database, go to Tasks and then to Deploy Database to Microsoft Azure SQL Database
- Skip the Introduction
- Click on Connect
- Set the Server name with the Azure SQL Server
- Choose SQL Server Authentication and set your Username and Password info
- Click on Connect
- Set the name for the database that will be deployed to the Azure SQL Server
- Choose the parameters accordingly to the database that will be imported
*This parameters are still undergoing testing and may change over time
|
|
| EdFi_Admin | EdFi_Security | EdFi_Ods_ | Edition | Standard | Standard | Standard | Maximum database size | 1 GB | 1 GB | 50 GB | Service Objective | S1 | S1 | S1 |
- Click on Next
- Review the summary and click Finish to deploy the Database
- Repeat the previous 11 steps (32 - 42) with the other 2 databases.
|
Step 3. Create App Services in the Portal
This section explains how to create the three app services in the Portal for the Web API, the Admin App, and Swagger.
Expand |
---|
Create App Service Plan- From the Azure portal create an App Service Plan
- Choose your Subscription and Resource Group
- Name your App Service Plan and select Windows as Operating system.
- Choose Standard SI as Pricing plan
- Click Next: Tags
- Set tags according to your company's best practices. At a minimum we recommend using the maintained by tag name and select your name as value from the dropdown
- Click on Next: Review + create
- Review the plan and click on Create
Create App ServicesThese steps will need to be repeated for three web applications, the Web API, the Admin App, and Swagger - From the portal create a new App Service.
- Select your Subscription and Resource Group
- Name your application The name is something that will have to be unique across all of Azure so include information about the company. You will need separate App services for the Web Api, Swagger, and the Admin App.
- Set .Net 6 (LTS) as Runtime stack and Windows as Operating System
- Select the App Service plan created on step 53
- Click Next: Deployment
- Set Continuous deployment to Disable
- Click Next: Networking
- Set Enable public access to On and Disable Network injection
- Click Next: Monitoring
- Enable Application Insights and leave the default Application Insights
- Click Next: Tags
- Set the maintained by tag name and select your name as value from the dropdown
- Set any other desired Tag name and its value
- Click Next: Review + create
- Review the deployment and click Create. Wait a minute until the resource is completed.
- Go to the newly created App Service, select the ‘Configuration’ menu on the left and ‘general’ menu on the top, and set it to 64 bit platform.
- Repeat for the other web apps.
|
A Key Vault will allow you to save the database connection username and passwords in a secure location.
Expand |
---|
4.1 Creating Key Vault- From the Azure portal home page, search “Key Vaults” resource.
- Select Create
- Under the Basic Tab enter the required information mark with (*)
- Select Next + create (Access Policy, Networking, and Tags can remain with default values, they will be configured later)
- Under Review + create review all the selected options.
- Select Create
Once the Key Vault is created, you can view it in the Key Vault Section.
4.2 Adding Secret to Key Vault- From the Azure portal home page, search for Key Vaults resource.
- Select the Key vault that was previously created
- Using the Left navigation panel, select Objects => Secrets
- Select Generate/Import
- Enter the required information mark with (*)
- Upload options should be Manual from the DDL
- Secret value should have the connection string to the database
- Select Create
Once created, go back to the key vault and select Object => Secret; you will see the secret created Repeat Adding Secret to key vault for all 3 connection strings - EdFi_Admin
- EdFi_Security
- EdFi_Ods
And you will end up with 3 secrets
4.3 Add Access Policy to Key Vault- From the azure portal home page, search for App Services
- Select the App Service where the WebApi is deployed
- From the Left navigation panel, select Settings => Identity
- Turn Status On (if not turned on)
- Select Saved; this will Generate an Object (principal) ID
- Copy the Object (principal) ID
The above steps should be repeated for the AdminApp - Go to the Key Vault created in step 3.1
- From the Left navigation panel, select Access policies
- Select Create
- Under Permissions tab
- Select Key Management from the Configure template DDL
- Under Secret Permissions area, check Get & List checkbox
- leave other options as default
- Select Next
- Under the Principal Tab paste the Object (principal) ID on the search bar and select the app service to which you want to add the policy
- Select Next
- Application (optional) tab does not need to be configured.
- Select Next
- Under the Review + create tab, all options can be reviewed before creating the policy
- Select Create
Repeat the above steps for the AdminApp App Service Once done you will end up with 2 access Policy - WebApi
- AdminApp
|
Step 5. Deploy Web API and Swagger to App Services
This section describes how to deploy these two applications. They will need to be built from Visual Studio.
Expand |
---|
- Open up the EdFi.sln file in the Ed-Fi-Ods-Implementation/application/EdFi.sln file in Visual Studio. You should sign into visual studio using the same credentials that have access to the Azure Portal.
- Right-click on the "Entry Points" → Edfi.Ods.WebApi folder and select "deploy"
- On the deployment screen that comes up select the green plus sign to build a new deployment
Deck |
---|
|
Card |
---|
default | true |
---|
id | AzureOption1 |
---|
label | Deploy Direct to Azure |
---|
title | Option1 |
---|
| - Option 1- deploy straight to Azure
- Select a target of "Azure"
- select a specific target of "Azure App Service Windows"
- choose the appropriate Resource Group and region
- Choose the App Service that you created the WebApi and "save"
- Change the target Runtime from Portable to "win-x64"
|
Card |
---|
id | Option2 |
---|
label | Deploy to Azure-ready .zip binary |
---|
| - Option 2 - build a portable zip file
- Select a target of "Folder"
- Choose a file location
- Change the target Runtime from Portable to "win-x64"
- Click Publish.
- Open up the folder that you deployed to is located and zip up the contents of the deploy directory, not including the directory, with the following PowerShell commands:
Code Block |
---|
| $compress = @{
Path = "WebApi\*"
CompressionLevel = "Fastest"
DestinationPath = "WebApi.zip"}
Compress-Archive @compress |
- This should make a WebApi.zip file.
- In the Portal, Navigate to the Web Api App Service Plan.
- On the sidebar search box look for Advanced Tools, click on it and then click Go
- You’ll be redirected to a service named Kudu. Locate Tools and click Zip Push Deploy
- Drag and drop the .zip file and wait until it is uploaded, decompressed and deployed
|
|
- Repeat for the Swagger application
|
Step 6. Deploy Admin App to Azure App Service
The Admin App is not deployed from Visual Studio. Instead, it is installed locally with the installer and then pushed up to the app service as a zip file.
Expand |
---|
- Copy the AdminApp web application folder (usually c:\inetpub\ed-fi\AdminApp) to a working directory.
- Edit the web.config and set stdoutLogEnabled="true". Check what log4netconfig file that appsettings.json is pointing to and in that file make sure that all of the logging files are NOT going to $program_data or something like that. I use ./logs directory
- Open the working directory in PowerShell.
- Zip up the contents of the admin app folder, but do not include the folder itself. In PowerShell the following commands to this:
Code Block |
---|
| $compress = @{
Path = "AdminApp\*"
CompressionLevel = "Fastest"
DestinationPath = "AdminApp.zip"}
Compress-Archive @compress |
- This should make an “AdminApp.zip” file.
- Log into Azure from the command line with the command;
az login |
Now that the Web Apps have been deployed then you will need to set up the connection strings and other variables that are found in the 'appsettings.json' file.
Expand |
---|
General instructions for setting App settings and connection strings- Open the App Service that you want to set values for
- On the left menu find the "Configuration" option
- There will be an option towards the top of the screen to create new AppSettings and one at the bottom to bulk-edit connection strings
- After adding or changing values make sure to save the app service.
WebApi application settings- The webApi should have an Application Setting of “AppSettings:Mode” and a Value of “YearSpecific”
- The webApi should have an Application Setting of "ApplicationInsights:InstrumentationKey" and the value should come from the Instrumentation key from your App Insight
Ex
Admin App application settings- The Admin App should have an Application Setting of “AppSettings:ProductionApiUrl” and a Value of the URL for the Web API
Swagger application settings- Swagger App should have an Application Setting of ““WebApiVersionUrl”” and a Value of the URL for the Web API
NOTE: Make sure when you finish adding the settings you save the changes There are two methods of database configuring database connection strings we are describing. The first one has the database credentials hardcoded in the settings for that App Service. This is a violation of some organization's security policy. The second method uses the key vault set up in step 4 to hold the credentials. Deck |
---|
id | Configure database connection |
---|
|
Card |
---|
label | Credentials hard coded in the connection string |
---|
| In this section, we will configure AdminApp and WebApi connection strings directly. WebApi- From the azure portal home page, search for App Services
- Select the WebApi App Service
- Go to select Settings => Configuration
- Under the Connection string section, select Advance Edit
- Paste the following code into the text area that is prompted, replacing the Server, User Id, and Passwords with the details for the Azure Sql Server.
Code Block |
---|
| [
{
"name": "EdFi_Admin",
"value": "Server=<Your SQL server name here>; Database=EdFi_Admin; User Id=<Your SQL username here>; Password= <your SQL Password here>; Application Name=EdFi.Ods.WebApi;",
"type": "SQLAzure",
"slotSetting": false
},
{
"name": "EdFi_Ods",
"value": "Server=<Your SQL server name here>; Database=EdFi_{0}; User Id=<Your SQL username here>; Password= <your SQL Password here>; Application Name=EdFi.Ods.WebApi;",
"type": "SQLAzure",
"slotSetting": false
},
{
"name": "EdFi_Security",
"value": "Server=<Your SQL server name here>; Database=EdFi_Security; User Id=<Your SQL username here>; Password= <your SQL Password here>; Application Name=EdFi.Ods.WebApi;",
"type": "SQLAzure",
"slotSetting": false
}
] |
Admin App- From the azure portal home page, search for App Services
- Select the Admin App app service
- Go to select Settings => Configuration
- Under the Connection string section, select Advance Edit
- Paste the following code into the text area that is prompted, replacing the Server, User Id, and Passwords with the details for the Azure Sql Server.
Code Block |
---|
| [
{
"name": "Admin",
"value": "Server=<Your SQL server name here>; Database=EdFi_Admin; User Id=<Your SQL username here>; Password= <your SQL Password here>; Application Name=EdFi.Ods.WebApi;",
"type": "SQLAzure",
"slotSetting": false
},
{
"name": "ProductionOds",
"value": "Server=<Your SQL server name here>; Database=EdFi_{0}; User Id=<Your SQL username here>; Password= <your SQL Password here>; Application Name=EdFi.Ods.WebApi;",
"type": "SQLAzure",
"slotSetting": false
},
{
"name": "Security",
"value": "Server=<Your SQL server name here>; Database=EdFi_Security; User Id=<Your SQL username here>; Password= <your SQL Password here>; Application Name=EdFi.Ods.WebApi;",
"type": "SQLAzure",
"slotSetting": false
}
]
|
|
Card |
---|
| In this section, we will configure AdminApp and WebApi to pull the database connection string from the key vault Prerequisite: you must have followed the instruction in step 4 on how to set up the azure key vault - From the azure portal home page, search for App Services
- Select the App service on which you have the WebApi configured
- Inside WebApi Service, select Settings => Configuration
- Under the Connection string section, select Advance Edit
- Paste the following code into the text area that is prompted
Code Block |
---|
| [
{
"name": "EdFi_Admin",
"value": "@Microsoft.KeyVault(SecretUri={Secret Identifier})",
"type": "SQLAzure",
"slotSetting": false
},
{
"name": "EdFi_Ods",
"value": "@Microsoft.KeyVault(SecretUri={Secret Identifier})",
"type": "SQLAzure",
"slotSetting": false
},
{
"name": "EdFi_Security",
"value": "@Microsoft.KeyVault(SecretUri={Secret Identifier})",
"type": "SQLAzure",
"slotSetting": false
}
] |
Configure the following properties inside the above JSON - value: "{Secret Identifier}" must be replaced with the secret identifier value that is auto-generated from the secret (Step 4.2)
- name: must match the name of the database that you have in your appsettings.json
Once done, you will end up with something similar to this
- Once done and verified that everything is correct, save the changes
Important: Repeat all the above steps for AdminApp |
|
|
...