Changes since Beta 1

• Using the same ODS/API code as recently released for version 3.3 (thus the ODS/API at "production quality")
• Supports Data Standard 3.2
• Admin App now included.
• Improvements to the deployment scripts

• . From end-user perspective, the primary change is to download previously-packaged versions of the web applications instead of re-compiling and re-packaging them.
• No longer testing on PostgreSQL 10, now that all three cloud providers support PostgreSQL 11.

• Admin and Security databases no longer have quoted-name identifiers, consistent with the ODS database.

Project Scope

As a reminder, project Roadrunner is simplified, binary-release of the Ed-Fi ODS/API, targeting

PostgreSQL

. This installation provides the following applications:

1. ODS/API with all resources from EFDS32, operating in "Shared Instance" mode.
   1. Does not include: extension support; API Composites; sandboxes; change queries; profiles.
2. AdminApp
3. /wiki/spaces/ODSAPI33/pages/26443888

2. Swagger UI
3. Deployment scripts for deploying the databases and configuring the four applications above.

Operational Requirements

The Roadrunner deployment scripts allow the user to deploy the three Ed-Fi databases (Admin, ODS, Security)

on a PostgreSQL database server. In addition, they download the ODS/API version 3.3 binary release of the API, Swagger UI, and Client Side Bulk Loader, and the current pre-release of Admin App v1.7. After download, the deployment scripts will reconfigure the web.config files by setting up the correct connection strings for the environment, and they will copy the files out to another directory of the user's choosing. These could be network paths for pushing the files out to another web server.

Deployment Server

For the server running the deployment scripts:

• Scripts are built on assumption of PowerShell version 5 or above.
• Requires .NET Core 2.2 SDK

• Requires the command line tools from PostgreSQL 10 or 11, with psql.exe  in your command path.

  Expand

• title Installing PostgreSQL tools using the installer...

  Download the 11.5 release from Enterprise DB (Windows x64 edition). Run the installer. When the installer gets to the point of selecting components, you can deselect the Server and Stack Builder options. pgAdmin4 is optional; it is a versatile GUI tool for managing your server, with the drawback that it cannot open or save SQL files. Image Added When the installation is complete, edit your path by adding C:\Program Files\PostgreSQL\11\bin (assuming you installed the x64 edition).

  
  

  Expand
  title Installing psql.exe from the binaries...
  Download the binaries ZIP file (not the installer) for the desired PostgreSQL version from the Enterprise DB binaries page. Extract the following files into a folder in your PATH environment variable (or to a folder you'll add to your path): libiconv-2.dll libintl-9.dll libpq.dll libwinpthread-1.dll psql.exe

  
  

• Internet access - this is not an offline deployment process.

Web Server

• Windows 10 Professional or Windows Server 2016+.

• Server role: Web Server. On our integration server we have the following IIS features installed:

  Expand
  title IIS Features ...
  Common HTTP Features Default Document Directory Browsing HTTP Errors Static Content HTTP Redirection Health and Diagnostics HTTP Logging Logging Tools Request Monitor Tracing Performance Static Content Compression Security Request Filtering Basic Authentication Windows Authentication Application Development .NET Extensibility 3.7 Application Initialization ASP.NET 4.7 ISAPI Extensions ISAPI Filters WebSocket Protocol

  
  

• URL Rewrite module
• .NET Framework 4.8 runtime
• Firewall access from web server to database server, if running on separate servers.
• Does not require MSMQ or .NET Framework 3.5, unlike the full ODS/API release.

Database Server

• PostgreSQL 10.x or 11.x. This release candidate was only tested on version 11.5; however, the beta release was also tested on

• a 10.x database, and no version 11 features have been used.
• Credentials with permission to create and drop databases.

• There must be a role called postgres. If you do not have a postgres  role, you can create one with command:

  Code Block
  create role postgres NOINHERIT;

  
  

grant postgres to edfi;

Install Process

1. Download NuGet Package

Roadrunner binaries and automation scripts are available via a NuGet package. You should download this on your webserver. Two options: NuGet command line or direct download.

This command will automatically unzip the contents into the working directory, under  EdFi.Ods.RoadRunner.1.0.0-rc1 .

nuget.exe install EdFi.ODS.Roadrunner -Version 1.0.0-rc1 -Source https://www.myget.org/F/ed-fi/api/v3/index.json

Alternately, download from https://www.myget.org/F/ed-fi/api/v2/package/EdFi.ODS.Roadrunner/1.0.0-rc1. This is just a fancy zip file - so if you want to inspect it, just open it with your favorite zip file client.

Inside that NuGet package, you'll find Initialize-Environment.ps1 . This orchestration script handles all of the deployment processes described above - installation to either database engine, download of the applications, and setting up connection strings. 

2. Run Initialize-Environment.ps1

Prepare

In order to run the script, you'll need to have the following information at hand:

1. Database server name
   
   1. In either case, is the database engine running on a non-default port? (default is 5432 for PostgreSQL)
      
      

2. Database user credentials

   1. While the username is passed in as a parameter, the password must be passed through environment variable $env:PGPASSWORD  or

1. 1. a pgpass config file.
      

1. Choose a template
   1. Empty: contains no data.  Not recommended as getting initial key/secret and descriptors setup is challenging.
   2. Minimal: contains default descriptors, making it possible to setup new education organizations and to create a client key and secret.
   3. Populated: contains a small set of sample data, in addition to the descriptors.
      
      
2. Additional database options
   1. If your instance requires SSL/TLS encryption, then set environment variable $env:PGSSLMODE="require" .

1. 2. If the destination databases already exist, do you want to drop them? ( for PostgreSQL, requires that you have the client utilities installed on the deployment server).
   3. Do you want to change the database names away from the defaults of EdFi_Admin , EdFi_ODS, and EdFi_Security?
      
      
2. Destination folder paths for the four applications:
   1. Web API
   2. Swagger UI
   3. Admin App
   4. API Bulk Loader
      
      

This script has a long list of parameters. It can be useful to create a PowerShell object (dictionary) with the values, instead of trying to string all of the values together in a single command. Doing so makes it easier to review your parameters before running the script.

# Prepare an object dictionary with your parameters
$parameters = @{
   # Add parameters here, e.g.
   engine = "postgresql"
}

# Can also set parameters one-by one
$parameters.databaseServer = "myserver"

# Can review all parameters
$parameters

# Everything look good? Then pass the parameters to the deployment script:
.\Initialize-Environment.ps1 @parameters

Command Syntax

Initialize-Environment [[-swaggerIisFolder] <String>] [[-webApiBaseUrl] <String>] [[-webApiIisFolder] <String>] [[-apiBulkFolder] <String>] [-engine] <String> [-databaseServer] <String> [[-databasePort] <String>] [[-databaseUser] <String>] [[-databasePassword] <String>] [-useIntegratedSecurity] [[-adminDatabaseName] <String>] [[-odsDatabaseName] <String>] [[-securityDatabaseName] <String>] [-useTemplates] [[-odsTemplate] <String>] [-noDuration] [-dropDatabases] [<CommonParameters>]

Parameter List

Examples

Install databases on PostgreSQL

and all three WebApps (Web API, Admin App, Swagger UI), and selecting the ODS populated template. (This is a good option for kicking the tires).

$parameters = @{ 
	engine = "postgresql"
	databaseServer = "myserver"
	databaseUser = "postgres" 
 

c:\inetpub\ed-fi\webapi"
    swaggerIisFolder = "c:\inetpub\ed-fi\swagger"
    adminAppIisFolder = "c:\inetpub\ed-fi\adminapp"
}
 .\Initialize-Environment.ps1 @parameters

Only install databases on PostgreSQL,

using all available default values.

$parameters = @{ 
	engine = "postgresql"

Only install databases on

PostgreSQL, using templates (populated ODS).

$parameters = @{ 
	engine = "

Only install databases on PostgreSQL, dropping existing database first.

$parameters = @{ 
	engine = "

.\Initialize-Environment.ps1 @parameters

3. Setup IIS Applications

As mentioned earlier,  configuration of IIS is up to the end user in the release candidate.  Some IIS configuration tips follow:

• If the deployment server is not also the web server, and if your "IIS Folder" paths were local rather than remote, then copy/move the three web application folders to your IIS web server(s). In IIS Manager, setup an Application for each, with custom app pools.
• Grant write access to the IIS_IUSRS account to the folder containing AdminApp assets, so that the secrets.json file can be updated during the Admin App's first-run configuration.

Running the Applications

Now that everything is setup, you can start using the applications.

• As a first step, you may want to confirm that the Web API is running. If you installed on server "myserver" with IIS Application named "webapi", then try loading https://myserver/webapi in a browser to confirm that the version endpoint does not generate any errors. If that works, next try https://myserver/webapi/metadata. Any problems? Check the ODS/API error log, which by default is in c:\ProgramData\Ed-Fi-ODS-API.
  
  
• Before accessing Swagger UI or loading data via the API Bulk Loader, you'll need to generate client credentials (key and secret). Two options:
  • Admin App 

  • Manual setup through the database: postgres manual vendor setup - Admin - rc1.sql
    

    Tip
    Improved version shared by InnovateEdu: improved postgres_create_vendor.sql

    
    
    

• Admin App initial setup
  • If you installed Admin App on server "myserver" with IIS Application named "AdminApp", then try loading https://myserver/AdminApp.

  • Click through the Admin App initial setup
    

    Tip
    If the initial setup screen reloads itself after the first time you click the setup button, then try waiting a few minutes for some background tasks to complete and then try again.

    
    

  • Stop and restart the WebAPI web site via IIS Manager to force a security cache refresh.
  • Reload the Admin App and create a vendor and generate a key and secret.
  • For more information on Admin App, please see:
    • User Guide
    • Securing the Admin App

Providing Feedback

Found a bug? Please file a Bug Report in the ODS project in Tracker. Ideally, use "roadrunner" in the labels field and put the name "Roadrunner" in the summary.

Feedback on the deployment scripts? Either file a ticket in Tracker, post a comment below, respond in #dev-roadrunner in Slack, or send email to Vinaya Mayya and Stephen Fuqua.

Troubleshooting

role "postgres" does not exist

If you get this error...

2020-01-08 16:35:18,701 [1] INFO  PostgresUpgradeEngineFactory - Npgsql exception has occured in script: 'EdFi.Ods.Standard.Artifacts.PgSql.Structure.Ods.0010-Schemas.sql'
2020-01-08 16:35:18,701 [1] ERROR PostgresUpgradeEngineFactory - Script block number: 0; Block line 5199; Position: 0; Message: 42704: role "postgres" does not exist
2020-01-08 16:35:18,745 [1] ERROR PostgresUpgradeEngineFactory - Npgsql.PostgresException (0x80004005): 42704: role "postgres" does not exist
   at Npgsql.NpgsqlConnector.DoReadMessage(Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isPrependedMessage)
   at Npgsql.NpgsqlConnector.ReadMessage(Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications)
   at Npgsql.NpgsqlConnector.ReadMessage(Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications)
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming)
   at Npgsql.NpgsqlDataReader.NextResult()
   at Npgsql.NpgsqlCommand.Execute(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteNonQuery(Boolean async, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteNonQuery()
   at DbUp.Support.ScriptExecutor.ExecuteNonQuery(IDbCommand command)
   at DbUp.Postgresql.PostgresqlScriptExecutor.ExecuteCommandsWithinExceptionHandler(Int32 index, SqlScript script, Action excuteCommand)

Then you are probably running on a system where the sys admin user was not created with the default name of "postgres". You can create the role and assign your superuser to it and then try re-running. For example if you have a sys admin account called "edfi" then issue this command:

create role postgres NOINHERIT;
grant postgres to edfi;