Summary Recommendation

Distribute software files in the most appropriate channel for the class of software:

Motivations

Too Many Solutions

To date, the Ed-File Alliance has used several different solutions for distributing software packages, including:

• Confluence
• AWS
• Azure
• MyGet
• GitHub Releases

When we have a new project, where should we store the binary output for distribution to the community? We do not have a coherent strategy.

MyGet Quota

MyGet has been an important channel for releasing .NET packaged binaries. The current subscription is very low cost on a legacy plan, but requires frequent maintenance in order to remain below the 4 GB quota. A new strategy is needed for these packages.

New Licenses

The Apache license opens up new opportunities for public distribution of those packages under the Apache license. But what about other applications that are not under the Apache license - MetaEd and Data Import?

Requirements for a Coherent Strategy

1. Choose the right tool for the job.
2. Closed-source applications do not require authentication / authorization, but should not be placed in highly-visible locations.
3. Must be able to automate pushing new binaries to the appropriate service.
4. Must be able to set appropriate retention policies for pre-release packages (to be determined).
5. For legacy applications, consider lower-tier storage options for cost optimization but do not spend much time on doing so.

Strategy by License Type

Apache Licensed Software

.NET Libraries and Packages

i.e. NuGet packages, including the PowerShell installer packages.

Pre-release packages will best be handled through Azure Artifacts. Released packages can be "promoted" to NuGet Gallery. This would require some additional engineering effort, with the benefit of greater visibility. If visibility is desired then it is a good solution. However, it would be simpler to consolidate only on the Azure Artifacts platform. The Alliance could store nearly fives times the amount of data on Azure Artifacts as it does on MyGet for the same low cost ($25/month). The cost savings of moving released packages to NuGet Gallery do not justify even an hour of additional engineering effort.

Anyone with older code will be hitting MyGet to retrieve packages, so leave it run and provide a transition plan.

Update TeamCity to publish packages to the new repository automatically.

Recommendations:

• Azure Artifacts
• One-year deprecation notice for packages on MyGet.org along with instructions on how to change to the new repository

.NET Application Installers and Cloud ODS

Executable installers ("msi") for the ODS/API and Admin App.

Azure or AWS Blob storage will provide an easier end-user experience than a NuGet package at the cost of having to paste a new URL into a Confluence page for each release. Given the low release frequency this seems reasonable.

GitHub release works just fine in this case. However, it is not a good solution for anything else, and it would be preferable to consolidate platforms.

Use TeamCity to push new artifacts to Azure Blob Storage on demand, but not automatically.

Recommendation: Azure Blob Storage

Docker Images

Public images should go to Docker Hub for visibility. At only $25/month for five years on an annual plan, the cost to have unlimited image retention and a reasonable amount of build time is well worth it. Use of Azure or AWS would be reasonable and cost effective but drives more network traffic between TeamCity and the host, and lowers the discoverability of the images. 

Similar to .NET application installers - Azure Blob Storage seems like a good and reasonable fit for this need.

Data Import is distributed as an executable installer. MetaEd has both a zip file and JavaScript packages (npm). The packages must continue to be distributed through npm. Anyone with older versions of MetaEd will be hitting the MyGet server to retrieve packages.

Update TeamCity to publish installers to Azure Blob Storage on demand, and npm packages (on demand/automatic yet to be determined).

For more information on license acceptance, see Closed Source License Acceptance. 

Recommendations:

• Azure Blob Storage for exe / zip files
• Azure Artifacts for npm
• One-year deprecation notice for packages on MyGet.org along with instructions on how to change to the new repository

Ed-Fi Legacy License

Long-Term Support

Those applications that are still supported - for example ODS/API 2.5.0.1 - can be migrated to Azure Artifacts.

Out-of-support Packages

Legacy binaries on MyGet should be moved to cloud storage, possibly in a lower-cost (low usage / high latency) tier. These do not need to be advertised to the community. Even the old NuGet packages should be placed in blob storage, as there is no need to make them available via NuGet feed.

Recommendation: Azure Blob Storage

Cloud ODS

No change to current practice.

Recommendation: Azure Blob Storage

Dashboards (Legacy)

No change to current practice.

Recommendation: Amazon S3

Next Steps

See Transition from MyGet to Azure Artifacts.