The Data Import (DI) SIG had 3 goals:
Understand and define pain points that Ed-Fi implementations experience while using Data Import as an Extract-Transform-Load (ETL) solution for non-API based data.
Understand other solutions that may be available for reliable ETL pathways for non-API based data.
Inform Data Import future development and roadmap priorities from community blockers, issues and identified needs
These were the outcomes of the SIG from these goals are:
1.) Data Import is reported as serving non-API ready data needs with an active community of users
- Active SIG with over 40+ members to contribute to the conversation
- We're seeing usage in numerous areas to accommodate non-API ready data - assessments, EPP data, finance data and so on.
- Recognized that Data Import carries a maintenance burden that the implementer has to maintain
- API path to data without Data Import is the best way to go
2.) Numerous viable open-source and low-cost alternatives exist for serving the ETL need
- EdAnalytics
- Great contribution for ETL tooling and report out from EA
- Cloud based in mind and Python based too
- https://github.com/edanalytics/earthmover
- https://github.com/edanalytics/lightbeam
3.) Data Import users will prefer an open-source path ahead for the product
Lorem ipsum...
4.) Additional Data Import SIG requests lead to these feature requests
Lorem ipsum...