/
TAG Meeting 2021-01-21 - Subgroup on Data Import Roadmap

TAG Meeting 2021-01-21 - Subgroup on Data Import Roadmap

Feb 03, 2021

Participants

  • David Clements Ed-Fi Alliance

  • Rohith Chintamaneni Arizona Department of Education

  • Thomas Christensen Wisconsin DPI

  • Gary Clarke Instructure

  • Rosh Joshua Dhanawade Indiana University INsite

  • Stephen Fuqua Ed-Fi Alliance

  • Jean-Francois Guertin EdWire

  • Jason Hoekstra Ed-Fi Alliance

  • Eric Jansson Ed-Fi Alliance

  • Erik Joranlien Education Analytics

  • Shannon Kerlick Ed-Fi Alliance

  • Alex Kichkailo Instructure

  • Jim McKay Instructure

  • Chris Moffatt Ed-Fi Alliance

  • Doug Quinton PowerSchool Group LLC

  • John Raub Wisconsin DPI

  • Andrew Rice Education Analytics

  • Jim Robertson PowerSchool Group LLC

  • J. Pablo Rodriguez San Diego County Office of Education

  • Sayee Srinivasan Ed-Fi Alliance

  • Patrick Yoho InnovateEDU

Agenda / Materials

Review of https://edfi.atlassian.net/wiki/display/EFTD/Data+Import+Preprocessor+Enhancements+Design

Instructure Demo Video

Notes

Background

  • Last year Ed-Fi Alliance began working with Certica/Instructure who offered community contributions into Data Import

  • Data Flow - precursor to Data Import source code

  • v1.2 update: new agent type, different level workflow. Template sharing in PowerShell (e.g. iReady), preprocessing ability using Powershell, security enhancements

  • Q: with net core change, should also consider powershell core since they are not necessarily mutually exclusive https://powershell.org/2019/02/tips-for-writing-cross-platform-powershell-code/
    A: A few different models to be looked at and support for Python and PowerShell on Linux base platform

  • PowerShell enhancements in v1.2 planned for release in March.  Python support in consideration for v1.3 along with .NET Core updates.
    V1.2 presentation by Instructure (appended PPT)


Design documents discussion

  • There were some comments on the pre-processor language including request for Powershell Core and Python to be supported

  • Most discussion concerned the questions around pre-processing and how to secure it (or if it could be made sufficiently secure) and if there should be some higher-level abstraction for such preprocessing (i.e. a DSL type language that encapsulates particular options). There were some comments that it would generally be hard to get away from scripting, and that scripting is a function to which there is reasonable community access.

  • There were some comments on possible Web UIs as well, which are seen in other environments where data must be pre-prepared prior to ingestion. 

  • There were also some observations/ comments about the difficulty of making anything based on scripting available to the wider Ed-Fi community, given that the target audience is a data analyst.

  • There were comments suggesting development of a CLI with a Web interface that calls the CLI, allowing for more development / consumption operations for users of the toolset. Generally there were suggestions of greater decoupling of components.

Next meeting: