/
Meeting 3 - 2021-06-24

Meeting 3 - 2021-06-24

Jun 25, 2021

Agenda

  1. Review sample visualizations - see LMS Starter Kit Sample Visualizations

  2. Review key architecture decisions

    1. Harmonizers to be "tailorable" to local needs and practices - see LMS Harmonizer

    2. Plans for an API per feedback in earlier LMS SIG meetings (also allows usage of Data Import) - see LMS API

  3. Cover issues joining sections (reliance on joining via a single section identifier) - see LMS - Joining Sections

  4. Google Classroom issues

    1. How Google Classroom captures district student IDs / proposal to join on email address

    2. How Google Classroom captures section identifiers (via Course Alias?)

  5. Gradebook domain issues

    1. SIG recommended that gradebook was separate from assignments

    2. Concerned that Gradebook has an issue (GradebookEntry key is likely volatile, as it has a title and a date) - see https://schema.ed-fi.org/datahandbook-v320c/#/GradebookEntry560

    3. Would the assignment identity carry over into the gradebook?

Check in sheet is here

Participants

  • Ed Comer

  • Molly Stewart

  • Mario Palmisano

  • Mark Masterson

  • Ryan Gravette

  • Brian Mullins

  • Tim Pritchett

  • Gabrielle Garonzik

  • Chris Moffatt

  • Eric Jansson

Notes

Quick review of where the project is to date.

Review of possible visualizations

  • two views: school view and student section view

  • drill down options: time period, grade level, courses, demographics

  • intended user: school admin for school view. curricular level in district.

Feedback/discussion

  •  add drill down by staff (teacher), as this reflects how principals or curricular staff intervene

  • add activity outside of assignment submissions for v2 release

  • add velocity - are students turning in a lot of assignments NOW but behind? (acceleration = good, status = bad) or slowing down? (acceleration = bad, status = good). go beyond just the current speed. also compare individuals against overall course/school/etc. would also need to account for steady speed (acceleration = zero) which is not necessarily bad.

  • clearly define what action each piece of this dashboard is intended to drive? note if a student's acceleration is increasing, even if behind that should go into disengaged metric.

  • celebration actions are just as important as remediation actions. call these out!

  • could include velocity in another bar in the bottom to indicate actions in the past week or two weeks. Proposal of having the last few grades, etc.

  • track difference between assignment submission dates and grade dates. more of a staff-centric metric.

Review of architecture

LMS harmonizer

Feedback/discussion

  • Concern around data loss between the LMD database and Ed-Fi LMS harmonizer. How are the decisions around what data is important being made?

  • mostly a process of loss. only writing data that can be joined to the ODS (from the SIS). this is not necessarily a bad thing because it provides some validation.

  • also a process of augmentation. some data cleansing (not a lot) can be done here as well.
    motivated to make the data more usable

  • with that in mind, then does it make sense to have the data in the ODS?

  • ideally, we want to no longer need the LMS. but given the lack of community standards in many areas, we need an in between option.

  • Rather than loading all the data, maybe there are key elements that should be, like an engagement measure, and the rest just gets reported out of the lms data directly.

joining on section
Current thinking is to just join on section identifier. this is based on feedback from sis - section identifier are unique for a school year and a school. one sis does not follow this pattern so would have to add local course code with section identifier.

Feedback/discussion

  • concern when there are multiple instances of a sis. section identifier would be unique per sis instance. would require school id to also disaggregate.

  • an LMS should be able to provide a school id. this could be added to the primary key (i believe it already is).

  • could also add school year in case those same conflicts could go across years.

  • when an integration is migrating to a new sis, may have duplicates for prior school years.

  • our convention for rostering sections in Canvas consists of using the school year, the SIS Section ID (text entered by the user), term, period, etc. That's been helpful when trying to reconnect them to SIS data for analysis.

Google Classroom questions

How does google store a student sis id? or does it at all? or do we default to email address as unique? email is our current approach.

Feedback/discussion:

  • general recommendation is to use the email to join

  • Google is building out automated sis sync. top two education tiers (pricing) have an auto sis sync. should be released this fall.

How does google capture section identifiers? is this via the course alias?

  • Will look into it

Ed-Fi gradebook domain model

  • initial thought from sig - keep gradebook separate domain from assignment but include the link between the two

  • older key structure issues - DateAssigned, GradebookEntryTitle, SectionReference. changes would be breaking so this will need careful consideration.

  • dates can changed and/or are frequently mis-entered

  • title can also be pretty volatile

  • section is pretty complex and includes fields an LMS may not have

Feedback/discussion

  • agree this is a problem. fixing it may be an even bigger problem. some integrations don't have an LMS- so what vendor systems would "own" this domain? data thrashing is a concern.

  • initial feedback seems to want the sis to have ownership and fill in the gaps with LMS data.

  • Should the identity system assume a SIS in the middle (that assigns an ID) or not?