RVWG 2019-04-11 Meeting notes - Tech Congress

Owned by Itzel Torres
Last updated: Aug 05, 2019

Date

Attendees

Goals

  • Define an initial Early Warning Use Case to use

Discussion Notes

While the goal was to select and define one complete use case around Early Warning System, we had 

Early Warning Indicators our session was joined by a few Ed-Fi members that are not part of the group which lead to a good discussion on gaps on some domains and work by other groups that we could potentially benefit from. High level notes below:

Early Warning 

Discussion started evaluating which Early Warning use case to start with, some brought up work from EDAM and Wisconsin as well as research available that could be used instead of coming up with use cases from scratch.

Billy described 2 approaches to Early Warning: Rule based and predictive, with predictive being more complex so we would start our MVP (Minimum Viable Product) with a rule based approach.

  • There is an EDAM group that is looking at students at risk of not graduate. This group is looking at what factors will affect graduation.
  • RVWG would want to distinguish our use case a bit more, so that EDAM group efforts are not being duplicated.
  • Probabilistic - Wisconsin is a good example where they are using Algorithms to identify kids at risk but may not know what indicator mix resulted in kids.

On what level to start with:

  • While there are some gaps at the Elementary school level,  at this level there is not a lot of data and grade information is harder to capture. For dashboard users in the room, early warning is achieved through dynamic watch lists created by teachers and admins.
  • High school seems more attractive to start with for system integrators because:
    • There is usually grant money available for high schools to work on data projects that support graduation rate improvements.
    • There is better and greater amounts of data to work with. Would make it easier to have a quick MVP since we already have good datasets to work with.
    • In Mississippi high school data is too late, we need to capture this a lot earlier. Can we add a middle school level measure?
    • Schools and 9th graders come together in a common indicator. How do we understand if the changes we are making are improving or not. Must consider longitudinal data to show how much is efforts are working.

Proposed path forward:

  • There is a lot of data for high school, why don't we look into this and pick one sample and work through it.  Start with the things that we know across the board affect students, after the MVP allow the schools and districts extend the model to tailor it t them.
  • Identify the 3-5 indicators that would work across the board.
  • Use the work of the Metro Nashville work as an MVP starting point.  This group can take a look at what they are doing and release a MVP to the community. 

Metro Nashville - Use Cases

FCPS Early Warning Indicator System Use Cases

Discipline

As the discussion on Early Warning, discipline came up as another viable option. For dashboard users the option available in the dashboard does not provide actionable data as it is mostly an incident list and not very granular and there is a gap on what is available in the market.

For some state/districts discipline is considered a good indicator of student success and if reported in an actionable way and there doesn't seem to be a good option available.

Challenges:

  • Districts push back on working with discipline data, they don't want to share that level of data if it is not required for state reporting they don't want to share it and risk that data getting out to the community and affecting students. But having effective discipline reporting can result in early intervention and improved outcomes in the classroom. An example provided: If as a teacher/staff if you could view that most incidents happen while kids are waiting for the bus, a school could change bus loading procedures and address underlying causes between groups.
  • There is a need for a PBIS (positive behavior intervention support) not a state reporting tool, this is a tool to give you a way to view behavior patterns, for example knowing that during the time that students are waiting for the bus is when more incidents happen
  • Should we consider adding discipline as a measure in early warning. In Mississippi discipline is not considered an important or relevant indicator.
  • Based on research others in this group (education analytics) they define early warning on attendance, discipline and grades (maybe credits depending on graduation plan). 

  • Not a lot of data is available currently. There are in-school/out of school suspension that matter and can be used and not a bad place to start.

High level needs/gaps:

  • Need to be able to think about local distributions so that school districts can take action which is where they can take advantage. At the superintendent level, they can look at 4000 students so there has to be an aggregate level in a meaningful way.
  • At the site level, you need the ability to set a cut point. At the district level we need to see everyone at the same level. At the site level you have different resources to work with so it will have to be flexible.
  • Nice to have would be to consider social emotional measures, at the district level when we talk with site administrator we always talk about life changes that are not being measured or tracked but have a big impact if it was made available.

Action items