RVWG 2019-09-29 Meeting Agenda

Participation

Support:

The meeting is scheduled at 2019-09-29 3:00 pm - 5:00 pm CT. Hilton Austin Hotel, Room 408

Agenda/Meeting Minutes:

1. Terms of the Ed-Fi Community Intellectual Property Disclosure Policy 
   1. Intellectual Property Slides
2. Tech Evaluation Overview: Technology Landscape, what is available now
3. Ed-Fi Dashboard Overview: Pros and Cons
4. Roadmap goals
5. Questions
6. Next Steps

Comments or feedback from the last meeting:  None

Discussion:

1. Ed-Fi Intellectual property community guidelines reminder.
2. Charter goals and updates - Slide 4 Phase I main goal or identifying priority use cases is now complete and can be viewed Use Cases Selection
   1. User Experience workgroup is working through user personas and functionality for the initial Student Discipline Use case.
   2. Evaluation of existing visualizations and Ed-Fi Dashboards is captured at a high level in the presentation. More detail available under the Technology Subgroup and the Ed-Fi Evaluation Subgroup links on presentation.
3. Technology Landscape - Slide 6 - 8
   1. More options available, with a focus on the separation of backend data management/concerns and UI visualizations as more independent of each other. Open source tooling, Off the shelf, and software as a service option.
4. Ed-Fi Dashboard - Slide 10 -11
   1. UI consistent since its inception in 2012, main updates to app architecture to make it more extensible and configurable.
   2. Costly to Implement & Maintain: Noted by the group that this is not necessarily unique to Ed-Fi Dashboards but a reality of another tooling as well.
   3. RVWG 2019-09-29 Meeting Agenda from INsite shared their journey to Ed-Fi ODS implementation + Dashboards:
      1. Took about 18 months
      2. Cost of ~ 900k.
      3. Had they had more in-depth knowledge of the data model, they could have saved some time, maybe 6 months. 
      4. The application follows complex patterns, but once you understand them it is easy to extend and maintain. In their case, it was harder to find resources with experience in known BI tools, but they had access to developers that can help customize and maintain.
   4. UI is non-responsive so newer monitors show a lot of blank wasted space. Not really usable in mobile devices
      1. Estimating ~ 250k to make UI responsive but no in-depth estimates have been gathered.

      2. ADA compliance is a challenge, there was an investment in v1.2 (2013) to address this but requirements change state to state with some being very restrictive. RVWG 2019-09-29 Meeting Agendashared that Kentucky has strict codes and as-is dashboards are not compliant. Group agreed that attempting to cover all possible requirements it's very complex but there is a need to keep in mind that visualization tools need to at least get implementations most of the way there.  

          ADA Compliance additional notes. Click to Expand...

         

         For our on-going discussion…

         Here’s one of several high-level summaries of what WCAG 2.0 specifies. This is clearer than how I tried to explain it in our session. Ed-Fi couldn’t “adopt WCAG 2.0” because the Dashboards couldn’t comply with all of the guidelines. If “most governments target Level AA,” then there are those that desire more. However, Level AA might be a reasonable one for Ed-Fi to adopt. Then, as the discussion pointed out, local implementations could enhance as required. The same is generally true for other standards, which have levels of their own, and requirements that don’t apply directly to the Ed-Fi solution. https://www.wuhcag.com/web-content-accessibility-guidelines/

         WCAG 2.0 levels

         The Web Content Accessibility Guidelines 2.0 are organised into three levels of conformance:

         • Level A – the most basic web accessibility features
         • Level AA – deals with the biggest and most common barriers for disabled users
         • Level AAA – the highest (and most complex) level of web accessibility

         For most websites, Level AA plus some Level AAA is the best target. That’s because some of the highest level guidelines simply can’t be applied to all websites. However, one of the problems with the three-tier structure is that if people know they can’t attain AAA, they won’t even look through the guidelines to see where they can improve accessibility. With all of your projects, you should comply with all the guidelines you can, whether you want Level AAA or not.

         Starting with Level A is a great way to make progress and begin helping out your users. Level AA is the standard many governments are using as a benchmark as this level targets the most common and most problematic issues for web users.

         Billy Buchanan: I think AA is the bare minimum that would be needed.  Some of the standards in WCAG aren’t directly relevant to the dashboard work (e.g., including transcripts of audio/video in alt attributes for screen readers, etc...).  I think AA also sounds like a reasonable target level given that some of the issues (e.g., color contrast ratios) do not have a level A conformance standard; this may have changed since the last time I looked at things, but from what I remember there were only standards for AA and AAA for color contrast ratios.

         If our goal is to enable all educators to leverage data assets to inform decision making processes I think we need to make sure that any technology put forward enables all educators to do that.  If some organizations are using the dashboards to involve students/parents in discussions I think ensuring that some minimal level of accessibility is satisfied aligns fairly well with that goal.

         
         

5. Goals for the Group - Slide 13
   1. Noted that there is a need to support those that find value in Ed-Fi Dashboards in a way that moves the community forward and makes the best use of the limited resources of the Ed-Fi Alliance.
   2. Glynn commented that considering making the Ed-Fi Dashboard codebase open source so others in the community can benefit and contribute to improvements.
   3. A middle reporting layer makes sense and seems to offer a way to obtain stability overtime.
   4. FL Code shared a bit of their experience with a core Ed-Fi Dashboard implementation and a move to newer visualization tools:
      1. Initial implementation like Rosh shared was painful and expensive, eventually, the technology caught up to us and what we wanted to do was just easier to achieve with newer technology.
      2. ODS updates are a concern, so this middle layer starts to make sense. Implementations are doing some iteration of this but it would be nice to see a common layer that can cut that getting started time so you can get to the other issues that will come up.
6. After the discussion, attendees were asked to submit notes and comments to be shared with the group on what they think about the ideas and challenges discussed.

Notes/Feedback from attendees:

1. Anonymous Recommendations:
1. Focus on building a service that can provide standardized data set that can be used by Industry-standard visualization tools such as Tableau or PowerBI
2. The ODS uses an entity-relationship model which is not always efficient for analytics. Consider creating a standard star schema using FACTS & dimensions with data extracted from the ODS
3. Consider creating a metric catalog that pre-calculates and stores metrics from the ODS
2. Anonymous Recommendations: Abstraction/Middle-tier, let use (implementations) decide the output.
3. Anonymous Recommendations: Rename Dashboards - Data-Informed Visualization
1. Move to Open source
4. Raj - ESP Solutions: My vote for Responsive UI as a priority.
1. The usage of the dashboards is going to go up drastically with this improvement
2. Nonetheless, adding a layer of abstraction is a very important, and significant point towards maintainability. 
5. Kimiko R - ESP Solutions:  Modularization of Ed-Fi Core components to support the maintenance of current Ed-Fi visualizations while also allowing a gradual move of use of other available visualizations. )
6. Anonymous Recommendations: Analytics Middle tier should be not only a set of views but should also have its own data structure similar to the DDS. Then it would be possible to store a history of facts and also be useful to create custom visualizations.
7. Anonymous Recommendations:
1. A common middle tier should build on the analytics middle-tier project
2. Simplify existing metrics so they are not separately defined for classroom, school & district
3. Keep technology simple: SQL, Simple API (Security), Client vueJS or React, chart library
4. Open-source technologies
8. Chris Sprague - Eduphoric: New Middle Tier
1. Foundational "views" a la Stephen's star schema: 
   1. Should not break on standard change.
   2. Should use only standard SQL
   3. Should include row-level security columns like Lea, school, section, student.
   1. Responsive???
   2. Question whether new dashboards should be use-case specific rather than the kitchen sink?
2. User-space "views" that are community contributed and are solution or use-case specific. However, they follow conventions for:
   1. Versioning
   2. State
   3. Maintain row-level security relationship
   4. Ideally an exchange
   5. With UI on top of foundational views
3. Old Dashboards
   1. Responsive???
   2. Question whether new dashboards should be use case-specific rather than kitchen sin
9. Happy Miller - Rio Rancho Public Schools NM: There is a real need for plug-and-play, low-o-no-cost way o display data for small and mid-sized districts. This sound like it is a ways off but hopefully, actions taken now can lead in that direction. If not, or actually in combination with; we need to continue working on data out strategies and get vendors like Illuminate willing to ingest the data to feed visualizations there. 
10. Erin - Los Alamos Public Schools: I'm from a small district with little technical know-how programming-wise. It would be great to have a basic visualization package to view some data in the ODS. Or instructions for how to work with Google Data Studio or R or something free. I love the idea of a university partnership but what research says and what principals/teachers use can be very different. 
11. Carolynn Fallon - Volusia County Schools:
1. Analytics Middle Tier (AMT)
2. Community Assistance
3. If .NET, then responsiveness and connection to AMT.
4. If Community, then grants/support for the community to provide solutions.
5. In my opinion, every district/state will have a different skillset available in their region. Some will do MS.NET, others PostgreSQL , tableau and others Azure stack w/PowerBI. My hope is that the eventual 5-year direction will be able to get the community to the point to do machine learning/predictive analytics. I think the community is the key but we just might need some help to start.
12. Jeremy - ESP Solutions: In support of Kimiko's & Glyn's ideas on the future. I will say that one of the biggest fears I sense (hear) from districts is "is this ANOTHER software/solution that will disappear in a year or two". People are pretty apprehensive to trust the longevity of Ed-Tech solutions so I think all of us having the same idea of the future plan will help buy-in at all levels. - Jeremy ESP Solutions

13. David Hefley - Nebraska Dept of Education: During our discussion, it occurred to me that a lot of the issues that were brought up came down to solving immediate problems.  The issue of ADA/508 compliance, dashboard modernization to responsiveness, and even if it should be called a "dashboard" all centered around concerns with usability issues of the present.  While these are certainly important and should be addressed, any effort and capital spent on solving them will only be temporary and require a future outlay of hours and dollars.  

    For example, if we address the ADA issue by imposing a minimum compliance to the WCAG AA standards, those will only last until the community decides they want WCAG AAA standards (or the federal government mandates it or some other standard) or if we make the dashboard responsive for today's devices, what happens when the next display style comes along that requires another refactoring of the code (smartwatches, 3D, VR?).  This is only compounded by any issues there might be with the tight integration (i.e. the lack of separation) between the dashboard and the data behind it. Ultimately, it is just kicking the can down the road for a future working group to solve!

    On the flip side, investing time and money into inserting an abstraction layer that makes the data easier to both retrieve and conceptualize and (hopefully) also removes any specific technology requirements (i.e. it is written in XYZ language), will ensure that a more vibrant ecosystem around reporting and visualizations can be sustained.   Schools, districts, state agencies, etc all have access to different knowledge bases, so anything that promotes the ability to use existing technology and knowledge will go a long way to reducing the burden placed on those institutions to implement a new solution.  Although it is much easier to point to a pretty visualization and explain how that helps, it is much harder to get people excited about this foundational element.  However, by building this model it could allow an open-source library of visualizations to be created and shared. At that point, visualization plugins/modules could/would be created both by the community at large and through a variety of other providers.  Ideally, then, any "dashboard" that gets created would be more a factor of picking which widgets to include to meet your needs and could vary depending on your role(administrator vs teacher, for example).  This, then, leads to a much more flexible and agile solution that can continue to evolve over time with less lost effort.  

    An abstraction layer would also set the standard so that entities would not fear investment in implementation since they would know that no matter the changes to the underlying infrastructure, their hours and dollars would still be viable and not need to refactor code for the new releases.  Of course, I think a lot of thought needs to be put in around versioning of this to ensure a smooth transition up through versions, but that is fairly standard practice.

    So, basically, I'm all for the middle tier architecture and think time and effort are better invested there since it will ultimately make all other visualization outcomes more flexible, easier to implement, and sustainable.  
14. Michael Minuto - MSDF

Next Steps:

• Review working session notes and send your feedback if you didn't do so during the working session.
• Welcome new members - Itzel Torres  to schedule onboarding session
• Itzel Torres will send out a new meeting time survey for general and subgroup sessions now that we have new members.

Next  meeting: Date and time – TBD