How To: Configure Assessment Metrics

The Ed-Fi Dashboards have a number of configuration points and data templates for displaying assessment metrics collectively called the Dynamic UI. This article provides a conceptual overview along with how-to information that can help you understand and use the capabilities of the Assessment Configuration options supported by the Metric Configuration Tool.

The Assessment configuration templates provide the ability to:

• Select student assessments for inclusion in a pre-defined metric calculation. Implementers have the ability to select the assessment category, academic subject area, metric instance value reporting type, and/or additional reporting methods to display.
• Configure the column display type based on pre-defined options such as text, trend, and status.
• Configure which drill-downs to include for the selected metrics.

The Metric Configuration Tool provides a user interface for all the required ETL and dashboard UI settings necessary for the application to populate assessment metrics data. See the ETL - Developers' Guide - Assessment Configuration section for more information.

Examples

The Assessment Configuration options provide a great deal of flexibility, so it's useful to go through a few examples. The following examples show the capabilities (and limits) of the Assessment Configuration.

Example 1. District Assessment

In this example, we'll look at district-level assessments in the Dashboard UI, followed by a breakdown of the different configuration points. 

Each numbered point in the screen capture is described in detail below:

1. District State Assessment (Container Metric). Metrics under this container are using the Rate Translator, all granular metrics below it inherit this configuration. Most of the district-level metrics are rate-type metrics that represent percentages of students in the district that met the defined goal for the metric.
   
2. District State Assessment Non-Participation (Container Metric). Metrics under this container are using the Non-Participation Translator. This metric calculation is specific to assessment non-participation, but it also calculates a rate based on the established non-participation business rules.
   
3. District State Assessment - ELA/Reading Granular Metric. At this level, the goal is configurable and the list of metrics to roll up to the district is also configurable. For this particular example, the goal is configured at 80% and the metric that rolls up to the district is "234 - School State Assessment - ELA/Reading".
   
4. District State Assessment Non-Participation - Mathematics. For this example, the goal is configured at 5% and the metric that rolls up to the district is "411 - School State Assessment Non-Participation - Mathematics".
   
5. The goals for each of these metrics is configured at the granular level. Examples of these can be seen in 2 and 3 above, under the Assessment tab in the Metric Configuration Tool.

Example 2. School Assessments

In this example, we'll look at school-level assessments in the Dashboard UI, followed by a breakdown of the different configuration points.

The numbered points in the screen capture are described below:

1. School Benchmark Assessment (Container Metric). In this example, the metric calculation is based on the percentage Translator type. Similar to the district sample above, metrics configured with this type calculate the percentage of students meeting the defined metric goal.

2. School Benchmark Assessment - Science (Granular Metric). This sample granular metric is configured to have a goal of 70% (labeled 3 in the screen capture above) and rolls up the results of metric "76 - Student Benchmark Assessment - Science" in its calculations.

Example 3. Student-Level Performance Based Assessments

Performance-based assessments allow users to configure the UI aggregate title columns based on the provided configuration. The UI allows the column order, column header text, and supports options for how to render the information in each cell.

The numbered points in the screen capture are described below:

1. Student State Standardize Assessments (Aggregate Metric). For these types of metrics, the UI is configured at the aggregate level. This allows all container and granular metrics to match the configured display columns. In this example, the Raw Score (1) is based on the selected reporting method and the Performance Level is a default column that always shows the reported performance level.
2. State Assessment Performance (Container Metric). This type of container metric inherits the assessment translator type of it's aggregate parent.
3. Raw Score column. The Raw Score column title, order and style of rendering is configurable. In this example, the title matches the selected reporting method type but can be changed to any desired text. The display order is 1 and the style of rendering has been set to status showing red/green based on the performance-level-met flag.
4. Performance Level column. The Performance Level column is always shown by this template. The title is configurable and the order is set to 2, and the style of rendering is simple text. 

For more information on how this template calculates metrics, see ETL Developer's Guide - Assessment Metrics Based on Performance Level.

Once the performance calculation template (i.e., the ETL Application Translator) is selected, the option to configure the UI will appear. Clicking on this option will display the options to configure the title, cell rendering type, and display order. The extended property is assigned following the ETL convention discussed in the ETL Developers' Guide referenced above – and this property is what merges the ETL calculations with the dashboard UI display.

At the granular metric level, the category and subject area are configured. For the example shown above, the Reading granular metric inherits the Performance calculation and the reporting method type setting from its parent metric. The categories included for "State Assessment Performance - Reading" are: "State summative assessment 3-8 general", "State high school subject assessment", and "State high school assessment for the subjects of Reading and English Language Arts".

Assessment categories and subject areas can be added or removed at this granular metric level.

Example 4. Student-Level Percentage-Based Assessments

The percentage assessment types measure the maximum possible score against the configured goal. The template for percentage-based assessments follows the core metric template – therefore no UI configuration is required.

Much like the performance example above, once the assessment calculation type (i.e., the ETL Translator) is selected, the metrics below this aggregate will inherit these settings. At the granular metric level, users can select what assessment categories and subject areas to include. In this example, the "Benchmark Assessment Mastery - Social Studies" includes assessments with a category of Benchmark test within the subject area of social studies.

The state of the metric value is then defined by the provided goal. Values equal to or above the defined goal of 70% in this sample will have a state of good/green. Values below 70% will display as bad/red. 

For more information on assessment percentage calculations see ETL Developers' Guide - Assessment Metrics Based on a Percentage Threshold.

Example 5. Learning Mastery Standard

The assessment learning mastery standard metric type calculates the number of learning standards correct against the number of learning standards assessed. Calculations are based on the configured assessment category and academic subject areas. Assessment metrics based on learning standards have no configurable UI options. Metrics are displayed using the default metric template.

The learning standard assessment calculation type (i.e., the ETL Translator) is selected at the aggregate level and the metrics below inherit these settings. At the granular metric level, users can select the assessment categories and subject areas to include. In this example, "Learning Standard Mastery by Core Subject Area - Science" includes assessments with a category of Benchmark test in the subject area of science.

The state of the metric value is then defined by the provided goal. Values equal to or above the defined goal of 70% in this sample will have a state of good/green. Values below 70% will display as bad/red. 

For more information on assessment learning standard calculations, see ETL Developers' Guide - Assessment Metrics Based on Learning Standards. 

Key Concepts

How that we've been through a few examples, let's look at a few key concepts. The following are concepts useful to understand when working with assessment metric configuration:

• Aggregate metrics. Aggregate metrics are a subcategory grouping for metrics. These are identified as the gray headers in the dashboard application.
• Container metrics. Container metrics are the main metric and a holder for other metrics. These usually provide the context and a brief description of the metric calculation. 
• Granular metrics. Granular metrics provided detailed metric calculations and can be reporting-period based (e.g., Last 20 days, Last 40 days) or subject-area based (e.g., Science, ELA). At this level, the dashboard usually includes more menu options to drill into the detailed data behind the metric.
• Assessment Calculation Types. Also referred to as ETL Translators since the responsibility of the calculation falls into that part of the architecture. However, it helps to think of these as Assessment Calculation templates. Each type of assessment translator has its own set of rules for calculating the metrics and knows to look for specific configurable criteria to include as part of the calculations. More technical background on each type of ETL Translator can be found in the ETL Developers' Guide - Assessment Configuration.