Ed-Fi API to Data Marts


This repository contains code that pulls data from a target Ed-Fi API and creates data marts in BigQuery that serve the analytics use case.

Demonstration video: https://youtu.be/A1a7C9pDVL4

More specifically, this repository is a Dagster workspace that contains a job designed to:

  • Extract data from a set of Ed-Fi API endpoints
  • Store the data as JSON files in a cloud based blob storage service
  • Query the JSON files from a cloud data warehouse to produce:
    • Tables that represent the Ed-Fi API endpoints
    • Dimension and fact tables