Meadowlark - Streaming to Filesystem

Owned by Stephen Fuqua
Last updated: Jul 06, 2022
1 min read

Things to consider

  • Kafka out to filesystem without additional stream processor?
  • For on-premises, probably a container reading from Kafka and writing to a mounted volume - thus the code treats as a Linux volume, even if hosted in Windows
  • Also demonstrate S3 output
  • Probably use Parquet format
  • Some kind of demonstration using the files
    • QuickSight
    • PowerBI
    • Python

placeholder