SpecMesh at Kafka Meetup, London, A Practical Guide: Building a Data Mesh on Apache Kafka. Jan 2022
A Kafka Meetup Tech talk that is targeted at Kafka Devs.
Summary
Kafka is great, it’s un-opinionated and lets you build whatever you want; however many times we need opinions! We need some guidance on establishing the fundamentals of topic-naming & structuring, and schemas. We also need a consistent means of provisioning topics, handling their change, updating schemas, testing, knowing who owns which topics, as well as many other things. SpecMesh OS brings all of this together under the banner of ‘data-mesh’. It provides a consistent means of modeling Kafka applications and resources as a ‘data product’, consistent tooling and automation to support governance and self-service that works with your tests and CI/CD. Think of it as an accelerator for doing things right when starting, building, and managing the ongoing lifecycle of your Kafka-based applications.
This talk deep dives into some of the common problems experienced on any Kafka project and how these problems should be avoided. Then it dives into Opensource automation and tooling that Specmesh uses to build enterprise level streaming applications
