Apiome Stream
Real-time ingest from Kafka, Kinesis, Pulsar, and MQTT into your Classes.
Apiome Stream ingests real-time events from Apache Kafka, AWS Kinesis, Apache Pulsar, and MQTT into apiome-db. It maps topics to Classes with configurable field-mapping rules and schema-evolution policies, supports schema-on-read and schema-on-write modes, and adds micro-batching, per-partition lag monitoring, and a dead-letter queue for failed events.
What ships with Stream
Every Stream surface is wired into the rest of the Apiome platform — no glue code, no separate identity, no bolt-on integrations.
Multi-broker connectivity
Manage credentials, test connectivity, and tune per-broker parallelism across Kafka, Kinesis, Pulsar, and MQTT.
Visual topic mapper
A three-pane configurator pairs a live event-stream preview with field-to-Class-property mapping and schema mode selection.
Schema-on-read or on-write
Validate events against the registered Class before writing, or defer typing to read time — per topic.
Schema registry & evolution
Track Avro, Protobuf, and JSON schemas per topic with compatibility levels, version history, and a diff viewer.
Lag & throughput monitoring
Per-partition lag, throughput, and P50/P95/P99 end-to-end latency, plus a cross-group lag heatmap with SLA alerts.
Dead-letter queue & replay
Capture failed events by cause and replay to topic, re-process inline, or discard with a full audit log.
A look inside Apiome Stream
Live design previews from the Stream mockup pack — 9 surfaces in total.
Live consumer-group dashboard with events/sec and bytes/sec throughput, total lag, DLQ count, and partition health.
Dead-letter queue triaging parse, schema-mismatch, and data-quality failures with replay, re-process, or discard.
Broker connection manager with health status and connectivity tests across Kafka, Kinesis, Pulsar, and MQTT.
Use cases
Stream is designed around the way real teams actually work — not the way a tool wants them to work.
Land high-volume event streams into typed Classes with back-pressure-aware micro-batching and no custom consumers.
Watch per-partition lag and latency percentiles in real time and get alerted before an SLA breach.
Map a new Kafka topic to a Class visually, previewing sample events and inferred fields before going live.
- Per-tenant broker isolation and configurable authentication methods
- Schema-evolution policies with unknown-field routing to the DLQ
- SLA breach thresholds with cross-group lag alerting
- Exactly-once micro-batching with configurable flush intervals and late-arrival windows
- Full replay and audit trail for every dead-lettered event
- Infer field types from live event samples and suggest Class property mappings
- Recommend schema-evolution policies when new fields appear on a topic
- Flag anomalous lag, throughput, and error-rate patterns per partition
- Cluster dead-letter failures by root cause to speed up triage
Field mapping and anomaly detection, assisted
Stream infers event structure from live samples and proposes topic-to-Class mappings, then watches ingest health to surface lag spikes and schema drift before they cause data loss.
Every StreamAI feature is grounded in your tenant's data, runs under your data-residency policy, and respects every role and ACL the platform enforces.
Every surface in Apiome Stream
A look at the 9 screens designed for this suite — covering everything from day-1 onboarding to day-100 operations.
