Apiome Forge
A visual ETL studio where every pipeline is versioned with your schema.
Apiome Forge is a drag-and-drop DAG pipeline studio for building source-to-transform-to-target flows without hand-written Spark. Pipelines are versioned alongside apiome schemas, so changing a class definition flags dependent pipelines for review, and execution runs on embedded DataFusion or cluster-scale Apache Spark.
What ships with Forge
Every Forge surface is wired into the rest of the Apiome platform — no glue code, no separate identity, no bolt-on integrations.
Drag-and-drop DAG canvas
Compose source, transform, and target nodes on a grid canvas with SVG edges and a live save state.
SQL / DataFusion transform editor
Author each transformation node with expression editing, live schema hints, and row preview.
Source connectors
Configure inbound Postgres, S3/CSV, REST API, Kafka, and BigQuery sources with schema preview.
Target loaders
Map pipeline output columns to dynamically generated apiome-db tables with an upsert strategy.
Dual execution engines
Run embedded workloads on DataFusion and cluster-scale payloads on Apache Spark, selected per pipeline.
Inline row-level debugger
Step through each DAG node to inspect column values and transformation deltas as data flows.
A look inside Apiome Forge
Live design previews from the Forge mockup pack — 9 surfaces in total.
SQL / DataFusion expression editor with live schema hints and row-level preview for a node.
Run dashboard with rows in/out, engine used, duration, error traces, and re-run controls.
Step-through, row-level inspection at each DAG node showing column values and transformation deltas.
Use cases
Forge is designed around the way real teams actually work — not the way a tool wants them to work.
Build CDC and enrichment pipelines visually instead of maintaining hand-written Spark jobs.
Ship schema changes safely — dependent pipelines are automatically flagged for review.
A/B test two transform variants on row counts, schema drift, and latency before promoting one.
- Pipelines versioned with schemas; class changes flag dependents for review
- Dual DataFusion / Spark execution with per-workload engine selection
- Full run history with row counts, engine, duration, and error traces
- A/B variant testing with schema-drift and latency comparison
- Curated template library for CDC, ETL, enrichment, and migration
- Generate DataFusion / SQL transforms from a natural-language goal
- Suggest a pipeline from source and target schemas
- Explain and fix failing rows in the debugger
- Recommend the right execution engine for a workload
AI that writes your transforms
Forge ships with an AI copilot that turns plain-language goals into DataFusion transforms, suggests whole pipelines from your schemas, and explains failing rows in the debugger.
Every ForgeAI feature is grounded in your tenant's data, runs under your data-residency policy, and respects every role and ACL the platform enforces.
Every surface in Apiome Forge
A look at the 9 screens designed for this suite — covering everything from day-1 onboarding to day-100 operations.
