Retail · Finance
Automated bulk ingestion by day/hour
Classify transaction · log · snapshot data by type and run hundreds of workflows automatically each day. Failed stages identified instantly in statistics reports.
PADION
EoH · ETL on Data lakehouse
Workflow · Type · SQL · Stats
Workflow-based ETL,
on the lakehouse
Define data characteristics, and the workflow automatically executes the matching load strategy. SQL-based ingestion avoids update/delete performance degradation; operational statistics all live in one place.
Workflow DAG (example)
Type definition → parallel transform → SQL load → lakehouse
What is
Why type-based ETL?
Traditional ETL accumulates per-source scripts, and update/delete operations slow down the lakehouse. Ops teams burn time tracing what failed when, every day.
PADION EoH defines data characteristics as types and applies the load strategy automatically by type. A workflow graph visualizes sequential/parallel execution; SQL-based ingestion avoids update/delete performance degradation.
Statistical reports — load volume, runtime, failure rate — are generated automatically so operators can see everything at a glance.
Key Features
4 Key Features
Workflow sequential · parallel ETL + scheduling
Define jobs as a DAG-based workflow. Dependent stages run sequentially; independent stages run in parallel automatically. Built-in cron scheduling.
Data type definition + per-type ingestion
Define data characteristics as types (e.g., transaction/event/log/snapshot). Each type maps to its load strategy automatically.
SQL-based ingestion
In lakehouse environments where update/delete is costly, SQL-based load avoids performance hits. ETL code accumulates as reusable SQL assets.
Rich statistical reports
Auto-aggregate ops metrics — load volume, runtime, failure rate, per-stage timing. Daily/weekly/monthly reports.
How it Works
Type definition → Workflow → Ingestion
1
Type definition
Classify data by characteristics. Distinguish data with different load patterns — transaction, event, log, snapshot, etc.
2
Workflow definition
Define sequential/parallel flow as a DAG. Specify SQL · transform function · dependencies per stage. Register schedule.
3
Execution + Statistics
SQL load → lakehouse. Execution results auto-aggregated into operational statistics.
Use Cases
Deployment Scenarios
Telco · AI/ML
ML training data pipeline
EoH standardizes and ingests training data for analyze into the lakehouse. Workflow stages guarantee training-quality consistency.
Tech Spec
Tech Spec
| Execution model | DAG-based workflow (sequential + parallel) |
|---|---|
| Scheduling | cron expression, dependency triggers |
| Ingestion method | SQL-based (avoids update/delete performance hits) |
| Data types | Freely defined — transaction · event · log · snapshot, etc. |
| Statistics reports | Load volume · runtime · failure rate · daily/weekly/monthly (auto) |
| PADION integration | data lakehouse (ingestion target) |
In the PADION Flow
Transform step — 06
Orchestrated by flowkeeper, EoH performs ingest · transform · load within that flow.
Not scripts —
type-based ETL.
PoC, workflow migration, and lakehouse integration.