Pipelines are extracted from your documents by specialized AI agents, or added manually by your team
AI-powered extraction
The agent reads integration documents and extracts pipeline definitions, source/target mappings, and scheduling information automatically.
Processes these document types:
Auto-Detection
Automatically identifies system connections
Manual entry & review
Data engineers and stewards can add pipelines directly or review and validate agent-extracted records.
Human actions:
Full Control
Every AI suggestion requires human approval
Every record carries a trust score based on its source document maturity level. High-maturity documents (like approved Technical Specifications) produce higher-trust inventory records.
High Trust
From approved integration specs, signed-off requirements
Medium Trust
From draft data flows, work-in-progress specs
Low Trust
From informal docs, emails, early drafts
Trust Score Distribution
Records start as "pending" and must be approved before they are used in downstream processing like lineage generation
Pending
Awaiting review
Use
Ready for lineage
Lineage
Connected to pipelines
Setting a pipeline record to "Use" enables it for lineage generation — connecting systems to visualize complete data flows.
Track data flows with source/target mappings, frequencies, and full traceability
| # | Trust | Action | Pipeline ID | Pipeline Name | Source System | Target System | Frequency | Source Documentation |
|---|---|---|---|---|---|---|---|---|
| 1 | High | Use | CRM-DWH-001 | CRM Customer Sync | CRM System | Data Warehouse | Daily | CRM Integration Spec v2.pdf |
| 2 | High | Use | ERP-DWH-001 | ERP Financial Extract | ERP Platform | Data Warehouse | Daily | ERP Data Pipeline Requirements.docx |
| 3 | Medium | Use | DWH-RPT-001 | Reporting Aggregation | Data Warehouse | Reporting Engine | Hourly | Reporting Pipeline Tech Spec.pdf |
| 4 | Medium | Pending | KYC-CRM-001 | KYC Customer Update | KYC System | CRM System | Real-time | KYC Integration Business Req.docx |
| 5 | Low | Pending | ATM-DWH-001 | ATM Transaction Feed | ATM System | Data Warehouse | Real-time | ATM Data Flow Specification.pdf |
| 6 | Low | Pending | DWH-ML-001 | ML Feature Pipeline | Data Warehouse | ML Pipeline | Daily | Manual entry via UI |
This is a preview — the real interface supports full CRUD operations
Pipelines define the connections — explore all four inventories that form your complete data landscape
See how IDA can automatically extract pipeline definitions from your integration documentation in a 30-minute demo.