Rakesh Randeria
Architecture & Integration

Data Integration in Enterprise Systems

APIs, EDI, files, events and data pipelines — choosing the right pattern and keeping it supportable

A practical enterprise integration playbook covering APIs, EDI, file exchange, events, databases and data pipelines, with emphasis on reliability, reconciliation and operational ownership.

Process map

SourceERP / CRM / WMS
partners / apps
ExchangeAPI / EDI / file
event / DB
ValidateSchema / format
business rules
TransformMap / enrich
standardise
DeliverTarget system
platform / partner
ReconcileAck / totals
exceptions
MonitorFailures / latency
ownership
ImproveRoot cause
simplify / automate

Choosing the right integration pattern

  • API — interactive or near-real-time system-to-system exchange where reusable services and responsive interactions matter.
  • EDI — structured B2B transaction exchange across supplier, customer, distributor, carrier and logistics ecosystems.
  • File / SFTP — scheduled batch exchange where simplicity, auditability and broad compatibility are important.
  • Events / messaging — asynchronous workflows and decoupled processing where resilience and scale matter.
  • Database integration — controlled internal use cases where direct data-layer access is appropriate and tightly governed.
  • ETL / ELT — bulk movement into reporting, analytics and data platforms where transformation, history and lineage are important.

EDI in modern enterprise integration

  • EDI remains a core integration pattern for structured B2B transactions, particularly where organisations exchange high-volume, repeatable business documents with suppliers, customers, distributors, carriers and logistics partners.
  • Common standards include ANSI X12 and UN/EDIFACT. XML-based industry schemas and partner-specific implementation guides are also common in modern integration estates.
  • Typical message types include purchase orders, order acknowledgements, invoices, advance shipping notices/despatch advices, inventory reports, remittance advice, shipment status messages and functional acknowledgements.
  • Widely recognised X12 transaction sets include 850 Purchase Order, 855 Purchase Order Acknowledgment, 856 Advance Ship Notice, 810 Invoice, 846 Inventory Inquiry/Advice, 820 Payment Order/Remittance Advice, 214 Transportation Carrier Shipment Status and 997 Functional Acknowledgment.
  • Comparable EDIFACT messages include ORDERS, ORDRSP, DESADV, INVOIC, INVRPT, REMADV and CONTRL.
  • EDI's strength is its maturity: well-defined transaction structures, partner agreements, acknowledgements and repeatable operational workflows make it highly effective for supply-chain and commercial processes.
  • EDI increasingly coexists with APIs rather than competing with them. APIs may support real-time enquiries or service interactions, while EDI remains well suited to structured transaction exchange and partner ecosystems.

Design principles

  • Choose integration style based on business timing and failure tolerance, not fashion.
  • Avoid unnecessary point-to-point connections where a shared integration layer or standard interface reduces complexity.
  • Define ownership for both successful processing and failed transactions.
  • Build reconciliation and observability into the design rather than treating them as support add-ons.
  • Treat master/reference data, data quality and schema ownership as integration concerns.
  • Document upstream/downstream dependencies so change in one system does not create silent failures elsewhere.

Operational controls

  • Authentication, encryption and least-privilege access.
  • Message, file and API logging with traceable correlation or transaction IDs.
  • Retry, dead-letter or exception handling paths.
  • Acknowledgements and reconciliation where business completion matters.
  • Monitoring for latency, backlog, volume anomalies and failed transformations.
  • Defined escalation paths across internal teams, vendors and trading partners.
  • Change/version control for schemas, mappings and interfaces.

Key lessons

  • Integration is an operational dependency, not simply a technical connector.
  • The simplest pattern that meets the business requirement is usually the most supportable.
  • EDI remains a strategic integration pattern for structured B2B flows and often complements APIs in modern enterprise architectures.
  • Monitoring and reconciliation are part of the integration design, not post-go-live support.
View detailed playbook on GitHubBack to Resources