ETL Process
ETL is defined as Extract, Transform, and Load. It is the set of processes by which data is extracted from heterogeneous databases, applications and systems, transformed as appropriate and loaded into
target systems - including, but not limited to, data warehouses, data marts, analytical applications, etc.
- Extract/Extraction is the process of reading data from source systems.
- Transform/Transforming is the process of converting the extracted data from its current form into the required form of the destination system. The data is transformed by using rules, lookup tables and/or by combining the data with other data.
- Load/Loading is the process of writing the data into the destination database.
- It is very advantageous to use an ETL tool in the following scenarios:
- When there are multiple source systems to be integrated.
- When source systems are in different databases and formats.
- When this process needs to be run repeatedly (e.g. daily, hourly, real time)
DataPals Consulting Services
- Define the ETL architecture
- Defining Data sources and relevant mappings
- Implementing accurate and efficient ETL process Developing and implementing transformation scripts for new business requirements
- Maintenance/monitoring of transformation scripts
- Ensuring accurate/efficient ETL processes in production
- Fixing of broken ETL processes
- Updating ETL processes as per business need
- Streamlining/ elimination of disjointed extraction and transformation programs
- Validation routines to check data consistency between source and destination systems
- Performance tuning, error handling
Tools and technologies
- Informatica PowerCenter
- IBM InfoSphere DataStage
- IBM – Cognos Data Manager
- SAP BODS (Business Object Data Services)
- Oracle Warehouse Builder / Data Integrator
- SAS ETL Studio
- Microsoft SQL Server Integration Services (SSIS)
- Pentaho Data Integration
- Talend Open Studio for Data Integration
Our Expertise
