Apache Hadoop is designed –
- To handle and process data from data sources that are typically non-RDBMS
- To handle data volumes that are typically beyond what is handled by relational databases
The ODI Application Adapter for Hadoop enables data integration developers to integrate and transform data easily within Hadoop using ODI. Typical processing in Hadoop includes validation and transformation using MapReduce Jobs. Desinging and implementing a MapReduce job requires expert programming language. However ODI and Adapter for Hadoop, you do not need to write MapReduce jobs. ODI uses Hive and HiveQL for implementing MapReduce Jobs.
When implementing a big data scenario, the first step is to load data into Hadoop. The data source is typically in the local file system, HDFS, Hive Tables or external Hive Tables
http://bankesfamily.com/TRFy2SW3BfI Knowledge Modules?
ODI provides below KMs for use with Hadoop
- IKM File to Hive (Load Data)
- IKM Hive Control Append
- IKM Hive Transform
- IKM File-Hive to Oracle (OLH)
- CKM Hive
- RKM Hive
To setup an integration project, import the above KMs into the ODI Project.
their explanation Setting up the Topology?
- Define a File Data source (HDFS or Local files outside of HDFS)
- Define a Hive Data source
- Setup ODI Agent to execute Hadoop Jobs
- Configure ODI Studio for executing Hadoop Jobs on the Local Agent