Apache Hadoop : Sqoop Script for Importing Data RDBMS to HDFS and RDBMS to HIVE

Sqoop is a tool designed to transfer data between Hadoop and relational database servers. It is used to import data from relational databases such as MySQL, Oracle to Hadoop HDFS, and export from Hadoop file system to relational databases. This is a brief tutorial that explains how to make use of Sqoop in Hadoop ecosystem.


Sqoop: “SQL to Hadoop and Hadoop to SQL”

Sqoop Import

 

 

 

Sqoop import command imports a table from an RDBMS to HDFS. Each record from a table is considered as a separate record in HDFS. Records can be stored as text files, or in binary representation as Avro or Sequence Files.

Importing a RDBMS to HDFS

Syntax:

$ sqoop import --connect --table --username --password --target-dir -m1

 

--connect        Takes JDBC url and connects to database (jdbc:mysql://localhost:3306/test )

--table             Source table name to be imported (sqooptest )

--username    Username to connect to database (root )

--password     Password of the connecting user(12345)

 --target-dir     Imports data to the specified directory (/output )

--m1

 sqoop import --connect jdbc:mysql://localhost:3306/ecafe --table mm01_billing --username root --hive-import --hive-table mm01_billing --target-dir  /apps/hive/warehouse/mm01_billing -m 1

 sqoop
 import --connect jdbc:mysql://localhost:3306/ecafe --table mm01_billing
 --username root --hive-import --hive-table mm01_billing --target-dir  
/apps/hive/warehouse/mm01_billing -m 1

Posted by : DataScience99.com 03:33 Tags : Sqoop

Subscribe by Email

Follow Updates Articles from This Blog via Email