HDFS, HIVE

HIVE Data-Copy – Import/Export

HIVE Data Copy:

 

With hive one better option to copy the Hive data and metadata from cluster A to cluster B is to use HIVE Import/Export

 

NOTE:  In HIVE CLI exporting the data directly to target cluster will work fine as the Path can be resolved, But Beeline won’t allow copying the data directly to the target cluster directly. To do via beeline, Export table to the local HDFS and copy the data to the target cluster and connect to target hiveServer 2 and call the import to import the data.

 

  • EXPORT TABLE tablename [PARTITION (part_column=”value”[, …])]

TO ‘export_target_path’ [ FOR replication(‘eventid’) ]

 

  • IMPORT [[EXTERNAL] TABLE new_or_original_tablename [PARTITION (part_column=”value”[, …])]]

FROM ‘source_path’

[LOCATION ‘import_target_path’]

 

 

Reference:

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ImportExport

Advertisements
HDFS

HDFS FEDERATION

Created to increase the namespace and Namenode load. If found that the number of files are growing, then the number of namespace will also grow and the same will grow the name space.

To federate the directories. say /finance is in one namenode and /hr is in another namenode. data node stores the data of both. The block pool managed for each namespace.

How to do it will be done on next post.