[HIVE-18117] Create TestCliDriver for HDFS EC

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Target Version/s: 3.0.0
• Assignee: Andrew Sherman
• Reporter: Sahil Takiar
• Created: 21/Nov/17 16:48
• Updated: 27/Feb/24 22:23
• Resolved: 25/May/18 20:20

Description

Should be able to do something similar to what we do for HDFS encryption.

TestErasureCodingHDFSCliDriver uses a test-only CommandProcessor "ErasureProcessor"
which allows .q files to contain Erasure Coding commands similar to those provided
by the hdfs ec command
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html.
The Erasure Coding functionality is exposed through a new shim "HdfsFileErasureCodingPolicy".
At this stage there are two .q files:
erasure_commnds.q (a simple test to show ERASURE commands can run on local fs via
TestCliDriver or on hdfs via TestErasureCodingHDFSCliDriver), and
erasure_simple.q (which does some trivial queries to demonstrate basic functionality).
More tests will come in future commits.

 

Issue Links

• is duplicated by: HIVE-17776 Add CliDriver and tests for HDFS with Erasure Coding enabled