[HIVE-1992] NCDFE: o.a.commons/configuration/Configuration with Hadoop 0.20.100

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 0.6.0
• Component/s: CLI

Environment

Red Hat 2.6.18

• Tags: Hadoop 0.20.100
• Assignee: Unassigned
• Reporter: Joep Rottinghuis
• Created: 14/Feb/11 22:01
• Updated: 17/Sep/14 16:57
• Resolved: 17/Sep/14 16:57

Description

When compiling Hive against Hadoop 0.20.100 all tests and CLI commands fail with NoClassDefFoundError: org/apache/commons/configuration/Configuration (see log output example below):
Hadoop classes depend on commons.configuration which is not available on classpath with Hive 0.6.

Sample test case failure (one of many):

test:
    [junit] Running org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore
    [junit] Unable to open the metastore
    [junit] java.lang.NoClassDefFoundError: org/apache/commons/configuration/Configuration
    [junit] 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<init>(DefaultMetricsSystem.java:37)
    [junit] 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<clinit>(DefaultMetricsSystem.java:34)
    [junit] 	at org.apache.hadoop.security.UgiInstrumentation.create(UgiInstrumentation.java:51)
    [junit] 	at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:196)
    [junit] 	at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:159)
    [junit] 	at org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:216)
    [junit] 	at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:409)
    [junit] 	at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:395)
    [junit] 	at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:1418)
    [junit] 	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1319)
    [junit] 	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:226)
    [junit] 	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:183)
    [junit] 	at org.apache.hadoop.hive.metastore.Warehouse.getFs(Warehouse.java:72)
    [junit] 	at org.apache.hadoop.hive.metastore.Warehouse.getDnsPath(Warehouse.java:104)
    [junit] 	at org.apache.hadoop.hive.metastore.Warehouse.getWhRoot(Warehouse.java:119)

As with HIVE-1990, I first applied HIVE-1264 and tweaked the build.properties to compile Hive against locally built Hadoop 0.20.100 (see HADOOP-7108).