Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-7201

Fix TestHiveConf#testConfProperties test case

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.13.1
    • Fix Version/s: 0.14.0
    • Component/s: Tests
    • Labels:
      None

      Description

      CHANGE 1:

      TEST CASE :
      The intention of TestHiveConf#testConfProperties() is to test the HiveConf properties being set in the priority as expected.

      Each HiveConf object is initialized as follows:
      1) Hadoop configuration properties are applied.
      2) ConfVar properties with non-null values are overlayed.
      3) hive-site.xml properties are overlayed.

      ISSUE :
      The mapreduce related configurations are loaded by JobConf and not Configuration.
      The current test tries to get the configuration properties like : HADOOPNUMREDUCERS ("mapred.job.reduces")
      from Configuration class. But these mapreduce related properties are loaded by JobConf class from mapred-default.xml.

      DETAILS :
      LINE 63 : checkHadoopConf(ConfVars.HADOOPNUMREDUCERS.varname, "1"); -->fails
      Because,
      private void checkHadoopConf(String name, String expectedHadoopVal) {
      Assert.assertEquals(expectedHadoopVal, new Configuration().get(name)); ----> Second parameter is null, since its the JobConf class and not the Configuration class that initializes mapred-default values.
      }

      Code that loads mapreduce resources is in ConfigUtil and JobConf makes a call like this (in static block):
      public class JobConf extends Configuration {

      private static final Log LOG = LogFactory.getLog(JobConf.class);

      static

      { ConfigUtil.loadResources(); --> loads mapreduce related resources (mapreduce-default.xml) }

      .....
      }

      Please note, the test case assertion works fine if HiveConf() constructor is called before this assertion since, HiveConf() triggers JobConf()
      which basically sets the default values of the properties pertaining to mapreduce.
      This is why, there won't be any failures if testHiveSitePath() was run before testConfProperties() as that would load mapreduce
      properties into config properties.

      FIX:
      Instead of using a Configuration object, we can use the JobConf object to get the default values used by hadoop/mapreduce.

      CHANGE 2:
      In TestHiveConf#testHiveSitePath(), a call to static method getHiveSiteLocation() should be called statically instead of using an object.

        Attachments

        1. HIVE-7201.03.patch
          1 kB
          Pankit Thapar
        2. HIVE-7201.04.patch
          1 kB
          Pankit Thapar
        3. HIVE-7201.patch
          1 kB
          Pankit Thapar
        4. HIVE-7201-1.patch
          1 kB
          Pankit Thapar
        5. HIVE-7201-2.patch
          1 kB
          Pankit Thapar

          Activity

            People

            • Assignee:
              pankit Pankit Thapar
              Reporter:
              pankit Pankit Thapar
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: