Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-7455 ROLLUP: Make HCatalog tests generic for all storage formats
  3. HIVE-7286

Parameterize HCatMapReduceTest for testing against all Hive storage formats

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.14.0
    • HCatalog
    • None

    Description

      Currently, HCatMapReduceTest, which is extended by the following test suites:

      • TestHCatDynamicPartitioned
      • TestHCatNonPartitioned
      • TestHCatPartitioned
      • TestHCatExternalDynamicPartitioned
      • TestHCatExternalNonPartitioned
      • TestHCatExternalPartitioned
      • TestHCatMutableDynamicPartitioned
      • TestHCatMutableNonPartitioned
      • TestHCatMutablePartitioned

      These tests run against RCFile. Currently, only TestHCatDynamicPartitioned is run against any other storage format (ORC).

      Ideally, HCatalog should be tested against all storage formats supported by Hive. The easiest way to accomplish this is to turn HCatMapReduceTest into a parameterized test fixture that enumerates all Hive storage formats. Until HIVE-5976 is implemented, we would need to manually create the mapping of SerDe to InputFormat and OutputFormat. This way, we can explicitly keep track of which storage formats currently work with HCatalog or which ones are untested or have test failures. The test fixture should also use Reflection to find all classes in the classpath that implements the SerDe interface and raise a failure if any of them are not enumerated.

      Attachments

        1. HIVE-7286.9.patch
          33 kB
          David Chen
        2. HIVE-7286.8.patch
          49 kB
          David Chen
        3. HIVE-7286.7.patch
          49 kB
          David Chen
        4. HIVE-7286.6.patch
          47 kB
          David Chen
        5. HIVE-7286.5.patch
          48 kB
          David Chen
        6. HIVE-7286.4.patch
          48 kB
          David Chen
        7. HIVE-7286.3.patch
          48 kB
          David Chen
        8. HIVE-7286.2.patch
          47 kB
          David Chen
        9. HIVE-7286.11.patch
          31 kB
          David Chen
        10. HIVE-7286.10.patch
          33 kB
          David Chen
        11. HIVE-7286.1.patch
          56 kB
          David Chen

        Issue Links

          is depended upon by

          Sub-task - The sub-task of the issue HIVE-7420 Parameterize tests for HCatalog Pig interfaces for testing against all storage formats

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HIVE-4329 HCatalog should use getHiveRecordWriter rather than getRecordWriter

          • Major - Major loss of function.
          • Open

          Test - A new unit, integration or system test. HIVE-7440 Remove custom code for Avro in HCatMapReduceTest

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Task - A task that needs to be done. HIVE-7485 Add documentation on new Serde test fixtures

          • Major - Major loss of function.
          • Open
          relates to

          Test - A new unit, integration or system test. HIVE-7484 Format-independent fixture for testing storage formats

          • Major - Major loss of function.
          • Open

          Task - A task that needs to be done. HIVE-5976 Decouple input formats from STORED as keywords

          • Major - Major loss of function.
          • Closed

          Activity

            People

              davidzchen David Chen
              davidzchen David Chen
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: