Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-7292 Hive on Spark
  3. HIVE-8503

UT: add TestSparkMinimrCliDriver to run UTs that use HDFS

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Spark
    • Labels:
      None

      Description

      The tests

      empty_dir_in_table
      file_with_header_footer
      import_exported_table
      load_hdfs_file_with_space_in_the_name
      external_table_with_space_in_location_path
      load_fs2
      remote_script
      root_dir_external_table
      schemeAuthority
      schemeAuthority2
      temp_table_external
      udf_using

      create directories in HDFS, but TestSparkCliDriver is not using HDFS.

      That's why the tests fail when creating a dir, for example mkdir -p hdfs:///tmp/temp_table_external
      In the non-spark case, these tests are executed with the TestMinimrCliDriver which initializes Hadoop and created a mini-cluster
      (see configuration in itests/qtest/pom.xml).

      We may need to configure a TestSparkMinimrCliDriver in itests/qtest-spark/pom.xml if we want to run these test cases.

        Attachments

          Activity

            People

            • Assignee:
              tfriedr Thomas Friedrich
              Reporter:
              tfriedr Thomas Friedrich
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: