Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14269 Performance optimizations for data on S3
  3. HIVE-14270

Write temporary data to HDFS when doing inserts on tables located on S3

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.3.0
    • None
    • None

    Description

      Currently, when doing INSERT statements on tables located at S3, Hive writes and reads temporary (or intermediate) files to S3 as well.

      If HDFS is still the default filesystem on Hive, then we can keep such temporary files on HDFS to keep things run faster.

      Attachments

        1. HIVE-14270.1.patch
          8 kB
          Sergio Peña
        2. HIVE-14270.2.patch
          19 kB
          Sergio Peña
        3. HIVE-14270.3.patch
          20 kB
          Sergio Peña
        4. HIVE-14270.4.patch
          15 kB
          Sergio Peña
        5. HIVE-14270.5.patch
          17 kB
          Sergio Peña
        6. HIVE-14270.6.patch
          16 kB
          Sergio Peña

        Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-21514 Hive has updated with new support for S3 and InsertIntoHiveTable.scala should update also

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link Review Board

          Activity

            People

              spena Sergio Peña
              spena Sergio Peña
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: