Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14269 Performance optimizations for data on S3
  3. HIVE-14270

Write temporary data to HDFS when doing inserts on tables located on S3

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3.0
    • Component/s: None
    • Labels:
      None

      Description

      Currently, when doing INSERT statements on tables located at S3, Hive writes and reads temporary (or intermediate) files to S3 as well.

      If HDFS is still the default filesystem on Hive, then we can keep such temporary files on HDFS to keep things run faster.

        Attachments

        1. HIVE-14270.1.patch
          8 kB
          Sergio Peña
        2. HIVE-14270.2.patch
          19 kB
          Sergio Peña
        3. HIVE-14270.3.patch
          20 kB
          Sergio Peña
        4. HIVE-14270.4.patch
          15 kB
          Sergio Peña
        5. HIVE-14270.5.patch
          17 kB
          Sergio Peña
        6. HIVE-14270.6.patch
          16 kB
          Sergio Peña

          Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-21514 Hive has updated with new support for S3 and InsertIntoHiveTable.scala should update also

          • Major - Major loss of function.
          • Open
          links to

          Web Link Review Board

            Activity

              People

              • Assignee:
                spena Sergio Peña
                Reporter:
                spena Sergio Peña
              • Votes:
                0 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: