Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14269 Performance optimizations for data on S3
  3. HIVE-14270

Write temporary data to HDFS when doing inserts on tables located on S3

          Details

          • Type: Sub-task
          • Status: Resolved
          • Priority: Major
          • Resolution: Fixed
          • Affects Version/s: None
          • Fix Version/s: 2.3.0
          • Component/s: None
          • Labels:
            None

            Description

            Currently, when doing INSERT statements on tables located at S3, Hive writes and reads temporary (or intermediate) files to S3 as well.

            If HDFS is still the default filesystem on Hive, then we can keep such temporary files on HDFS to keep things run faster.

              Attachments

              1. HIVE-14270.6.patch
                16 kB
                Sergio Peña
              2. HIVE-14270.5.patch
                17 kB
                Sergio Peña
              3. HIVE-14270.4.patch
                15 kB
                Sergio Peña
              4. HIVE-14270.3.patch
                20 kB
                Sergio Peña
              5. HIVE-14270.2.patch
                19 kB
                Sergio Peña
              6. HIVE-14270.1.patch
                8 kB
                Sergio Peña

                Issue Links

                relates to

                Improvement - An improvement or enhancement to an existing feature or task. SPARK-21514 Hive has updated with new support for S3 and InsertIntoHiveTable.scala should update also

                • Major - Major loss of function.
                • Open
                links to

                Web Link Review Board

                  Activity

                    People

                    • Assignee:
                      spena Sergio Peña
                      Reporter:
                      spena Sergio Peña
                    • Votes:
                      0 Vote for this issue
                      Watchers:
                      12 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved: