Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2523

Make HdfsTableSink aware of clustered input

          Details

          • Type: New Feature
          • Status: Resolved
          • Priority: Critical
          • Resolution: Fixed
          • Affects Version/s: Impala 2.2, Impala 2.3.0, Impala 2.5.0, Impala 2.4.0, Impala 2.6.0, Impala 2.7.0
          • Fix Version/s: Impala 2.8.0
          • Component/s: Backend

            Description

            The HdfsParquetTableWriter needs to be aware that incoming data is clustered if the corresponding insert statement has a "clustered" hint. Only a single open partition should be maintained and flushed by calling FinalizePartitionFile() when the partition-key values change.

            For now, no changes should be made to other table sinks.

              Attachments

                Issue Links

                is blocked by

                New Feature - A new feature of the product, which has yet to be developed. IMPALA-2521 Introduce CLUSTERED plan hint for insert statements

                • Critical - Crashes, loss of data, severe memory leak.
                • Resolved

                  Activity

                    People

                    • Assignee:
                      lv Lars Volker
                      Reporter:
                      mmokhtar Mostafa Mokhtar
                    • Votes:
                      0 Vote for this issue
                      Watchers:
                      5 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved: