Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8896

alterPartitionsWithTransaction() emits too many alter table operations

Agile BoardAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • Impala 3.4.0
    • None
    • None
    • ghx-label-13

    Description

      alterPartitionsWithTransaction() has a for loop that updates the write ids of the partitions.

      Unfortunately it also does an RPC inside the for loop that updates all partitions. This could be done outside of the for loop.

        public static void alterPartitionsWithTransaction(IMetaStoreClient client,
            String dbName, String tblName, List<Partition> partitions, TblTransaction tblTxn
            ) throws InvalidOperationException, MetaException, TException {
          for (Partition part : partitions) {
            part.setWriteId(tblTxn.writeId);
            // Correct validWriteIdList is needed
            // to commit the alter partitions operation in hms side.
            client.alter_partitions(dbName, tblName, partitions, null,
                 tblTxn.validWriteIds, tblTxn.writeId);
          }
        }

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            boroknagyz Zoltán Borók-Nagy
            boroknagyz Zoltán Borók-Nagy
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment