Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-26401

Refine the log of add_partitions if the partition already exists

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 4.0.0-alpha-1
    • None
    • Hive

    Description

      Currently add_partitions_xxx will log the complete information of a partition if it already exists, see in HMSHandler.java#L4320:

          if (!shouldAdd) {
            LOG.info("Not adding partition {} as it already exists", part);
            return false;
          }
      

      It will print a long message including the columns of this partition, we think it is unnecessary based on the following two points:
      1. The long message is redundant.
      We can get enough information from just cat_name.db_name.tbl_name[part_col1=part_val1/part_col2=part_val2...]

      2. The long message is not friendly to save and query.
      This log message will take up a large log space especially when the user need to execute MSCK REPAIR TABLE operation regularly because the old partition must be already existed. 

      Attachments

        Issue Links

          Activity

            People

              wechar Wechar
              wechar Wechar
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m