Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16360

Improve "No delta files or original files found to compact in" message

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.2.0
    • 3.0.0
    • Transactions
    • None

    Description

      CompactorMR.run() produces

      LOG.error("No delta files or original files found to compact in " + sd.getLocation() + " for compactionId=" + ci.id);
            

      in some cases even where there are plenty of delta directories in the partition.
      The reason is that compactor won't compact anything above the level of an open transaction. So if there is a very long running txn, it may filter out all deltas in a partition. This looks very confusing when looking at the logs.

      Attachments

        1. HIVE-16360.02.patch
          8 kB
          Eugene Koifman
        2. HIVE-16360.01.patch
          7 kB
          Eugene Koifman

        Issue Links

          Activity

            People

              ekoifman Eugene Koifman
              ekoifman Eugene Koifman
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: