Uploaded image for project: 'Accumulo'
  1. Accumulo
  2. ACCUMULO-4692

CompactionDriver leaves abandoned metadata scans

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: fate
    • Labels:
      None

      Description

      We wrote a tool to kick off tablet compactions in the background while minimizing compaction load per-server. The tool uses range compaction on one tablet per call. We're seeing a high number of scans on the metadata table (~7,000 on a ~100 node cluster).

      The metadata query in the isReady() method of CompactionDriver that is used to see if the compaction has completed uses a range that goes to the end of the metadata entries for the given table, but it stops consuming the results of the scanner at the end of the compaction range. isReady gets called in a pretty tight loop, especially with hundreds of compactions running concurrently. Seems like we should limit the scan to the metadata range associated with the compaction so that the scan can get cleaned up quickly.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              afuchs Adam Fuchs
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: