Accumulo
  1. Accumulo
  2. ACCUMULO-2889

Batch metadata table updates for new walogs

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Incomplete
    • Affects Version/s: 1.5.1, 1.6.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Currently, when we update the Metadata table with new loggers, we will update the metadata for each tablet serially. We could optimize this to instead use a batchwriter to send all metadata updates for all tablets in a batch.

      A few special cases include:

      • What if the !METADATA tablet was included in the batch?
      • What about the root tablet?

      Benefit:
      In one of our clusters, we're experiencing particularly slow HDFS operations leading to large oscillations in ingest performance. We haven't isolated the cause in HDFS but when we profile the tservers, we noticed that they were waiting for metadata table operations to complete. This would target the waiting.

      Potential downsides:
      Given the existing locking scheme, it looks like we may have to lock a tablet for slightly longer (we'll lock for the duration of the batch).

      1. ACCUMULO-2889.2.patch
        33 kB
        Jonathan Park
      2. accumulo-2889_withoutpatch.png
        475 kB
        Jonathan Park
      3. accumulo-2889-withpatch.png
        464 kB
        Jonathan Park
      4. ACCUMULO-2889.1.patch
        35 kB
        Jonathan Park
      5. start-ingest.sh
        2 kB
        Jonathan Park
      6. batch_perf_test.sh
        0.5 kB
        Jonathan Park
      7. run_all.sh
        0.3 kB
        Jonathan Park
      8. ACCUMULO-2889.0.patch.txt
        30 kB
        Jonathan Park

        Issue Links

          Activity

          Jonathan Park created issue -
          Josh Elser made changes -
          Field Original Value New Value
          Link This issue is related to ACCUMULO-1294 [ ACCUMULO-1294 ]
          Jonathan Park made changes -
          Assignee Jonathan Park [ parkjsung ]
          Jonathan Park made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Jonathan Park made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          Affects Version/s 1.6.0 [ 12322468 ]
          Jonathan Park made changes -
          Attachment ACCUMULO-2889.0.patch.txt [ 12650659 ]
          Jonathan Park made changes -
          Remote Link This issue links to "review (Web Link)" [ 15390 ]
          Eric Newton made changes -
          Issue Type Bug [ 1 ] Improvement [ 4 ]
          Jonathan Park made changes -
          Attachment run_all.sh [ 12653020 ]
          Attachment batch_perf_test.sh [ 12653021 ]
          Attachment start-ingest.sh [ 12653022 ]
          Attachment ACCUMULO-2889.1.patch [ 12653023 ]
          Attachment accumulo-2889-withpatch.png [ 12653024 ]
          Attachment accumulo-2889_withoutpatch.png [ 12653025 ]
          Jonathan Park made changes -
          Attachment ACCUMULO-2889.2.patch [ 12665761 ]
          Josh Elser made changes -
          Link This issue relates to ACCUMULO-3423 [ ACCUMULO-3423 ]
          Josh Elser made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Incomplete [ 4 ]

            People

            • Assignee:
              Jonathan Park
              Reporter:
              Jonathan Park
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development