HBase
  1. HBase
  2. HBASE-2041

Change WAL default configuration values

    Details

    • Type: Task Task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.90.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      My last email on the thread "Should we change the default value of hbase.regionserver.flushlogentries for 0.21?"

      Ok to make sure I get this right:

      • we enable deferred log flush by default
      • we set flushlogentries=1

      Also since 10 seconds is kind of a huge window I propose that:

      • we set optionalLogFlush=1000

      which is the MySQL default. We also have to update the wiki (there's
      already an entry on deferred log flush) by adding the configuration of
      flushlogentries.

      1. HBASE-2041-shell.patch
        2 kB
        Jean-Daniel Cryans
      2. HBASE-2041-interrupt-fix.patch
        2 kB
        Jean-Daniel Cryans
      3. HBASE-2041.patch
        3 kB
        Jean-Daniel Cryans

        Activity

        Hide
        Andrew Purtell added a comment -

        +1

        Saw your numbers up on the list.

        Show
        Andrew Purtell added a comment - +1 Saw your numbers up on the list.
        Hide
        Jean-Daniel Cryans added a comment -

        Patch that does the changes.

        Show
        Jean-Daniel Cryans added a comment - Patch that does the changes.
        Hide
        stack added a comment -

        +1

        Show
        stack added a comment - +1
        Hide
        Jean-Daniel Cryans added a comment -

        Thanks guys for discussing that on the mailing list so that we could improve HBase's 0.21 out of the box experience, committed to trunk.

        Show
        Jean-Daniel Cryans added a comment - Thanks guys for discussing that on the mailing list so that we could improve HBase's 0.21 out of the box experience, committed to trunk.
        Hide
        Jean-Daniel Cryans added a comment -

        Reopening since the commit broke the build and the doc isn't done yet.

        Show
        Jean-Daniel Cryans added a comment - Reopening since the commit broke the build and the doc isn't done yet.
        Hide
        Jean-Daniel Cryans added a comment -

        Lots of debugging later, I learned that not using awaitNanos on Condition, with a small waiting time and multiple threads, can actually completely hide interrupts. This patches changes that, checks the status of the Thread and also changes some comments.

        BTW even when calling this.isInterrupted() with the await(1000, MILLIS) didn't work, always returned false.

        Show
        Jean-Daniel Cryans added a comment - Lots of debugging later, I learned that not using awaitNanos on Condition, with a small waiting time and multiple threads, can actually completely hide interrupts. This patches changes that, checks the status of the Thread and also changes some comments. BTW even when calling this.isInterrupted() with the await(1000, MILLIS) didn't work, always returned false.
        Hide
        Jean-Daniel Cryans added a comment -

        Patch that adds the option of setting DEFERRED_LOG_FLUSH in the shell. I also changed the fact that if you don't specify the other settings, they won't get reset to default.

        Show
        Jean-Daniel Cryans added a comment - Patch that adds the option of setting DEFERRED_LOG_FLUSH in the shell. I also changed the fact that if you don't specify the other settings, they won't get reset to default.
        Hide
        Jean-Daniel Cryans added a comment -

        TestMergeTool is green on my machine, was green on Hudson after my last commit but then failed again... so I'm going to commit this too:

            try {
              logSyncerThread.interrupt();
              // Make sure we synced everything
              logSyncerThread.join(this.optionalFlushInterval*2);
            } catch (InterruptedException e) {
              LOG.error("Exception while waiting for syncer thread to die", e);
            }
        

        So we will still proceed even if the thread just hangs.

        Show
        Jean-Daniel Cryans added a comment - TestMergeTool is green on my machine, was green on Hudson after my last commit but then failed again... so I'm going to commit this too: try { logSyncerThread.interrupt(); // Make sure we synced everything logSyncerThread.join( this .optionalFlushInterval*2); } catch (InterruptedException e) { LOG.error( "Exception while waiting for syncer thread to die" , e); } So we will still proceed even if the thread just hangs.
        Hide
        Jean-Daniel Cryans added a comment -

        This was fixed a while ago.

        Show
        Jean-Daniel Cryans added a comment - This was fixed a while ago.

          People

          • Assignee:
            Jean-Daniel Cryans
            Reporter:
            Jean-Daniel Cryans
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development