HBase
  1. HBase
  2. HBASE-6350

Some logging improvements for RegionServer bulk loading

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.94.0
    • Fix Version/s: 0.95.0
    • Component/s: regionserver
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The current logging in the bulk loading RPC call to a RegionServer lacks some info in certain cases. For instance, I recently noticed that it is possible that IOException may be caused during bulk load file transfer (copy) off of another FS and that during the same time the client already times the socket out and thereby does not receive a thrown Exception back remotely (HBase prints a ClosedChannelException for the IPC when it attempts to send the real message, and hence the real cause is lost).

      Improvements around this kind of issue, wherein we could first log the IOException at the RS before sending, and a few other wording improvements are present in my patch.

        Activity

        Hide
        stack added a comment -

        Marking closed.

        Show
        stack added a comment - Marking closed.
        Hide
        Harsh J added a comment -

        Thanks Ted!

        (Marking as resolved, as I think it may have been accidentally left open)

        Show
        Harsh J added a comment - Thanks Ted! (Marking as resolved, as I think it may have been accidentally left open)
        Hide
        Ted Yu added a comment -

        Patch integrated to trunk.

        Thanks for the patch, Harsh.

        Show
        Ted Yu added a comment - Patch integrated to trunk. Thanks for the patch, Harsh.
        Hide
        Ted Yu added a comment -

        +1 on patch.

        Will integrate on Monday if there is no objection.

        Show
        Ted Yu added a comment - +1 on patch. Will integrate on Monday if there is no objection.
        Hide
        Harsh J added a comment -

        In the first hunk, I pre-create the exception, log it and only then throw it. Previously it was getting thrown directly and was not getting logged before getting thrown, thereby open to the edge case of client timeout causing an eat up of the message.

        Show
        Harsh J added a comment - In the first hunk, I pre-create the exception, log it and only then throw it. Previously it was getting thrown directly and was not getting logged before getting thrown, thereby open to the edge case of client timeout causing an eat up of the message.
        Hide
        Ted Yu added a comment -

        The changes look fine.
        However, I couldn't see where the difference is for the first hunk.

        Show
        Ted Yu added a comment - The changes look fine. However, I couldn't see where the difference is for the first hunk.
        Hide
        Harsh J added a comment -

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        I started an RS and did a remote FS bulk load call to verify some of the logging changes.

        -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings).

        I don't see how this came to be. I've merely added a few already existing objects here and there and changed a few string messages. I am not sure this warning is cause of me.

        -1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9) warnings.

        No, it does not. At least I don't see how the warnings apply to my changes.

        -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.TestZooKeeper

        This doesn't seem to be cause of my trivial patch either.

        Show
        Harsh J added a comment - -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. I started an RS and did a remote FS bulk load call to verify some of the logging changes. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). I don't see how this came to be. I've merely added a few already existing objects here and there and changed a few string messages. I am not sure this warning is cause of me. -1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9) warnings. No, it does not. At least I don't see how the warnings apply to my changes. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.TestZooKeeper This doesn't seem to be cause of my trivial patch either.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12535512/HBASE-6350.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings).

        -1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.TestZooKeeper

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2342//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2342//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2342//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2342//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12535512/HBASE-6350.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.TestZooKeeper Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2342//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2342//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2342//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2342//console This message is automatically generated.

          People

          • Assignee:
            Harsh J
            Reporter:
            Harsh J
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development