HBase
  1. HBase
  2. HBASE-4718

Backport HBASE-4552 to 0.90 branch

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.90.4
    • Fix Version/s: 0.90.5
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      HRegion.bulkLoadHFile() has been removed in 0.92 and TRUNK. It is deprecated in 0.90.5
      HRegion.bulkLoadHFiles() is introduced for multi-family bulk load.
      Default value for hbase.bulkload.retries.number is 0, meaning bulk load would never give up.
      Show
      HRegion.bulkLoadHFile() has been removed in 0.92 and TRUNK. It is deprecated in 0.90.5 HRegion.bulkLoadHFiles() is introduced for multi-family bulk load. Default value for hbase.bulkload.retries.number is 0, meaning bulk load would never give up.

      Description

      In discussion of HBASE-4552 / HBASE-4677 there has been some discussion about whether and how to backport HBASE-4552 to the 0.90 branch. This is a potentially compatibility breaking so several approaches hav ebeen suggested.

      1) provide patch but do not integrate
      2) integrate patch that extends and deprecates old api without removing old api. It has been argued that clients are supposed to use LoadIncrementalHFiles api and not at the internal HRegionServer RPC api.

      1. 4718-v2.90
        61 kB
        Ted Yu
      2. hbase-4718.0.90.patch
        56 kB
        Jonathan Hsieh
      3. hbase-4718.v3.includes-hbase-3316.patch
        63 kB
        Jonathan Hsieh
      4. hbase-4718.v4.patch
        66 kB
        Jonathan Hsieh

        Issue Links

          Activity

          Hide
          Jonathan Hsieh added a comment -

          Old LoadIncremtalHFiles clients will still work against this version.

          Show
          Jonathan Hsieh added a comment - Old LoadIncremtalHFiles clients will still work against this version.
          Hide
          Ted Yu added a comment -

          Integrated to 0.90 branch.

          Thanks for the patch Jonathan.

          Show
          Ted Yu added a comment - Integrated to 0.90 branch. Thanks for the patch Jonathan.
          Hide
          Ted Yu added a comment -

          +1 on patch v4

          Results :
          
          Tests run: 700, Failures: 0, Errors: 0, Skipped: 9
          
          [INFO] ------------------------------------------------------------------------
          [INFO] BUILD SUCCESS
          
          Show
          Ted Yu added a comment - +1 on patch v4 Results : Tests run: 700, Failures: 0, Errors: 0, Skipped: 9 [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS
          Hide
          Ted Yu added a comment -

          TestLoadIncrementalHFilesSplitRecovery and TestHRegionServerBulkLoad also passed.
          Running whole test suite.

          Show
          Ted Yu added a comment - TestLoadIncrementalHFilesSplitRecovery and TestHRegionServerBulkLoad also passed. Running whole test suite.
          Hide
          Jonathan Hsieh added a comment -

          Patch failed to apply..

          Show
          Jonathan Hsieh added a comment - Patch failed to apply..
          Hide
          Jonathan Hsieh added a comment -

          Patch filed to apply since this was targeted to the 0.90 branch instead of the trunk/0.92 branch. Attached output of selected unit tests run.

          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          Running org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
          Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 146.315 sec
          Running org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery
          Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.407 sec
          
          Results :
          
          Tests run: 10, Failures: 0, Errors: 0, Skipped: 0
          
          Show
          Jonathan Hsieh added a comment - Patch filed to apply since this was targeted to the 0.90 branch instead of the trunk/0.92 branch. Attached output of selected unit tests run. ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 146.315 sec Running org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.407 sec Results : Tests run: 10, Failures: 0, Errors: 0, Skipped: 0
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12503685/hbase-4718.v4.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 10 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/247//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12503685/hbase-4718.v4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 10 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/247//console This message is automatically generated.
          Hide
          Jonathan Hsieh added a comment -

          New version of patch that includes backport of HBASE-4552, HBASE-4716, HBASE-3316 and HBASE-4740.

          Show
          Jonathan Hsieh added a comment - New version of patch that includes backport of HBASE-4552 , HBASE-4716 , HBASE-3316 and HBASE-4740 .
          Hide
          Ted Yu added a comment -

          Combining patch v3 with addendum for HBASE-4716 passes all tests.

          @Jonathan:
          Once HBASE-4740 gets resolved, let's put that with the above and make patch v4.

          Thanks

          Show
          Ted Yu added a comment - Combining patch v3 with addendum for HBASE-4716 passes all tests. @Jonathan: Once HBASE-4740 gets resolved, let's put that with the above and make patch v4. Thanks
          Hide
          Ted Yu added a comment -

          I got the following when running test suite with patch v2:

          testLogRollOnPipelineRestart(org.apache.hadoop.hbase.regionserver.wal.TestLogRolling)  Time elapsed: 713.958 sec  <<< ERROR!
          java.io.FileNotFoundException: File does not exist: hdfs://localhost:54304/user/zhihyu/.logs/10.246.204.38,54320,1320336018579/10.246.204.38%3A54320.1320336018858
            at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:466)
            at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:676)
            at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1424)
            at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1419)
            at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.<init>(SequenceFileLogReader.java:57)
            at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:158)
            at org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:629)
            at org.apache.hadoop.hbase.regionserver.wal.TestLogRolling.testLogRollOnPipelineRestart(TestLogRolling.java:489)
          

          In the test output, I saw:

          2011-11-03 09:14:06,358 ERROR [Shutdown of DFS[DFSClient[clientName=DFSClient_hb_rs_10.246.204.38,54321,1320336018597_1320336018852, ugi=zhihyu.hfs.1,supergroup]]] hdfs.DFSClient$LeaseChecker(1074): Exception closing file /user/zhihyu/.logs/10.246.204.38,54321,1320336018597/10.246.204.38%3A54321.1320336374373 : java.io.IOException: Error Recovery for block blk_-8007982726961331998_1076 failed  because recovery from primary datanode 127.0.0.1:54952 failed 6 times.  Pipeline was 127.0.0.1:54952. Aborting...
          java.io.IOException: Error Recovery for block blk_-8007982726961331998_1076 failed  because recovery from primary datanode 127.0.0.1:54952 failed 6 times.  Pipeline was 127.0.0.1:54952. Aborting...
            at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2741)
            at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1500(DFSClient.java:2172)
            at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2371)
          

          I ran the test by itself and got same result.

          Show
          Ted Yu added a comment - I got the following when running test suite with patch v2: testLogRollOnPipelineRestart(org.apache.hadoop.hbase.regionserver.wal.TestLogRolling) Time elapsed: 713.958 sec <<< ERROR! java.io.FileNotFoundException: File does not exist: hdfs: //localhost:54304/user/zhihyu/.logs/10.246.204.38,54320,1320336018579/10.246.204.38%3A54320.1320336018858 at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:466) at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:676) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1424) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1419) at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.<init>(SequenceFileLogReader.java:57) at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:158) at org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:629) at org.apache.hadoop.hbase.regionserver.wal.TestLogRolling.testLogRollOnPipelineRestart(TestLogRolling.java:489) In the test output, I saw: 2011-11-03 09:14:06,358 ERROR [Shutdown of DFS[DFSClient[clientName=DFSClient_hb_rs_10.246.204.38,54321,1320336018597_1320336018852, ugi=zhihyu.hfs.1,supergroup]]] hdfs.DFSClient$LeaseChecker(1074): Exception closing file /user/zhihyu/.logs/10.246.204.38,54321,1320336018597/10.246.204.38%3A54321.1320336374373 : java.io.IOException: Error Recovery for block blk_-8007982726961331998_1076 failed because recovery from primary datanode 127.0.0.1:54952 failed 6 times. Pipeline was 127.0.0.1:54952. Aborting... java.io.IOException: Error Recovery for block blk_-8007982726961331998_1076 failed because recovery from primary datanode 127.0.0.1:54952 failed 6 times. Pipeline was 127.0.0.1:54952. Aborting... at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2741) at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1500(DFSClient.java:2172) at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2371) I ran the test by itself and got same result.
          Hide
          Ted Yu added a comment -

          Since enum was removed by HBASE-4716 addendum, I think patch v2 is closer to the current TRUNK.

          Show
          Ted Yu added a comment - Since enum was removed by HBASE-4716 addendum, I think patch v2 is closer to the current TRUNK.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12502112/hbase-4718.v3.includes-hbase-3316.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 10 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/153//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12502112/hbase-4718.v3.includes-hbase-3316.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 10 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/153//console This message is automatically generated.
          Hide
          Jonathan Hsieh added a comment -

          Updated patch generated using --no-prefix, uses enum from HBASE-4716, and also includes HBASE-3316.

          Show
          Jonathan Hsieh added a comment - Updated patch generated using --no-prefix, uses enum from HBASE-4716 , and also includes HBASE-3316 .
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12502096/4718-v2.90
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 12 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/150//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12502096/4718-v2.90 against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 12 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/150//console This message is automatically generated.
          Hide
          Ted Yu added a comment -

          Bulk-load related tests passed:
          953 mt -Dtest=TestLoadIncrementalHFilesSplitRecovery
          954 mt -Dtest=TestHRegionServerBulkLoad
          957 mt -Dtest=TestLoadIncrementalHFiles
          959 mt -Dtest=TestHFileOutputFormat

          Show
          Ted Yu added a comment - Bulk-load related tests passed: 953 mt -Dtest=TestLoadIncrementalHFilesSplitRecovery 954 mt -Dtest=TestHRegionServerBulkLoad 957 mt -Dtest=TestLoadIncrementalHFiles 959 mt -Dtest=TestHFileOutputFormat
          Hide
          Ted Yu added a comment -

          Patch v2 combines HBASE-4552, HBASE-4716 and HBASE-3316

          Show
          Ted Yu added a comment - Patch v2 combines HBASE-4552 , HBASE-4716 and HBASE-3316
          Hide
          stack added a comment -

          Want to reapply with --no-prefix mighty Jon?

          Show
          stack added a comment - Want to reapply with --no-prefix mighty Jon?
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12502059/hbase-4718.0.90.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 7 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/146//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12502059/hbase-4718.0.90.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/146//console This message is automatically generated.
          Hide
          Jonathan Hsieh added a comment -

          Here are the results of the relevent unit tests running tests on 0.90.x. I'm fairly confident that only known flakies could fail on the full run, will post any anomalies.

          
          ~/proj/hbase-0.90$ mvn test -Dtest=TestLoadIncrementalHFilesSplitRecovery,TestHRegionServerBulkLoad
          
          ....
          
          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          Running org.apache.hadoop.hbase.regionserver.TestHRegionServerBulkLoad
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 47.143 sec
          Running org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery
          Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 72.985 sec
          
          Results :
          
          Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
          
          Show
          Jonathan Hsieh added a comment - Here are the results of the relevent unit tests running tests on 0.90.x. I'm fairly confident that only known flakies could fail on the full run, will post any anomalies. ~/proj/hbase-0.90$ mvn test -Dtest=TestLoadIncrementalHFilesSplitRecovery,TestHRegionServerBulkLoad .... ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.apache.hadoop.hbase.regionserver.TestHRegionServerBulkLoad Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 47.143 sec Running org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 72.985 sec Results : Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
          Hide
          Jonathan Hsieh added a comment -

          I have verified that stock 0.90.4's non-atomic bulk import still works against a standalone cluster running 0.90.5-snapshot including the HBASE-4718 patch (combined backport of HBASE-4552/HBASE-4716) and the HBASE-3316 patch (separate but trivial backport)

          Show
          Jonathan Hsieh added a comment - I have verified that stock 0.90.4's non-atomic bulk import still works against a standalone cluster running 0.90.5-snapshot including the HBASE-4718 patch (combined backport of HBASE-4552 / HBASE-4716 ) and the HBASE-3316 patch (separate but trivial backport)
          Hide
          Jonathan Hsieh added a comment -

          Attached patch that is basically HBASE-4552 and the HBASE-4716 single colfam optimization.

          Show
          Jonathan Hsieh added a comment - Attached patch that is basically HBASE-4552 and the HBASE-4716 single colfam optimization.
          Hide
          Jonathan Hsieh added a comment -

          An initial backport was done by apurtell. I've taken it and made it work against 0.90. It requires a backport of HBASE-3316. Before I submit, I would like test cross version RPC to verify compatibility or reasonable warning messages.

          If it is decided not to integrate, I will post the patch after testing.

          Show
          Jonathan Hsieh added a comment - An initial backport was done by apurtell. I've taken it and made it work against 0.90. It requires a backport of HBASE-3316 . Before I submit, I would like test cross version RPC to verify compatibility or reasonable warning messages. If it is decided not to integrate, I will post the patch after testing.

            People

            • Assignee:
              Jonathan Hsieh
              Reporter:
              Jonathan Hsieh
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development