Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12334

Change Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage Throttling after retries

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: tools
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      HADOOP-11693 mitigated the problem of HMaster aborting regionserver due to Azure Storage Throttling event during HBase WAL archival. The way this was achieved was by applying an intensive exponential retry when throttling occurred.

      As a second level of mitigation we will change the mode of copy operation if the operation fails even after all retries -i.e. we will do a client side copy of the blob and then copy it back to destination. This operation will not be subject to throttling and hence should provide a stronger mitigation. However it is more expensive, hence we do it only in the case we fail after all retries

      1. HADOOP-12334.07.patch
        6 kB
        Gaurav Kanade
      2. HADOOP-12334.06.patch
        5 kB
        Gaurav Kanade
      3. HADOOP-12334.05.patch
        5 kB
        Gaurav Kanade
      4. HADOOP-12334.04.patch
        5 kB
        Gaurav Kanade
      5. HADOOP-12334.03.patch
        5 kB
        Gaurav Kanade
      6. HADOOP-12334.02.patch
        3 kB
        Gaurav Kanade
      7. HADOOP-12334.01.patch
        3 kB
        Gaurav Kanade

        Activity

        Hide
        gouravk Gaurav Kanade added a comment -
        Show
        gouravk Gaurav Kanade added a comment - Duo Xu Dushyanth Chris Nauroth
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 patch 0m 0s The patch command could not apply the patch during dryrun.



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12752297/HADOOP-12334.1.patch
        Optional Tests  
        git revision trunk / eee0d45
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7523/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12752297/HADOOP-12334.1.patch Optional Tests   git revision trunk / eee0d45 Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7523/console This message was automatically generated.
        Hide
        gouravk Gaurav Kanade added a comment -

        Please check latest attached file Duo Xu Dushyanth Chris Nauroth

        Show
        gouravk Gaurav Kanade added a comment - Please check latest attached file Duo Xu Dushyanth Chris Nauroth
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 16m 22s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        -1 javac 9m 24s The patch appears to cause the build to fail.



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12752318/HADOOP-12334.2.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / eee0d45
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7525/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 22s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javac 9m 24s The patch appears to cause the build to fail. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12752318/HADOOP-12334.2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / eee0d45 Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7525/console This message was automatically generated.
        Hide
        gouravk Gaurav Kanade added a comment -

        Added new patch after fixing build issues. Duo Xu Chris Nauroth Dushyanth

        Show
        gouravk Gaurav Kanade added a comment - Added new patch after fixing build issues. Duo Xu Chris Nauroth Dushyanth
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 15m 57s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        -1 javac 7m 40s The patch appears to cause the build to fail.



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12752345/HADOOP-12334.5.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / a4d9acc
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7527/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 57s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javac 7m 40s The patch appears to cause the build to fail. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12752345/HADOOP-12334.5.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / a4d9acc Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7527/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 20m 24s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        -1 javac 8m 30s The patch appears to cause the build to fail.



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12752365/HADOOP-12334.7.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / a4d9acc
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7528/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 20m 24s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javac 8m 30s The patch appears to cause the build to fail. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12752365/HADOOP-12334.7.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / a4d9acc Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7528/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 16m 10s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 8m 9s There were no new javac warning messages.
        +1 javadoc 11m 21s There were no new javadoc warning messages.
        +1 release audit 0m 30s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 26s There were no new checkstyle issues.
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 39s mvn install still works.
        +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
        +1 findbugs 0m 48s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 tools/hadoop tests 1m 12s Tests passed in hadoop-azure.
            40m 56s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12752528/HADOOP-12334.8.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / a4d9acc
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7531/artifact/patchprocess/whitespace.txt
        hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7531/artifact/patchprocess/testrun_hadoop-azure.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7531/testReport/
        Java 1.7.0_55
        uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7531/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 10s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 9s There were no new javac warning messages. +1 javadoc 11m 21s There were no new javadoc warning messages. +1 release audit 0m 30s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 26s There were no new checkstyle issues. -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 39s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. +1 findbugs 0m 48s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 1m 12s Tests passed in hadoop-azure.     40m 56s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12752528/HADOOP-12334.8.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / a4d9acc whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7531/artifact/patchprocess/whitespace.txt hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7531/artifact/patchprocess/testrun_hadoop-azure.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7531/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7531/console This message was automatically generated.
        Hide
        gouravk Gaurav Kanade added a comment -

        Duo Xu Chris Nauroth Dushyanth Please take a look

        Show
        gouravk Gaurav Kanade added a comment - Duo Xu Chris Nauroth Dushyanth Please take a look
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 18m 26s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 10m 21s There were no new javac warning messages.
        +1 javadoc 12m 17s There were no new javadoc warning messages.
        +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 30s There were no new checkstyle issues.
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 44s mvn install still works.
        +1 eclipse:eclipse 0m 39s The patch built with eclipse:eclipse.
        +1 findbugs 0m 53s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 tools/hadoop tests 1m 19s Tests passed in hadoop-azure.
            46m 37s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12752540/HADOOP-12334.01.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / a4d9acc
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7532/artifact/patchprocess/whitespace.txt
        hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7532/artifact/patchprocess/testrun_hadoop-azure.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7532/testReport/
        Java 1.7.0_55
        uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7532/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 26s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 10m 21s There were no new javac warning messages. +1 javadoc 12m 17s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 30s There were no new checkstyle issues. -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 44s mvn install still works. +1 eclipse:eclipse 0m 39s The patch built with eclipse:eclipse. +1 findbugs 0m 53s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 1m 19s Tests passed in hadoop-azure.     46m 37s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12752540/HADOOP-12334.01.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / a4d9acc whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7532/artifact/patchprocess/whitespace.txt hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7532/artifact/patchprocess/testrun_hadoop-azure.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7532/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7532/console This message was automatically generated.
        Hide
        gouravk Gaurav Kanade added a comment -

        Added log statements to previous patch

        Show
        gouravk Gaurav Kanade added a comment - Added log statements to previous patch
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 15m 45s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 49s There were no new javac warning messages.
        +1 javadoc 10m 4s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 0m 22s The applied patch generated 1 new checkstyle issues (total was 33, now 34).
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 27s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 0m 50s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 tools/hadoop tests 1m 12s Tests passed in hadoop-azure.
            38m 34s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12752856/HADOOP-12334.02.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / a9c8ea7
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7543/artifact/patchprocess/diffcheckstylehadoop-azure.txt
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7543/artifact/patchprocess/whitespace.txt
        hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7543/artifact/patchprocess/testrun_hadoop-azure.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7543/testReport/
        Java 1.7.0_55
        uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7543/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 45s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 49s There were no new javac warning messages. +1 javadoc 10m 4s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 22s The applied patch generated 1 new checkstyle issues (total was 33, now 34). -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 27s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 0m 50s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 1m 12s Tests passed in hadoop-azure.     38m 34s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12752856/HADOOP-12334.02.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / a9c8ea7 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7543/artifact/patchprocess/diffcheckstylehadoop-azure.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7543/artifact/patchprocess/whitespace.txt hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7543/artifact/patchprocess/testrun_hadoop-azure.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7543/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7543/console This message was automatically generated.
        Hide
        gouravk Gaurav Kanade added a comment -

        Ping... Duo Xu Dushyanth Chris Nauroth Please take a look now

        Show
        gouravk Gaurav Kanade added a comment - Ping... Duo Xu Dushyanth Chris Nauroth Please take a look now
        Hide
        onpduo Duo Xu added a comment -

        Gaurav Kanade

        1. there is a tab in the line of code

        "LOG.warn("Rename: CopyBlob: StorageException: ServerBusy: Retry complete, will attempt client side copy for page blob");"

        2. there is white spaces at the end of the last line of code

                        
                        opStream.flush();
                        opStream.close();
                        ipStream.close();
                      } else {
                          throw new AzureException(e);
                      }
                  }         
        

        3. make sure indention keeps the same. I see some places you use 4 blanks. It should be 2.
        4. add your code in here, if copyblob with retry fails, the exception will be catched here. I believe this is more clear and avoids nested try-catches, which causes the above checkstyle error.

              waitForCopyToComplete(dstBlob, getInstrumentedContext());
              safeDelete(srcBlob, lease);
            } catch (StorageException e) {
              LOG.warn("Rename: CopyBlob: StorageException: Failed");
              throw new AzureException(e);
            } catch (URISyntaxException e) {
              // Re-throw exception as an Azure storage exception.
              throw new AzureException(e);
            }
        

        5. you still need to add

        if (e.getErrorCode().equals(StorageErrorCode.SERVER_BUSY.toString()))

        wrapping your code, because storage exception may not always be the throttling.
        6.

        opStream.write(buffer, 0, len);

        it seems this line of code keeps writing the same offset, that is the first 512byte of the blob. You can write a small application to test and make sure the blobs you rewrite are exactly the same.

        Show
        onpduo Duo Xu added a comment - Gaurav Kanade 1. there is a tab in the line of code "LOG.warn(" Rename: CopyBlob: StorageException: ServerBusy: Retry complete, will attempt client side copy for page blob ");" 2. there is white spaces at the end of the last line of code opStream.flush(); opStream.close(); ipStream.close(); } else { throw new AzureException(e); } } 3. make sure indention keeps the same. I see some places you use 4 blanks. It should be 2. 4. add your code in here, if copyblob with retry fails, the exception will be catched here. I believe this is more clear and avoids nested try-catches, which causes the above checkstyle error. waitForCopyToComplete(dstBlob, getInstrumentedContext()); safeDelete(srcBlob, lease); } catch (StorageException e) { LOG.warn( "Rename: CopyBlob: StorageException: Failed" ); throw new AzureException(e); } catch (URISyntaxException e) { // Re- throw exception as an Azure storage exception. throw new AzureException(e); } 5. you still need to add if (e.getErrorCode().equals(StorageErrorCode.SERVER_BUSY.toString())) wrapping your code, because storage exception may not always be the throttling. 6. opStream.write(buffer, 0, len); it seems this line of code keeps writing the same offset, that is the first 512byte of the blob. You can write a small application to test and make sure the blobs you rewrite are exactly the same.
        Hide
        cnauroth Chris Nauroth added a comment -

        Gaurav Kanade, considering my existing backlog of code reviews, my earliest opportunity to review this will be Wednesday, 9/2.

        Show
        cnauroth Chris Nauroth added a comment - Gaurav Kanade , considering my existing backlog of code reviews, my earliest opportunity to review this will be Wednesday, 9/2.
        Hide
        gouravk Gaurav Kanade added a comment -

        Addressed comments from Duo Xu

        Show
        gouravk Gaurav Kanade added a comment - Addressed comments from Duo Xu
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 16m 4s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 53s There were no new javac warning messages.
        +1 javadoc 10m 11s There were no new javadoc warning messages.
        +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 0m 23s The applied patch generated 13 new checkstyle issues (total was 33, now 46).
        -1 whitespace 0m 0s The patch has 7 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 28s mvn install still works.
        +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
        +1 findbugs 0m 49s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 tools/hadoop tests 1m 15s Tests passed in hadoop-azure.
            39m 6s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12753073/HADOOP-12334.03.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 6d12cd8
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7548/artifact/patchprocess/diffcheckstylehadoop-azure.txt
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7548/artifact/patchprocess/whitespace.txt
        hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7548/artifact/patchprocess/testrun_hadoop-azure.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7548/testReport/
        Java 1.7.0_55
        uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7548/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 4s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 53s There were no new javac warning messages. +1 javadoc 10m 11s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 23s The applied patch generated 13 new checkstyle issues (total was 33, now 46). -1 whitespace 0m 0s The patch has 7 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. +1 findbugs 0m 49s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 1m 15s Tests passed in hadoop-azure.     39m 6s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12753073/HADOOP-12334.03.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 6d12cd8 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7548/artifact/patchprocess/diffcheckstylehadoop-azure.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7548/artifact/patchprocess/whitespace.txt hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7548/artifact/patchprocess/testrun_hadoop-azure.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7548/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7548/console This message was automatically generated.
        Hide
        gouravk Gaurav Kanade added a comment -

        Fixed some persistent whitelist errors

        Show
        gouravk Gaurav Kanade added a comment - Fixed some persistent whitelist errors
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 17m 19s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        -1 javac 8m 28s The patch appears to cause the build to fail.



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12753351/HADOOP-12334.04.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / caa04de
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7562/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 19s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javac 8m 28s The patch appears to cause the build to fail. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12753351/HADOOP-12334.04.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / caa04de Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7562/console This message was automatically generated.
        Hide
        gouravk Gaurav Kanade added a comment -

        Fix some whitespace issues

        Show
        gouravk Gaurav Kanade added a comment - Fix some whitespace issues
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 16m 7s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 59s There were no new javac warning messages.
        +1 javadoc 9m 59s There were no new javadoc warning messages.
        +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 23s There were no new checkstyle issues.
        -1 whitespace 0m 0s The patch has 5 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 27s mvn install still works.
        +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
        +1 findbugs 0m 49s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 tools/hadoop tests 1m 12s Tests passed in hadoop-azure.
            38m 59s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12753372/HADOOP-12334.04.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / caa04de
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7563/artifact/patchprocess/whitespace.txt
        hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7563/artifact/patchprocess/testrun_hadoop-azure.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7563/testReport/
        Java 1.7.0_55
        uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7563/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 7s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 59s There were no new javac warning messages. +1 javadoc 9m 59s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 23s There were no new checkstyle issues. -1 whitespace 0m 0s The patch has 5 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 27s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. +1 findbugs 0m 49s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 1m 12s Tests passed in hadoop-azure.     38m 59s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12753372/HADOOP-12334.04.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / caa04de whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7563/artifact/patchprocess/whitespace.txt hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7563/artifact/patchprocess/testrun_hadoop-azure.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7563/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7563/console This message was automatically generated.
        Hide
        gouravk Gaurav Kanade added a comment -

        Removing yet more (and hopefully last few) whitespace errors

        Show
        gouravk Gaurav Kanade added a comment - Removing yet more (and hopefully last few) whitespace errors
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 17m 58s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 10m 5s There were no new javac warning messages.
        +1 javadoc 11m 51s There were no new javadoc warning messages.
        +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 24s There were no new checkstyle issues.
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 39s mvn install still works.
        +1 eclipse:eclipse 0m 39s The patch built with eclipse:eclipse.
        +1 findbugs 0m 56s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 tools/hadoop tests 1m 24s Tests passed in hadoop-azure.
            45m 24s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12753389/HADOOP-12334.05.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 826ae1c
        hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7564/artifact/patchprocess/testrun_hadoop-azure.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7564/testReport/
        Java 1.7.0_55
        uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7564/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 58s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 10m 5s There were no new javac warning messages. +1 javadoc 11m 51s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 24s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 39s mvn install still works. +1 eclipse:eclipse 0m 39s The patch built with eclipse:eclipse. +1 findbugs 0m 56s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 1m 24s Tests passed in hadoop-azure.     45m 24s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12753389/HADOOP-12334.05.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 826ae1c hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7564/artifact/patchprocess/testrun_hadoop-azure.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7564/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7564/console This message was automatically generated.
        Hide
        gouravk Gaurav Kanade added a comment -

        Lin Chan Pravin Mittal Could you please review ?

        Show
        gouravk Gaurav Kanade added a comment - Lin Chan Pravin Mittal Could you please review ?
        Hide
        onpduo Duo Xu added a comment -

        Gaurav Kanade

        The patch 05 looks good to me. But still there are two format issues,

        1.

        LOG.warn("Rename: CopyBlob: StorageException: Failed"); 	      
        throw new AzureException(se);
           

        There is a tab before the second line.

        2. some lines exceed 80 character limit, for example

        LOG.warn("Rename: CopyBlob: StorageException: ServerBusy: Retry complete, will attempt client side copy for page blob");
        
        Show
        onpduo Duo Xu added a comment - Gaurav Kanade The patch 05 looks good to me. But still there are two format issues, 1. LOG.warn( "Rename: CopyBlob: StorageException: Failed" ); throw new AzureException(se); There is a tab before the second line. 2. some lines exceed 80 character limit, for example LOG.warn( "Rename: CopyBlob: StorageException: ServerBusy: Retry complete, will attempt client side copy for page blob" );
        Hide
        cnauroth Chris Nauroth added a comment -

        Hello Gaurav Kanade. Thank you for the patch. I have a few comments.

                  if(srcBlob.getProperties().getBlobType() == BlobType.PAGE_BLOB){
        

        Is it necessary to check that it's a page blob? Would the same logic also be useful for block blobs?

                    opStream.flush();
                    opStream.close();
                    ipStream.close();
        

        Here, if opStream.flush() throws an exception, then opStream and ipStream won't get closed. Likewise, if opStream.close() throws an exception, then ipStream won't get closed. I recommend using try-with-resources to guarantee the resources get closed.

        Is it possible to cover the new functionality with a unit test, perhaps using mocks to inject an exception with SERVER_BUSY as the reason?

        Show
        cnauroth Chris Nauroth added a comment - Hello Gaurav Kanade . Thank you for the patch. I have a few comments. if (srcBlob.getProperties().getBlobType() == BlobType.PAGE_BLOB){ Is it necessary to check that it's a page blob? Would the same logic also be useful for block blobs? opStream.flush(); opStream.close(); ipStream.close(); Here, if opStream.flush() throws an exception, then opStream and ipStream won't get closed. Likewise, if opStream.close() throws an exception, then ipStream won't get closed. I recommend using try-with-resources to guarantee the resources get closed. Is it possible to cover the new functionality with a unit test, perhaps using mocks to inject an exception with SERVER_BUSY as the reason?
        Hide
        linchan Lin Chan added a comment -

        +1 looks good.

        Show
        linchan Lin Chan added a comment - +1 looks good.
        Hide
        gouravk Gaurav Kanade added a comment -

        Addressed latest comments

        Show
        gouravk Gaurav Kanade added a comment - Addressed latest comments
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 16m 52s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 57s There were no new javac warning messages.
        +1 javadoc 10m 10s There were no new javadoc warning messages.
        +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 22s There were no new checkstyle issues.
        -1 whitespace 0m 0s The patch has 5 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 28s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 0m 49s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 tools/hadoop tests 1m 12s Tests passed in hadoop-azure.
            39m 51s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12756124/HADOOP-12334.06.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 6c52be7
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7665/artifact/patchprocess/whitespace.txt
        hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7665/artifact/patchprocess/testrun_hadoop-azure.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7665/testReport/
        Java 1.7.0_55
        uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7665/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 52s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 57s There were no new javac warning messages. +1 javadoc 10m 10s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 22s There were no new checkstyle issues. -1 whitespace 0m 0s The patch has 5 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 0m 49s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 1m 12s Tests passed in hadoop-azure.     39m 51s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12756124/HADOOP-12334.06.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 6c52be7 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7665/artifact/patchprocess/whitespace.txt hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7665/artifact/patchprocess/testrun_hadoop-azure.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7665/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7665/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 16m 30s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 8m 6s There were no new javac warning messages.
        +1 javadoc 10m 21s There were no new javadoc warning messages.
        +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 22s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 30s mvn install still works.
        +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse.
        +1 findbugs 0m 48s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 tools/hadoop tests 1m 11s Tests passed in hadoop-azure.
            39m 54s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12756136/HADOOP-12334.06.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 77aaf4c
        hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7667/artifact/patchprocess/testrun_hadoop-azure.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7667/testReport/
        Java 1.7.0_55
        uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7667/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 30s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 6s There were no new javac warning messages. +1 javadoc 10m 21s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 22s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse. +1 findbugs 0m 48s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 1m 11s Tests passed in hadoop-azure.     39m 54s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12756136/HADOOP-12334.06.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 77aaf4c hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7667/artifact/patchprocess/testrun_hadoop-azure.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7667/testReport/ Java 1.7.0_55 uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7667/console This message was automatically generated.
        Hide
        gouravk Gaurav Kanade added a comment -

        Lin Chan: Thanks!
        Chris Nauroth: Thanks for the review! Re your comments:
        1. The reason we are doing this for a page blob and not a block blob: a) We do not expect the throttling to occur on block blobs b) This is supposed to be a very rare case code path c) There might be some technical difficulties in doing the copy process in a simple manner like we did for page blobs - hence leaving it at page blobs for now
        2. I addressed the concerns re throwing of exceptions - see latest patch
        3. We have separately tested the individual functionality of the new method of copy; it would likely not be in the scope of a unit test to actually test with the injection of SERVER-BUSY error
        Dushyanth Could you please review now?

        Show
        gouravk Gaurav Kanade added a comment - Lin Chan : Thanks! Chris Nauroth : Thanks for the review! Re your comments: 1. The reason we are doing this for a page blob and not a block blob: a) We do not expect the throttling to occur on block blobs b) This is supposed to be a very rare case code path c) There might be some technical difficulties in doing the copy process in a simple manner like we did for page blobs - hence leaving it at page blobs for now 2. I addressed the concerns re throwing of exceptions - see latest patch 3. We have separately tested the individual functionality of the new method of copy; it would likely not be in the scope of a unit test to actually test with the injection of SERVER-BUSY error Dushyanth Could you please review now?
        Hide
        cnauroth Chris Nauroth added a comment -

        Gaurav Kanade, thank you for the updated patch and the responses.

        In patch v06, I still see a possibility that streams won't get closed after exceptions from within the finally block.

                } finally {
                  if(ipStream != null) {
                    ipStream.close();
                  }
                  if(opStream != null) {
                    opStream.close();
                  }
                }
        

        You could address this by using org.apache.hadoop.io.IOUtils#closeStream, which would handle the null check, the close, and the exception handling.

        It's great to hear that there has been manual testing done. Could you also please take another look at the possibility of adding a unit test? For example, TestAzureFileSystemErrorConditions shows use of some special testing hooks to simulate server-side failures. Is it possible to do something similar here? Sorry to push on this, but this has become a somewhat complex code path, and it would be great if we could get automated test coverage of it to avoid regressions.

        Show
        cnauroth Chris Nauroth added a comment - Gaurav Kanade , thank you for the updated patch and the responses. In patch v06, I still see a possibility that streams won't get closed after exceptions from within the finally block. } finally { if (ipStream != null ) { ipStream.close(); } if (opStream != null ) { opStream.close(); } } You could address this by using org.apache.hadoop.io.IOUtils#closeStream , which would handle the null check, the close, and the exception handling. It's great to hear that there has been manual testing done. Could you also please take another look at the possibility of adding a unit test? For example, TestAzureFileSystemErrorConditions shows use of some special testing hooks to simulate server-side failures. Is it possible to do something similar here? Sorry to push on this, but this has become a somewhat complex code path, and it would be great if we could get automated test coverage of it to avoid regressions.
        Hide
        gouravk Gaurav Kanade added a comment -

        Thanks Chris Nauroth. Quick follow up question: were you suggesting that these tests be over actual Azure Storage - because I do make some API calls. I am new to the test framework and Trying to understand what would be possible in this scenario.

        Duo Xu for reference

        Show
        gouravk Gaurav Kanade added a comment - Thanks Chris Nauroth . Quick follow up question: were you suggesting that these tests be over actual Azure Storage - because I do make some API calls. I am new to the test framework and Trying to understand what would be possible in this scenario. Duo Xu for reference
        Hide
        cnauroth Chris Nauroth added a comment -

        Hello Gaurav Kanade. I would not suggest trying to write a test for this against live Azure Storage. The patch relates to client-side error handling of server-side errors. I don't think there is a reliable way to force the service to return those errors.

        Instead, I was thinking we might be able to use mocks to simulate the errors and confirm that it can fall back to a client-side copy. TestAzureFileSystemErrorConditions is an example of a test suite that already does something similar, so you might try looking at that one as a sample.

        Please take a look, and let me know if you think it can be done or if you think it's not feasible. Thanks!

        Show
        cnauroth Chris Nauroth added a comment - Hello Gaurav Kanade . I would not suggest trying to write a test for this against live Azure Storage. The patch relates to client-side error handling of server-side errors. I don't think there is a reliable way to force the service to return those errors. Instead, I was thinking we might be able to use mocks to simulate the errors and confirm that it can fall back to a client-side copy. TestAzureFileSystemErrorConditions is an example of a test suite that already does something similar, so you might try looking at that one as a sample. Please take a look, and let me know if you think it can be done or if you think it's not feasible. Thanks!
        Hide
        gouravk Gaurav Kanade added a comment -

        Thanks Chris Nauroth for the clarification. I looked at the specific class of tests you mention. These tests seem to be designed to verify that if an invalid operation is attempted an error is indeed thrown. For e.g. testTransientErrorOnReadThrowsException, testAccessContainerWithWrongVersion throws exception and so on. In our scenario I do not believe the goal of the test is to check whether an exception is indeed raised; what we are doing is handling the exception thrown with the SERVER_BUSY error code. If anything needs to be tested it should be that the new copy mechanism copies accurately. I am not sure how we can use this suite of tests for this kind of verification. Like I mentioned earlier, we have tested that the copy happens correctly by manually imposing the mechanism (download first then copy) for rename. Or did you have something else in mind that you wanted tested ? Let me know your thoughts

        Show
        gouravk Gaurav Kanade added a comment - Thanks Chris Nauroth for the clarification. I looked at the specific class of tests you mention. These tests seem to be designed to verify that if an invalid operation is attempted an error is indeed thrown. For e.g. testTransientErrorOnReadThrowsException, testAccessContainerWithWrongVersion throws exception and so on. In our scenario I do not believe the goal of the test is to check whether an exception is indeed raised; what we are doing is handling the exception thrown with the SERVER_BUSY error code. If anything needs to be tested it should be that the new copy mechanism copies accurately. I am not sure how we can use this suite of tests for this kind of verification. Like I mentioned earlier, we have tested that the copy happens correctly by manually imposing the mechanism (download first then copy) for rename. Or did you have something else in mind that you wanted tested ? Let me know your thoughts
        Hide
        cnauroth Chris Nauroth added a comment -

        Gaurav Kanade, thank you for taking a look.

        I was hopeful that we might be able to hook in some failure simulation in a test, similar to the TestAzureFileSystemErrorConditions#injectTransientError method. I just spent some time experimenting with this and trying to hook on to the various event listeners exposed by the Azure Storage SDK. Unfortunately, these don't appear to give us a deep enough hook for the failure injection I had in mind. I don't see a way to rewrite the whole outbound HTTP response to make it look like a server-busy error. I think your manual testing will have to suffice for this patch.

        There is still one more piece of unresolved feedback on patch v06. Please see my comment from 21/Sep/15 about guaranteeing that the streams get closed. After that is addressed, I expect this patch will be ready to go. Thanks!

        Show
        cnauroth Chris Nauroth added a comment - Gaurav Kanade , thank you for taking a look. I was hopeful that we might be able to hook in some failure simulation in a test, similar to the TestAzureFileSystemErrorConditions#injectTransientError method. I just spent some time experimenting with this and trying to hook on to the various event listeners exposed by the Azure Storage SDK. Unfortunately, these don't appear to give us a deep enough hook for the failure injection I had in mind. I don't see a way to rewrite the whole outbound HTTP response to make it look like a server-busy error. I think your manual testing will have to suffice for this patch. There is still one more piece of unresolved feedback on patch v06. Please see my comment from 21/Sep/15 about guaranteeing that the streams get closed. After that is addressed, I expect this patch will be ready to go. Thanks!
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 10s docker + precommit patch detected.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 3m 10s trunk passed
        +1 compile 0m 15s trunk passed with JDK v1.8.0_60
        +1 compile 0m 12s trunk passed with JDK v1.7.0_79
        +1 checkstyle 0m 9s trunk passed
        +1 mvneclipse 0m 11s trunk passed
        +1 findbugs 0m 31s trunk passed
        +1 javadoc 0m 12s trunk passed with JDK v1.8.0_60
        +1 javadoc 0m 14s trunk passed with JDK v1.7.0_79
        +1 mvninstall 0m 15s the patch passed
        +1 compile 0m 11s the patch passed with JDK v1.8.0_60
        +1 javac 0m 11s the patch passed
        +1 compile 0m 13s the patch passed with JDK v1.7.0_79
        +1 javac 0m 13s the patch passed
        +1 checkstyle 0m 7s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) with tabs.
        +1 findbugs 0m 38s the patch passed
        +1 javadoc 0m 12s the patch passed with JDK v1.8.0_60
        +1 javadoc 0m 13s the patch passed with JDK v1.7.0_79
        +1 unit 1m 1s hadoop-azure in the patch passed with JDK v1.8.0_60.
        +1 unit 1m 12s hadoop-azure in the patch passed with JDK v1.7.0_79.
        -1 asflicense 0m 24s Patch generated 1 ASF License warnings.
        10m 39s



        Subsystem Report/Notes
        Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-22
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12767910/HADOOP-12334.07.patch
        JIRA Issue HADOOP-12334
        Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
        uname Linux 3ab2b3605b1f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-28a3a3d/dev-support/personality/hadoop.sh
        git revision trunk / 3dadf36
        Default Java 1.7.0_79
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7908/artifact/patchprocess/whitespace-tabs.txt
        JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7908/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/7908/artifact/patchprocess/patch-asflicense-problems.txt
        Max memory used 223MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7908/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 3m 10s trunk passed +1 compile 0m 15s trunk passed with JDK v1.8.0_60 +1 compile 0m 12s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 9s trunk passed +1 mvneclipse 0m 11s trunk passed +1 findbugs 0m 31s trunk passed +1 javadoc 0m 12s trunk passed with JDK v1.8.0_60 +1 javadoc 0m 14s trunk passed with JDK v1.7.0_79 +1 mvninstall 0m 15s the patch passed +1 compile 0m 11s the patch passed with JDK v1.8.0_60 +1 javac 0m 11s the patch passed +1 compile 0m 13s the patch passed with JDK v1.7.0_79 +1 javac 0m 13s the patch passed +1 checkstyle 0m 7s the patch passed +1 mvneclipse 0m 12s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) with tabs. +1 findbugs 0m 38s the patch passed +1 javadoc 0m 12s the patch passed with JDK v1.8.0_60 +1 javadoc 0m 13s the patch passed with JDK v1.7.0_79 +1 unit 1m 1s hadoop-azure in the patch passed with JDK v1.8.0_60. +1 unit 1m 12s hadoop-azure in the patch passed with JDK v1.7.0_79. -1 asflicense 0m 24s Patch generated 1 ASF License warnings. 10m 39s Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-22 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12767910/HADOOP-12334.07.patch JIRA Issue HADOOP-12334 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux 3ab2b3605b1f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-28a3a3d/dev-support/personality/hadoop.sh git revision trunk / 3dadf36 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7908/artifact/patchprocess/whitespace-tabs.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7908/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/7908/artifact/patchprocess/patch-asflicense-problems.txt Max memory used 223MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7908/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 7s docker + precommit patch detected.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 3m 19s trunk passed
        +1 compile 0m 14s trunk passed with JDK v1.8.0_60
        +1 compile 0m 13s trunk passed with JDK v1.7.0_79
        +1 checkstyle 0m 8s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 30s trunk passed
        +1 javadoc 0m 12s trunk passed with JDK v1.8.0_60
        +1 javadoc 0m 14s trunk passed with JDK v1.7.0_79
        +1 mvninstall 0m 15s the patch passed
        +1 compile 0m 13s the patch passed with JDK v1.8.0_60
        +1 javac 0m 13s the patch passed
        +1 compile 0m 14s the patch passed with JDK v1.7.0_79
        +1 javac 0m 14s the patch passed
        +1 checkstyle 0m 7s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 39s the patch passed
        +1 javadoc 0m 12s the patch passed with JDK v1.8.0_60
        +1 javadoc 0m 14s the patch passed with JDK v1.7.0_79
        +1 unit 1m 0s hadoop-azure in the patch passed with JDK v1.8.0_60.
        +1 unit 1m 13s hadoop-azure in the patch passed with JDK v1.7.0_79.
        -1 asflicense 0m 26s Patch generated 1 ASF License warnings.
        10m 51s



        Subsystem Report/Notes
        Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-22
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12768076/HADOOP-12334.07.patch
        JIRA Issue HADOOP-12334
        Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
        uname Linux c219b199ed65 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-28a3a3d/dev-support/personality/hadoop.sh
        git revision trunk / 4c0bae2
        Default Java 1.7.0_79
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
        findbugs v3.0.0
        JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7912/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/7912/artifact/patchprocess/patch-asflicense-problems.txt
        Max memory used 228MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7912/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 7s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 3m 19s trunk passed +1 compile 0m 14s trunk passed with JDK v1.8.0_60 +1 compile 0m 13s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 8s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 30s trunk passed +1 javadoc 0m 12s trunk passed with JDK v1.8.0_60 +1 javadoc 0m 14s trunk passed with JDK v1.7.0_79 +1 mvninstall 0m 15s the patch passed +1 compile 0m 13s the patch passed with JDK v1.8.0_60 +1 javac 0m 13s the patch passed +1 compile 0m 14s the patch passed with JDK v1.7.0_79 +1 javac 0m 14s the patch passed +1 checkstyle 0m 7s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 39s the patch passed +1 javadoc 0m 12s the patch passed with JDK v1.8.0_60 +1 javadoc 0m 14s the patch passed with JDK v1.7.0_79 +1 unit 1m 0s hadoop-azure in the patch passed with JDK v1.8.0_60. +1 unit 1m 13s hadoop-azure in the patch passed with JDK v1.7.0_79. -1 asflicense 0m 26s Patch generated 1 ASF License warnings. 10m 51s Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-22 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12768076/HADOOP-12334.07.patch JIRA Issue HADOOP-12334 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux c219b199ed65 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-28a3a3d/dev-support/personality/hadoop.sh git revision trunk / 4c0bae2 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7912/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/7912/artifact/patchprocess/patch-asflicense-problems.txt Max memory used 228MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7912/console This message was automatically generated.
        Hide
        cnauroth Chris Nauroth added a comment -

        +1 for patch v07. I have committed this to trunk and branch-2. Gaurav Kanade, thank you for the contribution.

        Show
        cnauroth Chris Nauroth added a comment - +1 for patch v07. I have committed this to trunk and branch-2. Gaurav Kanade , thank you for the contribution.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #570 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/570/)
        HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to (cnauroth: rev 47641fcbc9c41f4a338d8899501e4a310d2e81ad)

        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #570 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/570/ ) HADOOP-12334 . Change Mode Of Copy Operation of HBase WAL Archiving to (cnauroth: rev 47641fcbc9c41f4a338d8899501e4a310d2e81ad) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8691 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8691/)
        HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to (cnauroth: rev 47641fcbc9c41f4a338d8899501e4a310d2e81ad)

        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8691 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8691/ ) HADOOP-12334 . Change Mode Of Copy Operation of HBase WAL Archiving to (cnauroth: rev 47641fcbc9c41f4a338d8899501e4a310d2e81ad) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #1306 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1306/)
        HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to (cnauroth: rev 47641fcbc9c41f4a338d8899501e4a310d2e81ad)

        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #1306 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1306/ ) HADOOP-12334 . Change Mode Of Copy Operation of HBase WAL Archiving to (cnauroth: rev 47641fcbc9c41f4a338d8899501e4a310d2e81ad) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #585 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/585/)
        HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to (cnauroth: rev 47641fcbc9c41f4a338d8899501e4a310d2e81ad)

        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #585 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/585/ ) HADOOP-12334 . Change Mode Of Copy Operation of HBase WAL Archiving to (cnauroth: rev 47641fcbc9c41f4a338d8899501e4a310d2e81ad) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2517 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2517/)
        HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to (cnauroth: rev 47641fcbc9c41f4a338d8899501e4a310d2e81ad)

        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2517 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2517/ ) HADOOP-12334 . Change Mode Of Copy Operation of HBase WAL Archiving to (cnauroth: rev 47641fcbc9c41f4a338d8899501e4a310d2e81ad) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2465 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2465/)
        HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to (cnauroth: rev 47641fcbc9c41f4a338d8899501e4a310d2e81ad)

        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2465 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2465/ ) HADOOP-12334 . Change Mode Of Copy Operation of HBase WAL Archiving to (cnauroth: rev 47641fcbc9c41f4a338d8899501e4a310d2e81ad) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #528 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/528/)
        HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to (cnauroth: rev 47641fcbc9c41f4a338d8899501e4a310d2e81ad)

        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #528 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/528/ ) HADOOP-12334 . Change Mode Of Copy Operation of HBase WAL Archiving to (cnauroth: rev 47641fcbc9c41f4a338d8899501e4a310d2e81ad) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java hadoop-common-project/hadoop-common/CHANGES.txt

          People

          • Assignee:
            gouravk Gaurav Kanade
            Reporter:
            gouravk Gaurav Kanade
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Due:
              Created:
              Updated:
              Resolved:

              Development