Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11959

WASB should configure client side socket timeout in storage client blob request options

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: tools
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      On clusters/jobs where mapred.task.timeout is set to a larger value, we noticed that tasks can sometimes get stuck on the below stack.

      Thread 1: (state = IN_NATIVE)
      - java.net.SocketInputStream.socketRead0(java.io.FileDescriptor, byte[], int, int, int) @bci=0 (Interpreted frame)
      - java.net.SocketInputStream.read(byte[], int, int, int) @bci=87, line=152 (Interpreted frame)
      - java.net.SocketInputStream.read(byte[], int, int) @bci=11, line=122 (Interpreted frame)
      - java.io.BufferedInputStream.fill() @bci=175, line=235 (Interpreted frame)
      - java.io.BufferedInputStream.read1(byte[], int, int) @bci=44, line=275 (Interpreted frame)
      - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=334 (Interpreted frame)
      - sun.net.www.MeteredStream.read(byte[], int, int) @bci=16, line=134 (Interpreted frame)
      - java.io.FilterInputStream.read(byte[], int, int) @bci=7, line=133 (Interpreted frame)
      - sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(byte[], int, int) @bci=4, line=3053 (Interpreted frame)
      - com.microsoft.azure.storage.core.NetworkInputStream.read(byte[], int, int) @bci=7, line=49 (Interpreted frame)
      - com.microsoft.azure.storage.blob.CloudBlob$10.postProcessResponse(java.net.HttpURLConnection, com.microsoft.azure.storage.blob.CloudBlob, com.microsoft.azure
      .storage.blob.CloudBlobClient, com.microsoft.azure.storage.OperationContext, java.lang.Integer) @bci=204, line=1691 (Interpreted frame)
      - com.microsoft.azure.storage.blob.CloudBlob$10.postProcessResponse(java.net.HttpURLConnection, java.lang.Object, java.lang.Object, com.microsoft.azure.storage
      .OperationContext, java.lang.Object) @bci=17, line=1613 (Interpreted frame)
      - com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(java.lang.Object, java.lang.Object, com.microsoft.azure.storage.core.StorageRequest, com.mi
      crosoft.azure.storage.RetryPolicyFactory, com.microsoft.azure.storage.OperationContext) @bci=352, line=148 (Interpreted frame)
      - com.microsoft.azure.storage.blob.CloudBlob.downloadRangeInternal(long, java.lang.Long, byte[], int, com.microsoft.azure.storage.AccessCondition, com.microsof
      t.azure.storage.blob.BlobRequestOptions, com.microsoft.azure.storage.OperationContext) @bci=131, line=1468 (Interpreted frame)
      - com.microsoft.azure.storage.blob.BlobInputStream.dispatchRead(int) @bci=31, line=255 (Interpreted frame)
      - com.microsoft.azure.storage.blob.BlobInputStream.readInternal(byte[], int, int) @bci=52, line=448 (Interpreted frame)
      - com.microsoft.azure.storage.blob.BlobInputStream.read(byte[], int, int) @bci=28, line=420 (Interpreted frame)
      - java.io.BufferedInputStream.read1(byte[], int, int) @bci=39, line=273 (Interpreted frame)
      - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=334 (Interpreted frame)
      - java.io.DataInputStream.read(byte[], int, int) @bci=7, line=149 (Interpreted frame)
      - org.apache.hadoop.fs.azure.NativeAzureFileSystem$NativeAzureFsInputStream.read(byte[], int, int) @bci=10, line=734 (Interpreted frame)
      - java.io.BufferedInputStream.read1(byte[], int, int) @bci=39, line=273 (Interpreted frame)
      - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=334 (Interpreted frame)
      - java.io.DataInputStream.read(byte[]) @bci=8, line=100 (Interpreted frame)
      - org.apache.hadoop.util.LineReader.fillBuffer(java.io.InputStream, byte[], boolean) @bci=2, line=180 (Interpreted frame)
      - org.apache.hadoop.util.LineReader.readDefaultLine(org.apache.hadoop.io.Text, int, int) @bci=64, line=216 (Compiled frame)
      - org.apache.hadoop.util.LineReader.readLine(org.apache.hadoop.io.Text, int, int) @bci=19, line=174 (Interpreted frame)
      - org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue() @bci=108, line=185 (Interpreted frame)
      - org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue() @bci=13, line=553 (Interpreted frame)
      - org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue() @bci=4, line=80 (Interpreted frame)
      - org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue() @bci=4, line=91 (Interpreted frame)
      - org.apache.hadoop.mapreduce.Mapper.run(org.apache.hadoop.mapreduce.Mapper$Context) @bci=6, line=144 (Interpreted frame)
      - org.apache.hadoop.mapred.MapTask.runNewMapper(org.apache.hadoop.mapred.JobConf, org.apache.hadoop.mapreduce.split.JobSplit$TaskSplitIndex, org.apache.hadoop.
      mapred.TaskUmbilicalProtocol, org.apache.hadoop.mapred.Task$TaskReporter) @bci=228, line=784 (Interpreted frame)
      - org.apache.hadoop.mapred.MapTask.run(org.apache.hadoop.mapred.JobConf, org.apache.hadoop.mapred.TaskUmbilicalProtocol) @bci=148, line=341 (Interpreted frame)
      
      - org.apache.hadoop.mapred.YarnChild$2.run() @bci=29, line=163 (Interpreted frame)
      - java.security.AccessController.doPrivileged(java.security.PrivilegedExceptionAction, java.security.AccessControlContext) @bci=0 (Interpreted frame)
      - javax.security.auth.Subject.doAs(javax.security.auth.Subject, java.security.PrivilegedExceptionAction) @bci=42, line=415 (Interpreted frame)
      - org.apache.hadoop.security.UserGroupInformation.doAs(java.security.PrivilegedExceptionAction) @bci=14, line=1628 (Interpreted frame)
      - org.apache.hadoop.mapred.YarnChild.main(java.lang.String[]) @bci=514, line=158 (Interpreted frame)
      

      The issue is that the storage client is by default not setting the socket timeout on its HTTP connections causing that in some (rare) circumstances we encounter a deadlock (e.g. whether the server on the other side just dies unexpectedly).

      The fix is to configure the maximum operation time on the storage client request options.

      1. HADOOP-11959.2.patch
        8 kB
        Ivan Mitic
      2. HADOOP-11959.patch
        0.4 kB
        Ivan Mitic

        Activity

        Hide
        ivanmi Ivan Mitic added a comment -

        Attaching the patch.

        The fix is to move to the latest Azure storage client SDK where the SDK internally sets the reasonable socket timeout on the connection. This is actually the right fix, as this also automatically provides means for client SDK to internally retry on timeout errors.

        Storage client SDK release notes:
        https://github.com/Azure/azure-storage-java/releases
        Changed the socket timeout to default to 5 minutes rather than infinite when neither service side timeout or maximum execution time are set.

        Show
        ivanmi Ivan Mitic added a comment - Attaching the patch. The fix is to move to the latest Azure storage client SDK where the SDK internally sets the reasonable socket timeout on the connection. This is actually the right fix, as this also automatically provides means for client SDK to internally retry on timeout errors. Storage client SDK release notes: https://github.com/Azure/azure-storage-java/releases Changed the socket timeout to default to 5 minutes rather than infinite when neither service side timeout or maximum execution time are set.
        Hide
        ivanmi Ivan Mitic added a comment -

        Attaching the second iteration.

        Given that storage client sdk fixed the below bug we can remove the previously added workaround for below issue (from release notes of 2.1.0):
        Fixed an issue in startCopyFromBlob where if the URI of the source blob contained certain non-ASCII characters they would not be encoded appropriately. This would result in Authorization failures.

        I validated that all existing mock and live unittests are passing with the latest patch.

        Show
        ivanmi Ivan Mitic added a comment - Attaching the second iteration. Given that storage client sdk fixed the below bug we can remove the previously added workaround for below issue (from release notes of 2.1.0): Fixed an issue in startCopyFromBlob where if the URI of the source blob contained certain non-ASCII characters they would not be encoded appropriately. This would result in Authorization failures. I validated that all existing mock and live unittests are passing with the latest patch.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 45s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 3 new or modified test files.
        +1 javac 7m 35s There were no new javac warning messages.
        +1 javadoc 9m 36s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 0m 22s The applied patch generated 1 new checkstyle issues (total was 37, now 37).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 33s mvn install still works.
        +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
        +1 findbugs 0m 41s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 tools/hadoop tests 1m 10s Tests passed in hadoop-azure.
            36m 43s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12735814/HADOOP-11959.2.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 50eeea1
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6854/artifact/patchprocess/diffcheckstylehadoop-azure.txt
        hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6854/artifact/patchprocess/testrun_hadoop-azure.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6854/testReport/
        Java 1.7.0_55
        uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6854/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 45s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 3 new or modified test files. +1 javac 7m 35s There were no new javac warning messages. +1 javadoc 9m 36s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 22s The applied patch generated 1 new checkstyle issues (total was 37, now 37). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. +1 findbugs 0m 41s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 1m 10s Tests passed in hadoop-azure.     36m 43s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12735814/HADOOP-11959.2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 50eeea1 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6854/artifact/patchprocess/diffcheckstylehadoop-azure.txt hadoop-azure test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6854/artifact/patchprocess/testrun_hadoop-azure.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6854/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6854/console This message was automatically generated.
        Hide
        cnauroth Chris Nauroth added a comment -

        Hi Ivan Mitic. This looks good. The checkstyle warning in the last pre-commit run isn't relevant.

        I just have one question. When the UTF-8 workarounds were done for authorization, I remember there was also a change made in AzureNativeFileSystemStore#storeLinkAttribute. (See below.) Now that the new SDK version has fixed the bug, do we need to remove this code too, or is this part of the fix permanent?

          private static void storeLinkAttribute(CloudBlobWrapper blob,
              String linkTarget) throws UnsupportedEncodingException {
            // We have to URL encode the link attribute as the link URI could
            // have URI special characters which unless encoded will result
            // in 403 errors from the server. This is due to metadata properties
            // being sent in the HTTP header of the request which is in turn used
            // on the server side to authorize the request.
            String encodedLinkTarget = null;
            if (linkTarget != null) {
              encodedLinkTarget = URLEncoder.encode(linkTarget, "UTF-8");
            }
            storeMetadataAttribute(blob,
                LINK_BACK_TO_UPLOAD_IN_PROGRESS_METADATA_KEY,
                encodedLinkTarget);
            // Remove the old metadata key if present
            removeMetadataAttribute(blob,
                OLD_LINK_BACK_TO_UPLOAD_IN_PROGRESS_METADATA_KEY);
          }
        
        Show
        cnauroth Chris Nauroth added a comment - Hi Ivan Mitic . This looks good. The checkstyle warning in the last pre-commit run isn't relevant. I just have one question. When the UTF-8 workarounds were done for authorization, I remember there was also a change made in AzureNativeFileSystemStore#storeLinkAttribute . (See below.) Now that the new SDK version has fixed the bug, do we need to remove this code too, or is this part of the fix permanent? private static void storeLinkAttribute(CloudBlobWrapper blob, String linkTarget) throws UnsupportedEncodingException { // We have to URL encode the link attribute as the link URI could // have URI special characters which unless encoded will result // in 403 errors from the server. This is due to metadata properties // being sent in the HTTP header of the request which is in turn used // on the server side to authorize the request. String encodedLinkTarget = null ; if (linkTarget != null ) { encodedLinkTarget = URLEncoder.encode(linkTarget, "UTF-8" ); } storeMetadataAttribute(blob, LINK_BACK_TO_UPLOAD_IN_PROGRESS_METADATA_KEY, encodedLinkTarget); // Remove the old metadata key if present removeMetadataAttribute(blob, OLD_LINK_BACK_TO_UPLOAD_IN_PROGRESS_METADATA_KEY); }
        Hide
        ivanmi Ivan Mitic added a comment -

        Thanks Chris for reviewing!

        Now that the new SDK version has fixed the bug, do we need to remove this code too, or is this part of the fix permanent?

        Good question. Blob metadata properties are not encoded by the client library, I asked the same question back when we discussed encoding issue with the client sdk team.

        Show
        ivanmi Ivan Mitic added a comment - Thanks Chris for reviewing! Now that the new SDK version has fixed the bug, do we need to remove this code too, or is this part of the fix permanent? Good question. Blob metadata properties are not encoded by the client library, I asked the same question back when we discussed encoding issue with the client sdk team.
        Hide
        cnauroth Chris Nauroth added a comment -

        +1 for the patch. Thank you for the clarification on that part of the change. I committed this to trunk and branch-2. Ivan, thank you for the contribution.

        Show
        cnauroth Chris Nauroth added a comment - +1 for the patch. Thank you for the clarification on that part of the change. I committed this to trunk and branch-2. Ivan, thank you for the contribution.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7920 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7920/)
        HADOOP-11959. WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic. (cnauroth: rev 94e7d49a6dab7e7f4e873dcca67e7fcc98e7e1f8)

        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-project/pom.xml
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterface.java
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestAzureFileSystemErrorConditions.java
        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/MockStorageInterface.java
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java
        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestBlobDataValidation.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7920 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7920/ ) HADOOP-11959 . WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic. (cnauroth: rev 94e7d49a6dab7e7f4e873dcca67e7fcc98e7e1f8) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-project/pom.xml hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterface.java hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestAzureFileSystemErrorConditions.java hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/MockStorageInterface.java hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestBlobDataValidation.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #200 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/200/)
        HADOOP-11959. WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic. (cnauroth: rev 94e7d49a6dab7e7f4e873dcca67e7fcc98e7e1f8)

        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestBlobDataValidation.java
        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/MockStorageInterface.java
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterface.java
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestAzureFileSystemErrorConditions.java
        • hadoop-project/pom.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #200 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/200/ ) HADOOP-11959 . WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic. (cnauroth: rev 94e7d49a6dab7e7f4e873dcca67e7fcc98e7e1f8) hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestBlobDataValidation.java hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/MockStorageInterface.java hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterface.java hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestAzureFileSystemErrorConditions.java hadoop-project/pom.xml
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #212 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/212/)
        HADOOP-11959. WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic. (cnauroth: rev 94e7d49a6dab7e7f4e873dcca67e7fcc98e7e1f8)

        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestAzureFileSystemErrorConditions.java
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java
        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestBlobDataValidation.java
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/MockStorageInterface.java
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterface.java
        • hadoop-project/pom.xml
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #212 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/212/ ) HADOOP-11959 . WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic. (cnauroth: rev 94e7d49a6dab7e7f4e873dcca67e7fcc98e7e1f8) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestAzureFileSystemErrorConditions.java hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestBlobDataValidation.java hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/MockStorageInterface.java hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterface.java hadoop-project/pom.xml
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #942 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/942/)
        HADOOP-11959. WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic. (cnauroth: rev 94e7d49a6dab7e7f4e873dcca67e7fcc98e7e1f8)

        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java
        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestAzureFileSystemErrorConditions.java
        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/MockStorageInterface.java
        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestBlobDataValidation.java
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterface.java
        • hadoop-project/pom.xml
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #942 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/942/ ) HADOOP-11959 . WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic. (cnauroth: rev 94e7d49a6dab7e7f4e873dcca67e7fcc98e7e1f8) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestAzureFileSystemErrorConditions.java hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/MockStorageInterface.java hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestBlobDataValidation.java hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterface.java hadoop-project/pom.xml
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2140 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2140/)
        HADOOP-11959. WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic. (cnauroth: rev 94e7d49a6dab7e7f4e873dcca67e7fcc98e7e1f8)

        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestBlobDataValidation.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestAzureFileSystemErrorConditions.java
        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/MockStorageInterface.java
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterface.java
        • hadoop-project/pom.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2140 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2140/ ) HADOOP-11959 . WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic. (cnauroth: rev 94e7d49a6dab7e7f4e873dcca67e7fcc98e7e1f8) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestBlobDataValidation.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestAzureFileSystemErrorConditions.java hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/MockStorageInterface.java hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterface.java hadoop-project/pom.xml
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #210 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/210/)
        HADOOP-11959. WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic. (cnauroth: rev 94e7d49a6dab7e7f4e873dcca67e7fcc98e7e1f8)

        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestBlobDataValidation.java
        • hadoop-project/pom.xml
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java
        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/MockStorageInterface.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestAzureFileSystemErrorConditions.java
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterface.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #210 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/210/ ) HADOOP-11959 . WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic. (cnauroth: rev 94e7d49a6dab7e7f4e873dcca67e7fcc98e7e1f8) hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestBlobDataValidation.java hadoop-project/pom.xml hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/MockStorageInterface.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestAzureFileSystemErrorConditions.java hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterface.java

          People

          • Assignee:
            ivanmi Ivan Mitic
            Reporter:
            ivanmi Ivan Mitic
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development