Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9290

DFSClient#callAppend() is not backward compatible for slightly older NameNodes

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.8.0, 2.7.2, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      HDFS-7210 combined 2 RPC calls used at file append into a single one. Specifically getFileInfo() is combined with append(). While backward compatibility for older client is handled by the new NameNode (protobuf). Newer client's append() call does not work with older NameNodes. One will run into an exception like the following:

      java.lang.NullPointerException
              at org.apache.hadoop.hdfs.DFSOutputStream.isLazyPersist(DFSOutputStream.java:1741)
              at org.apache.hadoop.hdfs.DFSOutputStream.getChecksum4Compute(DFSOutputStream.java:1550)
              at org.apache.hadoop.hdfs.DFSOutputStream.<init>(DFSOutputStream.java:1560)
              at org.apache.hadoop.hdfs.DFSOutputStream.<init>(DFSOutputStream.java:1670)
              at org.apache.hadoop.hdfs.DFSOutputStream.newStreamForAppend(DFSOutputStream.java:1717)
              at org.apache.hadoop.hdfs.DFSClient.callAppend(DFSClient.java:1861)
              at org.apache.hadoop.hdfs.DFSClient.append(DFSClient.java:1922)
              at org.apache.hadoop.hdfs.DFSClient.append(DFSClient.java:1892)
              at org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:340)
              at org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:336)
              at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
              at org.apache.hadoop.hdfs.DistributedFileSystem.append(DistributedFileSystem.java:336)
              at org.apache.hadoop.hdfs.DistributedFileSystem.append(DistributedFileSystem.java:318)
              at org.apache.hadoop.fs.FileSystem.append(FileSystem.java:1164)
      

      The cause is that the new client code is expecting both the last block and file info in the same RPC but the old NameNode only replied with the first. The exception itself does not reflect this and one will have to look at the HDFS source code to really understand what happened.

      We can have the client detect it's talking to a old NameNode and send an extra getFileInfo() RPC. Or we should improve the exception being thrown to accurately reflect the cause of failure.

      1. HDFS-9290.002.patch
        1 kB
        Tony Wu
      2. HDFS-9290.001.patch
        1 kB
        Tony Wu

        Issue Links

          Activity

          Hide
          twu Tony Wu added a comment -

          HI Kihwal Lee,
          Thanks for taking the time to manually run these tests. I didn't know Hadoop QA does not kick off test runs for client changes. Will make sure I include my manual run results in the future.
          Thanks,
          Tony

          Show
          twu Tony Wu added a comment - HI Kihwal Lee , Thanks for taking the time to manually run these tests. I didn't know Hadoop QA does not kick off test runs for client changes. Will make sure I include my manual run results in the future. Thanks, Tony
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2468 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2468/)
          HDFS-9290. DFSClient#callAppend() is not backward compatible for (kihwal: rev b9e0417bdf2b9655dc4256bdb43683eca1ab46be)

          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2468 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2468/ ) HDFS-9290 . DFSClient#callAppend() is not backward compatible for (kihwal: rev b9e0417bdf2b9655dc4256bdb43683eca1ab46be) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #532 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/532/)
          HDFS-9290. DFSClient#callAppend() is not backward compatible for (kihwal: rev b9e0417bdf2b9655dc4256bdb43683eca1ab46be)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #532 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/532/ ) HDFS-9290 . DFSClient#callAppend() is not backward compatible for (kihwal: rev b9e0417bdf2b9655dc4256bdb43683eca1ab46be) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2522 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2522/)
          HDFS-9290. DFSClient#callAppend() is not backward compatible for (kihwal: rev b9e0417bdf2b9655dc4256bdb43683eca1ab46be)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2522 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2522/ ) HDFS-9290 . DFSClient#callAppend() is not backward compatible for (kihwal: rev b9e0417bdf2b9655dc4256bdb43683eca1ab46be) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #590 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/590/)
          HDFS-9290. DFSClient#callAppend() is not backward compatible for (kihwal: rev b9e0417bdf2b9655dc4256bdb43683eca1ab46be)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #590 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/590/ ) HDFS-9290 . DFSClient#callAppend() is not backward compatible for (kihwal: rev b9e0417bdf2b9655dc4256bdb43683eca1ab46be) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #577 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/577/)
          HDFS-9290. DFSClient#callAppend() is not backward compatible for (kihwal: rev b9e0417bdf2b9655dc4256bdb43683eca1ab46be)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #577 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/577/ ) HDFS-9290 . DFSClient#callAppend() is not backward compatible for (kihwal: rev b9e0417bdf2b9655dc4256bdb43683eca1ab46be) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1312 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1312/)
          HDFS-9290. DFSClient#callAppend() is not backward compatible for (kihwal: rev b9e0417bdf2b9655dc4256bdb43683eca1ab46be)

          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1312 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1312/ ) HDFS-9290 . DFSClient#callAppend() is not backward compatible for (kihwal: rev b9e0417bdf2b9655dc4256bdb43683eca1ab46be) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8698 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8698/)
          HDFS-9290. DFSClient#callAppend() is not backward compatible for (kihwal: rev b9e0417bdf2b9655dc4256bdb43683eca1ab46be)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8698 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8698/ ) HDFS-9290 . DFSClient#callAppend() is not backward compatible for (kihwal: rev b9e0417bdf2b9655dc4256bdb43683eca1ab46be) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          Hide
          kihwal Kihwal Lee added a comment -

          Tony Wu, thanks for reporting this and providing a fix. I've committed this to trunk, branch-2 and branch-2.7.

          Show
          kihwal Kihwal Lee added a comment - Tony Wu , thanks for reporting this and providing a fix. I've committed this to trunk, branch-2 and branch-2.7.
          Hide
          kihwal Kihwal Lee added a comment - - edited

          The patch looks good +1.

          Show
          kihwal Kihwal Lee added a comment - - edited The patch looks good +1.
          Hide
          kihwal Kihwal Lee added a comment -

          Since no test was run for hdoop-hdfs-client, I ran the tests manually. No need to run server-side tests, since this is client-only change.

          -------------------------------------------------------
          T E S T S
          -------------------------------------------------------
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.fs.contract.hdfs.TestHDFSContractAppend
          Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.864 sec - in org.apache.hadoop.fs.contract.hdfs.TestHDFSContractAppend
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.server.namenode.ha.TestHAAppend
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.835 sec - in org.apache.hadoop.hdfs.server.namenode.ha.TestHAAppend
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.TestFileAppend3
          Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 48.2 sec - in org.apache.hadoop.hdfs.TestFileAppend3
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.TestFileAppendRestart
          Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.929 sec - in org.apache.hadoop.hdfs.TestFileAppendRestart
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.TestFileAppend2
          Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.247 sec - in org.apache.hadoop.hdfs.TestFileAppend2
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.TestFileAppend
          Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 32.374 sec - in org.apache.hadoop.hdfs.TestFileAppend
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.TestAppendSnapshotTruncate
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.968 sec - in org.apache.hadoop.hdfs.TestAppendSnapshotTruncate
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.FileAppendTest4
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 77.958 sec - in org.apache.hadoop.hdfs.FileAppendTest4
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.TestFileAppend4
          Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 37.462 sec - in org.apache.hadoop.hdfs.TestFileAppend4
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.TestAppendDifferentChecksum
          Tests run: 3, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 9.81 sec - in org.apache.hadoop.hdfs.TestAppendDifferentChecksum

          Results :

          Tests run: 48, Failures: 0, Errors: 0, Skipped: 1

          Show
          kihwal Kihwal Lee added a comment - Since no test was run for hdoop-hdfs-client , I ran the tests manually. No need to run server-side tests, since this is client-only change. ------------------------------------------------------- T E S T S ------------------------------------------------------- Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.fs.contract.hdfs.TestHDFSContractAppend Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.864 sec - in org.apache.hadoop.fs.contract.hdfs.TestHDFSContractAppend Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.server.namenode.ha.TestHAAppend Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.835 sec - in org.apache.hadoop.hdfs.server.namenode.ha.TestHAAppend Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.TestFileAppend3 Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 48.2 sec - in org.apache.hadoop.hdfs.TestFileAppend3 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.TestFileAppendRestart Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.929 sec - in org.apache.hadoop.hdfs.TestFileAppendRestart Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.TestFileAppend2 Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.247 sec - in org.apache.hadoop.hdfs.TestFileAppend2 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.TestFileAppend Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 32.374 sec - in org.apache.hadoop.hdfs.TestFileAppend Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.TestAppendSnapshotTruncate Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.968 sec - in org.apache.hadoop.hdfs.TestAppendSnapshotTruncate Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.FileAppendTest4 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 77.958 sec - in org.apache.hadoop.hdfs.FileAppendTest4 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.TestFileAppend4 Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 37.462 sec - in org.apache.hadoop.hdfs.TestFileAppend4 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.TestAppendDifferentChecksum Tests run: 3, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 9.81 sec - in org.apache.hadoop.hdfs.TestAppendDifferentChecksum Results : Tests run: 48, Failures: 0, Errors: 0, Skipped: 1
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 19m 26s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 9m 19s There were no new javac warning messages.
          +1 javadoc 11m 55s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 53s The applied patch generated 1 new checkstyle issues (total was 55, now 55).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 36s mvn install still works.
          +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse.
          +1 findbugs 2m 18s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 34s Pre-build of native portion
          +1 hdfs tests 0m 31s Tests passed in hadoop-hdfs-client.
              51m 35s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12768311/HDFS-9290.002.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 35a303d
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13154/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt
          hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/13154/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13154/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13154/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 19m 26s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 9m 19s There were no new javac warning messages. +1 javadoc 11m 55s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 53s The applied patch generated 1 new checkstyle issues (total was 55, now 55). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse. +1 findbugs 2m 18s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 34s Pre-build of native portion +1 hdfs tests 0m 31s Tests passed in hadoop-hdfs-client.     51m 35s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768311/HDFS-9290.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 35a303d checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13154/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/13154/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13154/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13154/console This message was automatically generated.
          Hide
          twu Tony Wu added a comment -

          Hi Kihwal Lee,

          Thanks a lot for taking the time to look at the patch! I have addressed your comment by changing the log level. Please take a look and let me know if you have any other comments.

          Regards,
          Tony Wu

          Show
          twu Tony Wu added a comment - Hi Kihwal Lee , Thanks a lot for taking the time to look at the patch! I have addressed your comment by changing the log level. Please take a look and let me know if you have any other comments. Regards, Tony Wu
          Hide
          kihwal Kihwal Lee added a comment -

          The fix looks good. One minor nit is that logging at INFO can sometimes be noisy. I think end-users rarely care about the fact that it is talking to an older namenode. Let's make it DEBUG.

          Show
          kihwal Kihwal Lee added a comment - The fix looks good. One minor nit is that logging at INFO can sometimes be noisy. I think end-users rarely care about the fact that it is talking to an older namenode. Let's make it DEBUG .
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 23m 24s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 10m 15s There were no new javac warning messages.
          +1 javadoc 13m 31s There were no new javadoc warning messages.
          +1 release audit 0m 32s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 58s The applied patch generated 1 new checkstyle issues (total was 55, now 55).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 56s mvn install still works.
          +1 eclipse:eclipse 0m 42s The patch built with eclipse:eclipse.
          +1 findbugs 2m 41s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 4m 4s Pre-build of native portion
          +1 hdfs tests 0m 39s Tests passed in hadoop-hdfs-client.
              59m 47s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12768126/HDFS-9290.001.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 0fce5f9
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13137/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt
          hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/13137/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13137/testReport/
          Java 1.7.0_55
          uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13137/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 23m 24s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 10m 15s There were no new javac warning messages. +1 javadoc 13m 31s There were no new javadoc warning messages. +1 release audit 0m 32s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 58s The applied patch generated 1 new checkstyle issues (total was 55, now 55). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 56s mvn install still works. +1 eclipse:eclipse 0m 42s The patch built with eclipse:eclipse. +1 findbugs 2m 41s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 4m 4s Pre-build of native portion +1 hdfs tests 0m 39s Tests passed in hadoop-hdfs-client.     59m 47s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768126/HDFS-9290.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 0fce5f9 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13137/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/13137/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13137/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13137/console This message was automatically generated.
          Hide
          twu Tony Wu added a comment -

          In this patch:

          • Detect that DFSClient is talking to an older NameNode and make an extra getFileInfo() RPC call to request file info.
          Show
          twu Tony Wu added a comment - In this patch: Detect that DFSClient is talking to an older NameNode and make an extra getFileInfo() RPC call to request file info.

            People

            • Assignee:
              twu Tony Wu
              Reporter:
              twu Tony Wu
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development