Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0-alpha, 3.0.0
    • Component/s: contrib/raid
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

          [javac] Compiling 33 source files to /Users/szetszwo/hadoop/t2/hadoop-mapreduce-project/build/contrib/raid/classes
          [javac] /Users/szetszwo/hadoop/t2/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRaidUtil.java:42:
       unreported exception org.apache.hadoop.ipc.StandbyException; must be caught or declared to be thrown
          [javac]     return namesystem.getFileInfo(src, resolveLink);
          [javac]                                  ^
      

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1046 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1046/)
          MAPREDUCE-4057. Update RAID for the HA and fsdataset changes. Contributed by Devaraj K (Revision 1311959)

          Result = SUCCESS
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311959
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRaidUtil.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1046 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1046/ ) MAPREDUCE-4057 . Update RAID for the HA and fsdataset changes. Contributed by Devaraj K (Revision 1311959) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311959 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRaidUtil.java /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1011 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1011/)
          MAPREDUCE-4057. Update RAID for the HA and fsdataset changes. Contributed by Devaraj K (Revision 1311959)

          Result = FAILURE
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311959
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRaidUtil.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1011 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1011/ ) MAPREDUCE-4057 . Update RAID for the HA and fsdataset changes. Contributed by Devaraj K (Revision 1311959) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311959 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRaidUtil.java /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2055 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2055/)
          MAPREDUCE-4057. Update RAID for the HA and fsdataset changes. Contributed by Devaraj K (Revision 1311959)

          Result = ABORTED
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311959
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRaidUtil.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2055 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2055/ ) MAPREDUCE-4057 . Update RAID for the HA and fsdataset changes. Contributed by Devaraj K (Revision 1311959) Result = ABORTED szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311959 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRaidUtil.java /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2042 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2042/)
          MAPREDUCE-4057. Update RAID for the HA and fsdataset changes. Contributed by Devaraj K (Revision 1311959)

          Result = SUCCESS
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311959
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRaidUtil.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2042 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2042/ ) MAPREDUCE-4057 . Update RAID for the HA and fsdataset changes. Contributed by Devaraj K (Revision 1311959) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311959 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRaidUtil.java /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2116 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2116/)
          MAPREDUCE-4057. Update RAID for the HA and fsdataset changes. Contributed by Devaraj K (Revision 1311959)

          Result = SUCCESS
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311959
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRaidUtil.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2116 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2116/ ) MAPREDUCE-4057 . Update RAID for the HA and fsdataset changes. Contributed by Devaraj K (Revision 1311959) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1311959 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRaidUtil.java /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have committed this. Thanks, Devaraj!

          Show
          Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Devaraj!
          Hide
          Tsz Wo Nicholas Sze added a comment -

          The ant command did not work when running in hadoop-mapreduce-project/src/contrib/raid/ but it worked when running it in hadoop-mapreduce-project/ . Thanks for the hint, Devaraj.

          +1 patch looks good.

          Show
          Tsz Wo Nicholas Sze added a comment - The ant command did not work when running in hadoop-mapreduce-project/src/contrib/raid/ but it worked when running it in hadoop-mapreduce-project/ . Thanks for the hint, Devaraj. +1 patch looks good.
          Hide
          Devaraj K added a comment -

          Thanks Nicholas for looking into the patch.

          I am not getting any errors after applying the patch in my env.

          compile-test:
               [echo] contrib: raid
              [javac] D:\opensource\hadoop-trunk\hadoop-mapreduce-project\src\contrib\build-contrib.xml:227: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
          
          init:
          
          ivy-download:
          

          As per the error, it is giving MiniMRCluster cannot find.

          Can you please check any jar is missing or am I doing anything wrong?

          Show
          Devaraj K added a comment - Thanks Nicholas for looking into the patch. I am not getting any errors after applying the patch in my env. compile-test: [echo] contrib: raid [javac] D:\opensource\hadoop-trunk\hadoop-mapreduce-project\src\contrib\build-contrib.xml:227: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds init: ivy-download: As per the error, it is giving MiniMRCluster cannot find. Can you please check any jar is missing or am I doing anything wrong?
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Devaraj, thanks for posting a patch. Could you also fix the errors in the tests?

          Show
          Tsz Wo Nicholas Sze added a comment - Devaraj, thanks for posting a patch. Could you also fix the errors in the tests?
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Raid still cannot be compiled after the patch:

          compile-test:
               [echo] contrib: raid
              [javac] /Users/szetszwo/hadoop/t-commit/hadoop-mapreduce-project/src/contrib/build-contrib.xml:227: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
              [javac] Compiling 20 source files to /Users/szetszwo/hadoop/t-commit/hadoop-mapreduce-project/build/contrib/raid/test
              [javac] /Users/szetszwo/hadoop/t-commit/hadoop-mapreduce-project/src/contrib/raid/src/test/org/apache/hadoop/raid/TestBlockFixer.java:42: cannot find symbol
              [javac] symbol  : class MiniMRCluster
              [javac] location: package org.apache.hadoop.mapred
              [javac] import org.apache.hadoop.mapred.MiniMRCluster;
              [javac]                                ^
              ...
              [javac] Note: Some input files use or override a deprecated API.
              [javac] Note: Recompile with -Xlint:deprecation for details.
              [javac] 12 errors
          
          Show
          Tsz Wo Nicholas Sze added a comment - Raid still cannot be compiled after the patch: compile-test: [echo] contrib: raid [javac] /Users/szetszwo/hadoop/t-commit/hadoop-mapreduce-project/src/contrib/build-contrib.xml:227: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds [javac] Compiling 20 source files to /Users/szetszwo/hadoop/t-commit/hadoop-mapreduce-project/build/contrib/raid/test [javac] /Users/szetszwo/hadoop/t-commit/hadoop-mapreduce-project/src/contrib/raid/src/test/org/apache/hadoop/raid/TestBlockFixer.java:42: cannot find symbol [javac] symbol : class MiniMRCluster [javac] location: package org.apache.hadoop.mapred [javac] import org.apache.hadoop.mapred.MiniMRCluster; [javac] ^ ... [javac] Note: Some input files use or override a deprecated API. [javac] Note: Recompile with -Xlint:deprecation for details. [javac] 12 errors
          Hide
          Devaraj K added a comment -

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService
          org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry
          org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
          org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs

          These test failures are not related to this patch. Already MAPREDUCE-4094
          jira exist to address these failures.

          Show
          Devaraj K added a comment - -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs These test failures are not related to this patch. Already MAPREDUCE-4094 jira exist to address these failures.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12521892/MAPREDUCE-4057.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService
          org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry
          org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
          org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2172//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2172//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12521892/MAPREDUCE-4057.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2172//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2172//console This message is automatically generated.
          Hide
          Devaraj K added a comment -

          I have attached patch to fix these compilation errors.

          Show
          Devaraj K added a comment - I have attached patch to fix these compilation errors.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          HDFS-3089 is going to break RAID since it is changing FSDatasetInterface.

          -- ./src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java ----
          444:     private final FSDatasetInterface data;
          446:     private BlockInputStreamFactory(ExtendedBlock block, FSDatasetInt ...
             2 occurrence(s)
          

          However, RAID is already broken before HDFS-3089. It looks like that it was broken by the HA changes.

          Show
          Tsz Wo Nicholas Sze added a comment - HDFS-3089 is going to break RAID since it is changing FSDatasetInterface. -- ./src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java ---- 444: private final FSDatasetInterface data; 446: private BlockInputStreamFactory(ExtendedBlock block, FSDatasetInt ... 2 occurrence(s) However, RAID is already broken before HDFS-3089 . It looks like that it was broken by the HA changes.

            People

            • Assignee:
              Devaraj K
              Reporter:
              Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development