Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-872

DFSClient 0.20.1 is incompatible with HDFS 0.20.2

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1, 0.20.2
    • Fix Version/s: 0.20.2
    • Component/s: datanode, hdfs-client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      After upgrading to that latest HDFS 0.20.2 (r896310 from /branches/branch-0.20), old DFS clients (0.20.1) seem to not work anymore. HBase uses the 0.20.1 hadoop core jars and the HBase master will no longer startup. Here is the exception from the HBase master log:

      2010-01-06 09:59:46,762 WARN org.apache.hadoop.hdfs.DFSClient: DFS Read: java.io.IOException: Could not obtain block: blk_338051
      2596555557728_1002 file=/hbase/hbase.version
              at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1788)
              at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:1616)
              at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1743)
              at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1673)
              at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:320)
              at java.io.DataInputStream.readUTF(DataInputStream.java:572)
              at org.apache.hadoop.hbase.util.FSUtils.getVersion(FSUtils.java:189)
              at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:208)
              at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:208)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
              at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
              at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
              at org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1241)
              at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1282)
      
      2010-01-06 09:59:46,763 FATAL org.apache.hadoop.hbase.master.HMaster: Not starting HMaster because:
      java.io.IOException: Could not obtain block: blk_3380512596555557728_1002 file=/hbase/hbase.version
              at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1788)
              at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:1616)
              at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1743)
              at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1673)
              at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:320)
              at java.io.DataInputStream.readUTF(DataInputStream.java:572)
              at org.apache.hadoop.hbase.util.FSUtils.getVersion(FSUtils.java:189)
              at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:208)
              at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:208)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
              at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
              at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
              at org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1241)
              at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1282)
      

      If I switch the hadoop jars in the hbase/lib directory with 0.20.2 version it works well, which what led me to open this bug here and not in the HBASE project.

      1. hdfs-872.txt
        8 kB
        Todd Lipcon
      2. hdfs-793-branch20.txt
        15 kB
        Todd Lipcon
      3. hdfs-793-branch20.txt
        15 kB
        Todd Lipcon

        Issue Links

          Activity

          Bassam Tabbara created issue -
          Allen Wittenauer made changes -
          Field Original Value New Value
          Link This issue is related to HDFS-793 [ HDFS-793 ]
          Todd Lipcon made changes -
          Attachment hdfs-872.txt [ 12430167 ]
          Todd Lipcon made changes -
          Assignee Todd Lipcon [ tlipcon ]
          Tsz Wo Nicholas Sze made changes -
          Component/s data-node [ 12312927 ]
          Component/s hdfs client [ 12312928 ]
          Todd Lipcon made changes -
          Attachment hdfs-793-branch20.txt [ 12430447 ]
          Todd Lipcon made changes -
          Attachment hdfs-793-branch20.txt [ 12430472 ]
          Hairong Kuang made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Bassam Tabbara
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development