Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-927

DFSInputStream retries too many times for new block locations

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.20-append, 0.21.0, 0.22.0
    • Fix Version/s: 0.20.2, 0.21.0
    • Component/s: hdfs-client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      I think this is a regression caused by HDFS-127 – DFSInputStream is supposed to only go back to the NN max.block.acquires times, but in trunk it goes back twice as many - the default is 3, but I am counting 7 calls to getBlockLocations before an exception is thrown.

      1. hdfs-927-branch-0.21.txt
        14 kB
        Todd Lipcon
      2. hdfs-927-branch0.20.txt
        13 kB
        Todd Lipcon
      3. hdfs-927.txt
        14 kB
        Todd Lipcon

        Issue Links

          Activity

          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Nicolas Spiegelberg made changes -
          Affects Version/s 0.20-append [ 12315103 ]
          stack made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.20.2 [ 12314204 ]
          Fix Version/s 0.21.0 [ 12314046 ]
          Resolution Fixed [ 1 ]
          Todd Lipcon made changes -
          Attachment hdfs-927-branch-0.21.txt [ 12434482 ]
          Todd Lipcon made changes -
          Attachment hdfs-927-branch0.20.txt [ 12434481 ]
          Todd Lipcon made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Todd Lipcon made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Tsz Wo Nicholas Sze made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Tsz Wo Nicholas Sze made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Todd Lipcon made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Todd Lipcon made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Todd Lipcon made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Todd Lipcon made changes -
          Attachment hdfs-927.txt [ 12431471 ]
          Todd Lipcon made changes -
          Description I think this is a regression caused by HDFS-127 -- DFSInputStream is supposed to only go back to the NN max.block.acquires times, but in trunk it goes back twice as many - the default is 6, but I am counting 7 calls to getBlockLocations before an exception is thrown. I think this is a regression caused by HDFS-127 -- DFSInputStream is supposed to only go back to the NN max.block.acquires times, but in trunk it goes back twice as many - the default is 3, but I am counting 7 calls to getBlockLocations before an exception is thrown.
          Todd Lipcon made changes -
          Field Original Value New Value
          Link This issue relates to HDFS-127 [ HDFS-127 ]
          Todd Lipcon created issue -

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development