Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2282

Semi-harmless race between block reports and block invalidation

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Not A Problem
    • Affects Version/s: 0.20.203.0, 2.0.0-alpha
    • Fix Version/s: None
    • Component/s: datanode
    • Labels:
      None
    • Target Version/s:

      Description

      In the 0.20.203 codebase, block reports are not synchronized in any way against mutations to the actual file structure on disk. If a file is removed from a directory while the block report is scanning that directory, it will be mistakenly reported as existing with a length of 0, since File.length() on a non-existent file returns 0.

      This results in an error being logged on the DataNode when the NN sends it a second block deletion request for the already-deleted block. I believe it to be harmless, but the error message can concern users.

      This was fixed in the 0.20 code line in HDFS-2379. This jira remains open to track the port to 0.24.

      1. hdfs-2282-20.txt
        0.9 kB
        Todd Lipcon
      2. hdfs-2282-20.txt
        0.8 kB
        Todd Lipcon

        Issue Links

          Activity

          Todd Lipcon created issue -
          Hide
          Todd Lipcon added a comment -

          Here's a patch for 20. Before resolving we should write a unit test and also double check that this bug doesn't show up in trunk.

          Show
          Todd Lipcon added a comment - Here's a patch for 20. Before resolving we should write a unit test and also double check that this bug doesn't show up in trunk.
          Todd Lipcon made changes -
          Field Original Value New Value
          Attachment hdfs-2282-20.txt [ 12491437 ]
          Hide
          Todd Lipcon added a comment -

          oops, correcting slight error in previous upload

          Show
          Todd Lipcon added a comment - oops, correcting slight error in previous upload
          Todd Lipcon made changes -
          Attachment hdfs-2282-20.txt [ 12491438 ]
          Hide
          Eli Collins added a comment -

          Looks like this was fixed in HDFS-2379.

          Show
          Eli Collins added a comment - Looks like this was fixed in HDFS-2379 .
          Hide
          Todd Lipcon added a comment -

          Yea, fixed it in 20 with HDFS-2379, but might still need a small fix in trunk.

          Show
          Todd Lipcon added a comment - Yea, fixed it in 20 with HDFS-2379 , but might still need a small fix in trunk.
          Eli Collins made changes -
          Target Version/s 0.24.0 [ 12317653 ]
          Hide
          Eli Collins added a comment -

          Cool.. using our new fancy fix/target scheme to indicate this was fixed in 206 but needs a fix in 204.

          Show
          Eli Collins added a comment - Cool.. using our new fancy fix/target scheme to indicate this was fixed in 206 but needs a fix in 204.
          Eli Collins made changes -
          Fix Version/s 0.20.206.0 [ 12317959 ]
          Hide
          Matt Foley added a comment -

          Since this patch was not applied (from this jira) to 1.0, changing "fix version" usage to clean up release notes for 1.1.0.

          Show
          Matt Foley added a comment - Since this patch was not applied (from this jira) to 1.0, changing "fix version" usage to clean up release notes for 1.1.0.
          Matt Foley made changes -
          Fix Version/s 1.1.0 [ 12317959 ]
          Affects Version/s 0.24.0 [ 12317653 ]
          Description In the 0.20 codebase, block reports are not synchronized in any way against mutations to the actual file structure on disk. If a file is removed from a directory while the block report is scanning that directory, it will be mistakenly reported as existing with a length of 0, since File.length() on a non-existent file returns 0.

          This results in an error being logged on the DataNode when the NN sends it a second block deletion request for the already-deleted block. I believe it to be harmless, but the error message can concern users.
          In the 0.20.203 codebase, block reports are not synchronized in any way against mutations to the actual file structure on disk. If a file is removed from a directory while the block report is scanning that directory, it will be mistakenly reported as existing with a length of 0, since File.length() on a non-existent file returns 0.

          This results in an error being logged on the DataNode when the NN sends it a second block deletion request for the already-deleted block. I believe it to be harmless, but the error message can concern users.

          This was fixed in the 0.20 code line in HDFS-2379. This jira remains open to track the port to 0.24.
          Matt Foley made changes -
          Link This issue is related to HDFS-2379 [ HDFS-2379 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I believe this is "Not A Problem" anymore. Please feel free to reopen it if it is not the case.

          Show
          Tsz Wo Nicholas Sze added a comment - I believe this is "Not A Problem" anymore. Please feel free to reopen it if it is not the case.
          Tsz Wo Nicholas Sze made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Not A Problem [ 8 ]
          Allen Wittenauer made changes -
          Affects Version/s 2.0.0-alpha [ 12320353 ]
          Affects Version/s 0.24.0 [ 12317653 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          938d 20h 14m 1 Tsz Wo Nicholas Sze 19/Mar/14 23:07

            People

            • Assignee:
              Unassigned
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development