Hadoop Common
  1. Hadoop Common
  2. HADOOP-1838

Files created with an pre-0.15 gets blocksize as zero, causing performance degradation

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.15.0
    • Fix Version/s: 0.15.0
    • Component/s: None
    • Labels:
      None

      Description

      HADOOP-1656 introduced the support for storing block size persistently as inode metadata. Previously, if the file has only one block then it was not possible to accurately determine the blocksize that the application has requested at file-creation time.

      The upgrade of an older layout to the new layout kept the blocksize as zero for single-block files that were upgraded to the new layout. This was done to indicate the DFS really does not know the "true" blocksize of this file. This caused map-reduce to determine that a split is 1 byte in length!

      1. blockSizeZero.patch
        5 kB
        dhruba borthakur

        Issue Links

          Activity

          dhruba borthakur created issue -
          dhruba borthakur made changes -
          Field Original Value New Value
          Link This issue is related to HADOOP-1656 [ HADOOP-1656 ]
          dhruba borthakur made changes -
          Attachment blockSizeZero.patch [ 12365147 ]
          Owen O'Malley made changes -
          Fix Version/s 0.15.0 [ 12312565 ]
          Priority Major [ 3 ] Blocker [ 1 ]
          dhruba borthakur made changes -
          Attachment blockSizeZero.patch [ 12365147 ]
          dhruba borthakur made changes -
          Attachment blockSizeZero.patch [ 12365210 ]
          Owen O'Malley made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          dhruba borthakur made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Doug Cutting made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Owen O'Malley made changes -
          Component/s dfs [ 12310710 ]

            People

            • Assignee:
              dhruba borthakur
              Reporter:
              dhruba borthakur
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development