Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2727

libhdfs should get the default block size from the server

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 1.0.0, 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: libhdfs
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      libhdfs now uses the server block size configuration rather than the deprecated dfs.block.size client configuration.

      Description

      libhdfs uses the deprecated property dfs.block.size. It would be better for libhdfs to use the FileSystem#getDefaultBlockSize API instead.

      This API has two advantages:
      1. It uses the new, non-deprecated dfs.blocksize property
      2. It handles string escapes like 'M' for 'megabytes' etc. which can appear in dfs.blocksize and dfs.block.size

      1. HDFS-2727.002.patch
        14 kB
        Colin Patrick McCabe
      2. HDFS-2727.001.patch
        9 kB
        Colin Patrick McCabe

        Activity

        Sho Shimauchi created issue -
        Colin Patrick McCabe made changes -
        Field Original Value New Value
        Assignee Colin Patrick McCabe [ cmccabe ]
        Colin Patrick McCabe made changes -
        Attachment HDFS-2727.001.patch [ 12541291 ]
        Colin Patrick McCabe made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Colin Patrick McCabe made changes -
        Attachment HDFS-2727.002.patch [ 12541686 ]
        Colin Patrick McCabe made changes -
        Affects Version/s 2.2.0-alpha [ 12322472 ]
        Target Version/s 2.2.0-alpha [ 12322472 ]
        Description hdfs.c uses deprecated property dfs.block.size.
        It should use new property dfs.blocksize instead.
        libhdfs uses the deprecated property {{dfs.block.size}}. It would be better for libhdfs to use the FileSystem#getDefaultBlockSize API instead.

        This API has two advantages:
        1. It uses the new, non-deprecated {{dfs.blocksize}} property
        2. It handles string escapes like 'M' for 'megabytes' etc. which can appear in {{dfs.blocksize}} and {{dfs.block.size}}
        Eli Collins made changes -
        Summary hdfs.c uses deprecated property dfs.block.size libhdfs should get the default block size fro the server
        Hadoop Flags Reviewed [ 10343 ]
        Eli Collins made changes -
        Parent HDFS-1314 [ 12469946 ]
        Issue Type Sub-task [ 7 ] Improvement [ 4 ]
        Eli Collins made changes -
        Summary libhdfs should get the default block size fro the server libhdfs should get the default block size from the server
        Affects Version/s 2.0.0-alpha [ 12320353 ]
        Affects Version/s 1.0.0 [ 12318243 ]
        Affects Version/s 2.2.0-alpha [ 12322472 ]
        Component/s libhdfs [ 12313126 ]
        Eli Collins made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Target Version/s 2.2.0-alpha [ 12322472 ]
        Fix Version/s 2.2.0-alpha [ 12322472 ]
        Resolution Fixed [ 1 ]
        Eli Collins made changes -
        Hadoop Flags Reviewed [ 10343 ] Incompatible change,Reviewed [ 10342, 10343 ]
        Release Note libhdfs now uses the server block size configuration rather than the deprecated dfs.block.size client configuration.
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Colin Patrick McCabe
            Reporter:
            Sho Shimauchi
          • Votes:
            1 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development