Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1428

Make block size and the size of archive created files configurable.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: harchive
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently the block size used by archives is the default block size of the hdfs filesystem. We need to make it configurable so that the block size can be higher for the part files that archives create.
      Also, we need to make the size of part files in archives configurable again to make it bigger in size and create less number of such files.

      1. BinaryFileGenerator.java
        7 kB
        Tsz Wo Nicholas Sze
      2. BinaryFileGenerator.java
        10 kB
        Tsz Wo Nicholas Sze
      3. BinaryFileGenerator.java
        18 kB
        Tsz Wo Nicholas Sze
      4. MAPREDUCE-1428.patch
        9 kB
        Mahadev konar
      5. MAPREDUCE-1428.patch
        9 kB
        Mahadev konar

        Issue Links

          Activity

          Mahadev konar created issue -
          Mahadev konar made changes -
          Field Original Value New Value
          Link This issue is duplicated by MAPREDUCE-1465 [ MAPREDUCE-1465 ]
          Tsz Wo Nicholas Sze made changes -
          Link This issue is duplicated by MAPREDUCE-1509 [ MAPREDUCE-1509 ]
          Tsz Wo Nicholas Sze made changes -
          Attachment BinaryFileGenerator.java [ 12437506 ]
          Tsz Wo Nicholas Sze made changes -
          Attachment BinaryFileGenerator.java [ 12437581 ]
          Tsz Wo Nicholas Sze made changes -
          Attachment BinaryFileGenerator.java [ 12438055 ]
          Mahadev konar made changes -
          Attachment MAPREDUCE-1428.patch [ 12440165 ]
          Mahadev konar made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mahadev konar made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mahadev konar made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Tsz Wo Nicholas Sze made changes -
          Hadoop Flags [Reviewed]
          Mahadev konar made changes -
          Attachment MAPREDUCE-1428.patch [ 12440531 ]
          Tsz Wo Nicholas Sze made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Tsz Wo Nicholas Sze made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mahadev konar made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mahadev konar made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Tsz Wo Nicholas Sze made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Tom White made changes -
          Fix Version/s 0.21.0 [ 12314045 ]
          Fix Version/s 0.22.0 [ 12314184 ]
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Mahadev konar
              Reporter:
              Mahadev konar
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development