Hadoop Common
  1. Hadoop Common
  2. HADOOP-66

dfs client writes all data for a chunk to /tmp

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.1.0
    • Fix Version/s: 0.1.0
    • Component/s: None
    • Labels:
      None

      Description

      The dfs client writes all the data for the current chunk to a file in /tmp, when the chunk is complete it is shipped out to the Datanodes. This can cause /tmp to fill up fast when a lot of files are being written. A potentially better scheme is to buffer the written data in RAM (application code can set the buffer size) and flush it to the Datanodes when the buffer fills up.

      1. tmp-delete.patch
        0.6 kB
        Owen O'Malley
      2. no-tmp.patch
        8 kB
        Doug Cutting

        Activity

          People

          • Assignee:
            Doug Cutting
            Reporter:
            Sameer Paranjpye
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development