Hadoop Common
  1. Hadoop Common
  2. HADOOP-66

dfs client writes all data for a chunk to /tmp

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.1.0
    • Fix Version/s: 0.1.0
    • Component/s: None
    • Labels:
      None

      Description

      The dfs client writes all the data for the current chunk to a file in /tmp, when the chunk is complete it is shipped out to the Datanodes. This can cause /tmp to fill up fast when a lot of files are being written. A potentially better scheme is to buffer the written data in RAM (application code can set the buffer size) and flush it to the Datanodes when the buffer fills up.

      1. no-tmp.patch
        8 kB
        Doug Cutting
      2. tmp-delete.patch
        0.6 kB
        Owen O'Malley

        Activity

        Owen O'Malley made changes -
        Component/s dfs [ 12310710 ]
        Doug Cutting made changes -
        Workflow no-reopen-closed [ 12373361 ] no-reopen-closed, patch-avail [ 12377669 ]
        Doug Cutting made changes -
        Workflow no reopen closed [ 12373025 ] no-reopen-closed [ 12373361 ]
        Doug Cutting made changes -
        Workflow jira [ 12351616 ] no reopen closed [ 12373025 ]
        Doug Cutting made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Owen O'Malley made changes -
        Attachment tmp-delete.patch [ 12324144 ]
        Doug Cutting made changes -
        Resolution Fixed [ 1 ]
        Status Open [ 1 ] Resolved [ 5 ]
        Assignee Doug Cutting [ cutting ]
        Doug Cutting made changes -
        Field Original Value New Value
        Attachment no-tmp.patch [ 12323938 ]
        Sameer Paranjpye created issue -

          People

          • Assignee:
            Doug Cutting
            Reporter:
            Sameer Paranjpye
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development