Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9182

the buffer used in hdfsRead seems leaks when the thread exits

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: filecache
    • Labels:
      None
    • Environment:

      Linux RHEP x64

      Description

      I use multi-threads in my c++ program to download 3000 files in HDFS use libhdfs. Every thread is created by "pthread_create" to download a file and exit. We monitor the memory status when the program is running. It seems every thread will create a buffer which size is specified by the buffersize parameter in "hdfsOpenFile" function. But when the thread finish the task and exit, it doesn't free the buffer. So our program will eventually abort by Java's "out of memory" exception. I just don't know how to free the buffer or I use these functions in wrong way. Thanks!

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              dycforever dingyichuan

              Dates

              • Created:
                Updated:

                Issue deployment