Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3808

fuse_dfs: postpone libhdfs intialization until after fork

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 2.0.2-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: fuse-dfs
    • Labels:
      None

      Description

      libhdfs may create threads or initialize internal JNI data structures when it is used. So we should be careful to daemonize() before initializing libhdfs, not after.

      Unfortunately, fuse_dfs is doing just this – initializing libhdfs in its main function, prior to calling fuse_main which does a daemonize.

      daemonize() does not preserve threads, because it is implemented by fork()}}ing and then continuing execution in the child and allowing the parent to {{exit().

      1. HDFS-3808.003.patch
        5 kB
        Colin Patrick McCabe
      2. HDFS-3808.002.patch
        5 kB
        Colin Patrick McCabe
      3. HDFS-3808.001.patch
        4 kB
        Colin Patrick McCabe

        Activity

        Colin Patrick McCabe created issue -
        Colin Patrick McCabe made changes -
        Field Original Value New Value
        Attachment HDFS-3808.001.patch [ 12541148 ]
        Colin Patrick McCabe made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Colin Patrick McCabe made changes -
        Priority Major [ 3 ] Critical [ 2 ]
        Andy Isaacson made changes -
        Description {{libhdfs}} may create threads or initialize internal JNI data structures when it is used. So we should not call fork() after the library is first used.

        Unfortunately, fuse_dfs is doing just this-- initializing libhdfs in its {{main}} function, prior to calling {{fuse_main}} which does a {{daemonize}} (which involves forking.) We should correct this.
        {{libhdfs}} may create threads or initialize internal JNI data structures when it is used. So we should be careful to {{daemonize()}} before initializing {{libhdfs}}, not after.

        Unfortunately, fuse_dfs is doing just this -- initializing libhdfs in its {{main}} function, prior to calling {{fuse_main}} which does a {{daemonize}}.

        {{daemonize()}} does not preserve threads, because it is implemented by {{fork()}}ing and then continuing execution in the child and allowing the parent to {{exit()}}.
        Colin Patrick McCabe made changes -
        Attachment HDFS-3808.002.patch [ 12541274 ]
        Colin Patrick McCabe made changes -
        Attachment HDFS-3808.002.patch [ 12541274 ]
        Colin Patrick McCabe made changes -
        Attachment HDFS-3808.002.patch [ 12541276 ]
        Colin Patrick McCabe made changes -
        Attachment HDFS-3808.003.patch [ 12541283 ]
        Aaron T. Myers made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 2.2.0-alpha [ 12322472 ]
        Resolution Fixed [ 1 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Colin Patrick McCabe
            Reporter:
            Colin Patrick McCabe
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development