Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
2.0.0-alpha, 2.6.0
-
None
-
Confirmed with Cloudera hadoop-hdfs-fuse-2.6.0+cdh5.5.0+921-1.cdh5.5.0.p0.15.el6.x86_64 on CentOS 6
-
Patch
Description
When a user does an 'ls' on a HDFS FUSE mount, dfs_getattr() is called and fuse_dfs attempts to resolve the user's uid into a username string with getUsername(). If this lookup is unsuccessful, getUsername() returns NULL leading to a segfault in hdfsConnCompare().
Sites storing NSS info in a remote database (such as LDAP) will occasionally have NSS failures if there are connectivity or daemon issues. Running processes accessing the HDFS mount during this time may cause the fuse_dfs process to crash, disabling the mount.
To reproduce the issue:
1) Add a new local user
2) su to the new user
3) As root, edit /etc/passwd, changing the new user's uid number
4) As the new user, do an ls on an HDFS FUSE mount. This should cause a segfault.
Backtrace from fuse_dfs segfault (hadoop-hdfs-fuse-2.0.0+545-1.cdh4.1.1.p0.21.osg33.el6.x86_64)
#0 0x0000003f43c32625 in raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x0000003f43c33e05 in abort () at abort.c:92 #2 0x0000003f46beb785 in os::abort (dump_core=true) at /usr/src/debug/java-1.7.0-openjdk/openjdk/hotspot/src/os/linux/vm/os_linux.cpp:1640 #3 0x0000003f46d5f03f in VMError::report_and_die (this=0x7ffa3cdf86f0) at /usr/src/debug/java-1.7.0-openjdk/openjdk/hotspot/src/share/vm/utilities/vmError.cpp:1075 #4 0x0000003f46d5f70b in crash_handler (sig=11, info=0x7ffa3cdf88b0, ucVoid=0x7ffa3cdf8780) at /usr/src/debug/java-1.7.0-openjdk/openjdk/hotspot/src/os/linux/vm/vmError_linux.cpp:106 #5 <signal handler called> #6 os::is_first_C_frame (fr=<value optimized out>) at /usr/src/debug/java-1.7.0-openjdk/openjdk/hotspot/src/share/vm/runtime/os.cpp:1025 #7 0x0000003f46d5e071 in VMError::report (this=0x7ffa3cdf9560, st=0x7ffa3cdf93e0) at /usr/src/debug/java-1.7.0-openjdk/openjdk/hotspot/src/share/vm/utilities/vmError.cpp:617 #8 0x0000003f46d5ebad in VMError::report_and_die (this=0x7ffa3cdf9560) at /usr/src/debug/java-1.7.0-openjdk/openjdk/hotspot/src/share/vm/utilities/vmError.cpp:1009 #9 0x0000003f46bf0322 in JVM_handle_linux_signal (sig=11, info=0x7ffa3cdf9730, ucVoid=0x7ffa3cdf9600, abort_if_unrecognized=1021285600) at /usr/src/debug/java-1.7.0-openjdk/openjdk/hotspot/src/os_cpu/linux_x86/vm/os_linux_x86.cpp:531 #10 <signal handler called> #11 __strcmp_sse42 () at ../sysdeps/x86_64/multiarch/strcmp.S:259 #12 0x0000000000403d3d in hdfsConnCompare (head=<value optimized out>, elm=<value optimized out>) at /usr/src/debug/hadoop-2.0.0-cdh4.1.1/src/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/fuse_connect.c:204 #13 hdfsConnTree_RB_FIND (head=<value optimized out>, elm=<value optimized out>) at /usr/src/debug/hadoop-2.0.0-cdh4.1.1/src/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/fuse_connect.c:81 #14 0x0000000000404245 in hdfsConnFind (usrname=0x0, ctx=0x7ff95013b800, out=0x7ffa3cdf9c60) at /usr/src/debug/hadoop-2.0.0-cdh4.1.1/src/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/fuse_connect.c:220 #15 fuseConnect (usrname=0x0, ctx=0x7ff95013b800, out=0x7ffa3cdf9c60) at /usr/src/debug/hadoop-2.0.0-cdh4.1.1/src/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/fuse_connect.c:517 #16 0x0000000000404337 in fuseConnectAsThreadUid (conn=0x7ffa3cdf9c60) at /usr/src/debug/hadoop-2.0.0-cdh4.1.1/src/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/fuse_connect.c:544 #17 0x0000000000404c55 in dfs_getattr (path=0x7ff950150de0 "/user/users01", st=0x7ffa3cdf9d20) at /usr/src/debug/hadoop-2.0.0-cdh4.1.1/src/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/fuse_impls_getattr.c:37 #18 0x0000003f47c0b353 in lookup_path (f=0x15e39f0, nodeid=22546, name=0x7ff9602d0058 "users01", path=<value optimized out>, e=0x7ffa3cdf9d10, fi=<value optimized out>) at fuse.c:1824 #19 0x0000003f47c0d865 in fuse_lib_lookup (req=0x7ff950003fe0, parent=22546, name=0x7ff9602d0058 "users01") at fuse.c:2017 #20 0x0000003f47c120ef in fuse_do_work (data=0x7ff9600e3f30) at fuse_loop_mt.c:107 #21 0x0000003f44407aa1 in start_thread (arg=0x7ffa3cdfa700) at pthread_create.c:301 #22 0x0000003f43ce893d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115