[HDFS-15944] Prevent truncation by snprintf

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.4.0
• Component/s: fuse-dfs, libhdfs
• Labels: pull-request-available
• Hadoop Flags: Reviewed
• Assignee: Gautham Banasandra
• Reporter: Gautham Banasandra
• Due: 03/Apr/21
• Created: 02/Apr/21 07:43
• Updated: 02/Apr/21 16:38
• Resolved: 02/Apr/21 16:38

Description

There are some areas of code in libhdfs and fuse-dfs components where the destination buffer is smaller than the source that's trying to write. This would cause truncation. Thus we need to ensure that the source that's being written doesn't exceed the destination buffer size.

The following warnings are reported for this issue -

/mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_threaded.c: In function ‘doTestHdfsOperations.isra.0’:
/mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_threaded.c:226:39: warning: ‘/many_files_’ directive output may be truncated writing 12 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
  226 |       snprintf(filename, PATH_MAX, "%s/many_files_%d", listDirTest, nFile);
      |                                       ^~~~~~~~~~~~
/mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_threaded.c:226:36: note: directive argument in the range [0, 9999]
  226 |       snprintf(filename, PATH_MAX, "%s/many_files_%d", listDirTest, nFile);
      |                                    ^~~~~~~~~~~~~~~~~~
In file included from /usr/include/stdio.h:867,
                 from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
                 from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_threaded.c:19:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 14 and 4112 bytes into a destination of size 4096
   67 |   return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   68 |        __bos (__s), __fmt, __va_arg_pack ());
      |        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

/mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:255:33: warning: ‘/a’ directive output may be truncated writing 2 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
  255 |   snprintf(tmp, sizeof(tmp), "%s/a", base);
      |                                 ^~
In file included from /usr/include/stdio.h:867,
                 from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
                 from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 3 and 4098 bytes into a destination of size 4096
   67 |   return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   68 |        __bos (__s), __fmt, __va_arg_pack ());
      |        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:263:33: warning: ‘/b’ directive output may be truncated writing 2 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
  263 |   snprintf(tmp, sizeof(tmp), "%s/b", base);
      |                                 ^~
In file included from /usr/include/stdio.h:867,
                 from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
                 from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 3 and 4098 bytes into a destination of size 4096
   67 |   return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   68 |        __bos (__s), __fmt, __va_arg_pack ());
      |        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:271:33: warning: ‘/a’ directive output may be truncated writing 2 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
  271 |   snprintf(src, sizeof(src), "%s/a", base);
      |                                 ^~
In file included from /usr/include/stdio.h:867,
                 from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
                 from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 3 and 4098 bytes into a destination of size 4096
   67 |   return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   68 |        __bos (__s), __fmt, __va_arg_pack ());
      |        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:272:33: warning: ‘/c’ directive output may be truncated writing 2 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
  272 |   snprintf(dst, sizeof(dst), "%s/c", base);
      |                                 ^~
In file included from /usr/include/stdio.h:867,
                 from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
                 from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 3 and 4098 bytes into a destination of size 4096
   67 |   return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   68 |        __bos (__s), __fmt, __va_arg_pack ());
      |        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:297:35: warning: ‘/b/’ directive output may be truncated writing 3 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
  297 |     snprintf(tmp, sizeof(tmp), "%s/b/%d", base, i);
      |                                   ^~~
/mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:297:32: note: directive argument in the range [0, 2]
  297 |     snprintf(tmp, sizeof(tmp), "%s/b/%d", base, i);
      |                                ^~~~~~~~~
In file included from /usr/include/stdio.h:867,
                 from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
                 from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 5 and 4100 bytes into a destination of size 4096
   67 |   return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   68 |        __bos (__s), __fmt, __va_arg_pack ());
      |        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:187:35: warning: ‘/trunc.txt’ directive output may be truncated writing 10 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
  187 |   snprintf(path, sizeof(path), "%s/trunc.txt", base);
      |                                   ^~~~~~~~~~
In file included from /usr/include/stdio.h:867,
                 from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
                 from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 11 and 4106 bytes into a destination of size 4096
   67 |   return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   68 |        __bos (__s), __fmt, __va_arg_pack ());
      |        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Issue Links

• links to: GitHub Pull Request #2853