Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-15944

Prevent truncation by snprintf

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      There are some areas of code in libhdfs and fuse-dfs components where the destination buffer is smaller than the source that's trying to write. This would cause truncation. Thus we need to ensure that the source that's being written doesn't exceed the destination buffer size.

      The following warnings are reported for this issue -

      /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_threaded.c: In function ‘doTestHdfsOperations.isra.0’:
      /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_threaded.c:226:39: warning: ‘/many_files_’ directive output may be truncated writing 12 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
        226 |       snprintf(filename, PATH_MAX, "%s/many_files_%d", listDirTest, nFile);
            |                                       ^~~~~~~~~~~~
      /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_threaded.c:226:36: note: directive argument in the range [0, 9999]
        226 |       snprintf(filename, PATH_MAX, "%s/many_files_%d", listDirTest, nFile);
            |                                    ^~~~~~~~~~~~~~~~~~
      In file included from /usr/include/stdio.h:867,
                       from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
                       from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_threaded.c:19:
      /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 14 and 4112 bytes into a destination of size 4096
         67 |   return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
            |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         68 |        __bos (__s), __fmt, __va_arg_pack ());
            |        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      
      /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:255:33: warning: ‘/a’ directive output may be truncated writing 2 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
        255 |   snprintf(tmp, sizeof(tmp), "%s/a", base);
            |                                 ^~
      In file included from /usr/include/stdio.h:867,
                       from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
                       from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
      /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 3 and 4098 bytes into a destination of size 4096
         67 |   return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
            |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         68 |        __bos (__s), __fmt, __va_arg_pack ());
            |        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:263:33: warning: ‘/b’ directive output may be truncated writing 2 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
        263 |   snprintf(tmp, sizeof(tmp), "%s/b", base);
            |                                 ^~
      In file included from /usr/include/stdio.h:867,
                       from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
                       from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
      /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 3 and 4098 bytes into a destination of size 4096
         67 |   return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
            |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         68 |        __bos (__s), __fmt, __va_arg_pack ());
            |        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:271:33: warning: ‘/a’ directive output may be truncated writing 2 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
        271 |   snprintf(src, sizeof(src), "%s/a", base);
            |                                 ^~
      In file included from /usr/include/stdio.h:867,
                       from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
                       from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
      /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 3 and 4098 bytes into a destination of size 4096
         67 |   return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
            |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         68 |        __bos (__s), __fmt, __va_arg_pack ());
            |        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:272:33: warning: ‘/c’ directive output may be truncated writing 2 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
        272 |   snprintf(dst, sizeof(dst), "%s/c", base);
            |                                 ^~
      In file included from /usr/include/stdio.h:867,
                       from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
                       from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
      /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 3 and 4098 bytes into a destination of size 4096
         67 |   return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
            |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         68 |        __bos (__s), __fmt, __va_arg_pack ());
            |        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:297:35: warning: ‘/b/’ directive output may be truncated writing 3 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
        297 |     snprintf(tmp, sizeof(tmp), "%s/b/%d", base, i);
            |                                   ^~~
      /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:297:32: note: directive argument in the range [0, 2]
        297 |     snprintf(tmp, sizeof(tmp), "%s/b/%d", base, i);
            |                                ^~~~~~~~~
      In file included from /usr/include/stdio.h:867,
                       from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
                       from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
      /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 5 and 4100 bytes into a destination of size 4096
         67 |   return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
            |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         68 |        __bos (__s), __fmt, __va_arg_pack ());
            |        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:187:35: warning: ‘/trunc.txt’ directive output may be truncated writing 10 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
        187 |   snprintf(path, sizeof(path), "%s/trunc.txt", base);
            |                                   ^~~~~~~~~~
      In file included from /usr/include/stdio.h:867,
                       from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
                       from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
      /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 11 and 4106 bytes into a destination of size 4096
         67 |   return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
            |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         68 |        __bos (__s), __fmt, __va_arg_pack ());
            |        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      

      Attachments

        Issue Links

          Activity

            People

              gaurava Gautham Banasandra
              gaurava Gautham Banasandra
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m