Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3914

Mismatched free() / delete / delete [] in HadoopPipes

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.205.0, 0.23.0, 1.0.0
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: pipes
    • Labels:
      None
    • Environment:

      Based upon map reduce pipes task executed on Ubuntu 11.10

      Description

      When running valgrind on a simple MapReduce pipes job, valgrind identifies a mismatched new / delete:

      ==20394== Mismatched free() / delete / delete []
      ==20394== at 0x4C27FF2: operator delete(void*) (vg_replace_malloc.c:387)
      ==20394== by 0x4328A5: HadoopPipes::runTask(HadoopPipes::Factory const&) (HadoopPipes.cc:1171)
      ==20394== by 0x424C33: main (ProcessRow.cpp:118)
      ==20394== Address 0x9c5b540 is 0 bytes inside a block of size 131,072 alloc'd
      ==20394== at 0x4C2864B: operator new[](unsigned long) (vg_replace_malloc.c:305)
      ==20394== by 0x431E5D: HadoopPipes::runTask(HadoopPipes::Factory const&) (HadoopPipes.cc:1121)
      ==20394== by 0x424C33: main (ProcessRow.cpp:118)
      ==20394==
      ==20394== Mismatched free() / delete / delete []
      ==20394== at 0x4C27FF2: operator delete(void*) (vg_replace_malloc.c:387)
      ==20394== by 0x4328AF: HadoopPipes::runTask(HadoopPipes::Factory const&) (HadoopPipes.cc:1172)
      ==20394== by 0x424C33: main (ProcessRow.cpp:118)
      ==20394== Address 0x9c7b580 is 0 bytes inside a block of size 131,072 alloc'd
      ==20394== at 0x4C2864B: operator new[](unsigned long) (vg_replace_malloc.c:305)
      ==20394== by 0x431E6A: HadoopPipes::runTask(HadoopPipes::Factory const&) (HadoopPipes.cc:1122)
      ==20394== by 0x424C33: main (ProcessRow.cpp:118)

      The new [] calls in Lines 1121 and 1122 of HadoopPipes.cc:
      bufin = new char[bufsize];
      bufout = new char[bufsize];
      should have matching delete [] calls but are instead bracketed my delete on lines 1171 and 1172:
      delete bufin;
      delete bufout;
      So these should be replaced by delete[]

      1. MAPREDUCE-3914.patch
        0.6 kB
        Joe Mudd
      2. MAPREDUCE-3914-branch-0.23.patch
        0.6 kB
        Charles Earl
      3. MAPREDUCE-3914-branch-1.0.patch
        0.5 kB
        Charles Earl

        Activity

        Hide
        charlescearl Charles Earl added a comment -

        Patches for Hadoop 1.0 and 0.23

        Show
        charlescearl Charles Earl added a comment - Patches for Hadoop 1.0 and 0.23
        Hide
        JoeM Joe Mudd added a comment -

        Trunk version of the patch that passes dev-support/test-patch and is ready for code review.

        Show
        JoeM Joe Mudd added a comment - Trunk version of the patch that passes dev-support/test-patch and is ready for code review.
        Hide
        JoeM Joe Mudd added a comment -

        Update patch to be relative to the root

        Show
        JoeM Joe Mudd added a comment - Update patch to be relative to the root
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12652401/MAPREDUCE-3914.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4688//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4688//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12652401/MAPREDUCE-3914.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in . +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4688//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4688//console This message is automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12652401/MAPREDUCE-3914.patch
        against trunk revision 7711049.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5276//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5276//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12652401/MAPREDUCE-3914.patch against trunk revision 7711049. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5276//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5276//console This message is automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 5s docker + precommit patch detected.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 3m 8s trunk passed
        +1 compile 0m 17s trunk passed with JDK v1.8.0_60
        +1 compile 0m 17s trunk passed with JDK v1.7.0_79
        +1 mvneclipse 0m 10s trunk passed
        +1 mvninstall 0m 10s the patch passed
        +1 compile 0m 17s the patch passed with JDK v1.8.0_60
        +1 cc 0m 17s the patch passed
        +1 javac 0m 17s the patch passed
        +1 compile 0m 17s the patch passed with JDK v1.7.0_79
        +1 cc 0m 17s the patch passed
        +1 javac 0m 17s the patch passed
        +1 mvneclipse 0m 9s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 unit 0m 16s hadoop-pipes in the patch passed with JDK v1.8.0_60.
        +1 unit 0m 17s hadoop-pipes in the patch passed with JDK v1.7.0_79.
        +1 asflicense 0m 22s Patch does not generate ASF License warnings.
        5m 56s



        Subsystem Report/Notes
        Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-06
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12652401/MAPREDUCE-3914.patch
        JIRA Issue MAPREDUCE-3914
        Optional Tests asflicense cc unit javac compile
        uname Linux 120d39c7f1be 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh
        git revision trunk / 286cc64
        Default Java 1.7.0_79
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
        JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6119/testReport/
        modules C: hadoop-tools/hadoop-pipes U: hadoop-tools/hadoop-pipes
        Max memory used 226MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6119/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 5s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 3m 8s trunk passed +1 compile 0m 17s trunk passed with JDK v1.8.0_60 +1 compile 0m 17s trunk passed with JDK v1.7.0_79 +1 mvneclipse 0m 10s trunk passed +1 mvninstall 0m 10s the patch passed +1 compile 0m 17s the patch passed with JDK v1.8.0_60 +1 cc 0m 17s the patch passed +1 javac 0m 17s the patch passed +1 compile 0m 17s the patch passed with JDK v1.7.0_79 +1 cc 0m 17s the patch passed +1 javac 0m 17s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 unit 0m 16s hadoop-pipes in the patch passed with JDK v1.8.0_60. +1 unit 0m 17s hadoop-pipes in the patch passed with JDK v1.7.0_79. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 5m 56s Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-06 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12652401/MAPREDUCE-3914.patch JIRA Issue MAPREDUCE-3914 Optional Tests asflicense cc unit javac compile uname Linux 120d39c7f1be 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh git revision trunk / 286cc64 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6119/testReport/ modules C: hadoop-tools/hadoop-pipes U: hadoop-tools/hadoop-pipes Max memory used 226MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6119/console This message was automatically generated.
        Hide
        aw Allen Wittenauer added a comment -

        +1 committed.

        Thanks!

        Show
        aw Allen Wittenauer added a comment - +1 committed. Thanks!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8764 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8764/)
        MAPREDUCE-3914. Mismatched free() / delete / delete [] in HadoopPipes (aw: rev fc7cd46faf5cafc73d4316a21d59cb2b83a6b48f)

        • hadoop-tools/hadoop-pipes/src/main/native/pipes/impl/HadoopPipes.cc
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8764 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8764/ ) MAPREDUCE-3914 . Mismatched free() / delete / delete [] in HadoopPipes (aw: rev fc7cd46faf5cafc73d4316a21d59cb2b83a6b48f) hadoop-tools/hadoop-pipes/src/main/native/pipes/impl/HadoopPipes.cc hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #634 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/634/)
        MAPREDUCE-3914. Mismatched free() / delete / delete [] in HadoopPipes (aw: rev fc7cd46faf5cafc73d4316a21d59cb2b83a6b48f)

        • hadoop-tools/hadoop-pipes/src/main/native/pipes/impl/HadoopPipes.cc
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #634 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/634/ ) MAPREDUCE-3914 . Mismatched free() / delete / delete [] in HadoopPipes (aw: rev fc7cd46faf5cafc73d4316a21d59cb2b83a6b48f) hadoop-tools/hadoop-pipes/src/main/native/pipes/impl/HadoopPipes.cc hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #644 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/644/)
        MAPREDUCE-3914. Mismatched free() / delete / delete [] in HadoopPipes (aw: rev fc7cd46faf5cafc73d4316a21d59cb2b83a6b48f)

        • hadoop-tools/hadoop-pipes/src/main/native/pipes/impl/HadoopPipes.cc
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #644 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/644/ ) MAPREDUCE-3914 . Mismatched free() / delete / delete [] in HadoopPipes (aw: rev fc7cd46faf5cafc73d4316a21d59cb2b83a6b48f) hadoop-tools/hadoop-pipes/src/main/native/pipes/impl/HadoopPipes.cc hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #1368 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1368/)
        MAPREDUCE-3914. Mismatched free() / delete / delete [] in HadoopPipes (aw: rev fc7cd46faf5cafc73d4316a21d59cb2b83a6b48f)

        • hadoop-tools/hadoop-pipes/src/main/native/pipes/impl/HadoopPipes.cc
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #1368 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1368/ ) MAPREDUCE-3914 . Mismatched free() / delete / delete [] in HadoopPipes (aw: rev fc7cd46faf5cafc73d4316a21d59cb2b83a6b48f) hadoop-tools/hadoop-pipes/src/main/native/pipes/impl/HadoopPipes.cc hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2574 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2574/)
        MAPREDUCE-3914. Mismatched free() / delete / delete [] in HadoopPipes (aw: rev fc7cd46faf5cafc73d4316a21d59cb2b83a6b48f)

        • hadoop-tools/hadoop-pipes/src/main/native/pipes/impl/HadoopPipes.cc
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2574 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2574/ ) MAPREDUCE-3914 . Mismatched free() / delete / delete [] in HadoopPipes (aw: rev fc7cd46faf5cafc73d4316a21d59cb2b83a6b48f) hadoop-tools/hadoop-pipes/src/main/native/pipes/impl/HadoopPipes.cc hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #576 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/576/)
        MAPREDUCE-3914. Mismatched free() / delete / delete [] in HadoopPipes (aw: rev fc7cd46faf5cafc73d4316a21d59cb2b83a6b48f)

        • hadoop-tools/hadoop-pipes/src/main/native/pipes/impl/HadoopPipes.cc
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #576 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/576/ ) MAPREDUCE-3914 . Mismatched free() / delete / delete [] in HadoopPipes (aw: rev fc7cd46faf5cafc73d4316a21d59cb2b83a6b48f) hadoop-tools/hadoop-pipes/src/main/native/pipes/impl/HadoopPipes.cc hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2515 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2515/)
        MAPREDUCE-3914. Mismatched free() / delete / delete [] in HadoopPipes (aw: rev fc7cd46faf5cafc73d4316a21d59cb2b83a6b48f)

        • hadoop-tools/hadoop-pipes/src/main/native/pipes/impl/HadoopPipes.cc
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2515 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2515/ ) MAPREDUCE-3914 . Mismatched free() / delete / delete [] in HadoopPipes (aw: rev fc7cd46faf5cafc73d4316a21d59cb2b83a6b48f) hadoop-tools/hadoop-pipes/src/main/native/pipes/impl/HadoopPipes.cc hadoop-mapreduce-project/CHANGES.txt

          People

          • Assignee:
            JoeM Joe Mudd
            Reporter:
            charlescearl Charles Earl
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 1h
              1h
              Remaining:
              Remaining Estimate - 1h
              1h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development