Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1125

SerialUtils.cc: deserializeFloat is out of sync with SerialUtils.hh

    Details

    • Type: Bug Bug
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.21.0
    • Fix Version/s: None
    • Component/s: pipes
    • Labels:
    • Tags:
      utils

      Description

      *** SerialUtils.hh ***
        float deserializeFloat(InStream& stream);
      
      *** SerialUtils.cc ***
        void deserializeFloat(float& t, InStream& stream)
        {
          char buf[sizeof(float)];
          stream.read(buf, sizeof(float));
          XDR xdrs;
          xdrmem_create(&xdrs, buf, sizeof(float), XDR_DECODE);
          xdr_float(&xdrs, &t);
        }
      
      1. MAPREDUCE-1125.004.patch
        1 kB
        Masatake Iwasaki
      2. MAPREDUCE-1125-3.patch
        4 kB
        Simone Leo
      3. MAPREDUCE-1125-2.patch
        4 kB
        Simone Leo

        Activity

        Hide
        Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 5m 31s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 47s There were no new javac warning messages.
        +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 36s mvn install still works.
        +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse.
            15m 48s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12741423/MAPREDUCE-1125.004.patch
        Optional Tests javac unit
        git revision trunk / 49dfad9
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5833/testReport/
        Java 1.7.0_55
        uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5833/console

        This message was automatically generated.

        Show
        Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 5m 31s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 47s There were no new javac warning messages. +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse.     15m 48s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12741423/MAPREDUCE-1125.004.patch Optional Tests javac unit git revision trunk / 49dfad9 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5833/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5833/console This message was automatically generated.
        Hide
        Masatake Iwasaki added a comment -

        I attached rebased patch. I left the version of old signature for backward compatibility. There is no test code for .cc in the current tree of hadoop-pipes

        Show
        Masatake Iwasaki added a comment - I attached rebased patch. I left the version of old signature for backward compatibility. There is no test code for .cc in the current tree of hadoop-pipes
        Hide
        Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 patch 0m 0s The patch command could not apply the patch during dryrun.



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12494006/MAPREDUCE-1125-3.patch
        Optional Tests javac unit
        git revision trunk / 6471d18
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5692/console

        This message was automatically generated.

        Show
        Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12494006/MAPREDUCE-1125-3.patch Optional Tests javac unit git revision trunk / 6471d18 Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5692/console This message was automatically generated.
        Hide
        Brahma Reddy Battula added a comment -

        Simone Leo can you please rebase the patch with latest trunk..?

        Show
        Brahma Reddy Battula added a comment - Simone Leo can you please rebase the patch with latest trunk..?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12494006/MAPREDUCE-1125-3.patch
        against trunk revision d6e05c5.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5249//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12494006/MAPREDUCE-1125-3.patch against trunk revision d6e05c5. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5249//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12494006/MAPREDUCE-1125-3.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 5 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12494006/MAPREDUCE-1125-3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//console This message is automatically generated.
        Hide
        Simone Leo added a comment -

        rebased for current trunk

        Show
        Simone Leo added a comment - rebased for current trunk
        Hide
        Arun C Murthy added a comment -

        Sorry to come in late, the patch has gone stale. Can you please rebase? Thanks.

        Show
        Arun C Murthy added a comment - Sorry to come in late, the patch has gone stale. Can you please rebase? Thanks.
        Hide
        Simone Leo added a comment -

        In Pydoop we are currently patching SerialUtils.cc at compile time, before building the Python extension with Boost.

        Show
        Simone Leo added a comment - In Pydoop we are currently patching SerialUtils.cc at compile time, before building the Python extension with Boost.
        Hide
        Todd Lipcon added a comment -

        This is a sort of incompatible change - some user might be explicitly declaring the extern void deserializeFloat(float &t, InStream &stream), right?

        Do we need to leave the old version around for a version? Perhaps someone who makes use of SerialUtils can comment - I never have worked on significant amounts of C++ code with Hadoop

        Show
        Todd Lipcon added a comment - This is a sort of incompatible change - some user might be explicitly declaring the extern void deserializeFloat(float &t, InStream &stream), right? Do we need to leave the old version around for a version? Perhaps someone who makes use of SerialUtils can comment - I never have worked on significant amounts of C++ code with Hadoop
        Hide
        Simone Leo added a comment -

        Looks like some tests failed due to timeout. Those tests have nothing to do with this patch, anyway.

        Show
        Simone Leo added a comment - Looks like some tests failed due to timeout. Those tests have nothing to do with this patch, anyway.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12472943/MAPREDUCE-1125-2.patch
        against trunk revision 1079072.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 5 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/130//testReport/
        Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/130//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/130//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12472943/MAPREDUCE-1125-2.patch against trunk revision 1079072. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/130//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/130//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/130//console This message is automatically generated.
        Hide
        Simone Leo added a comment -

        Attached version 2 of the patch – put in sync with current trunk

        Show
        Simone Leo added a comment - Attached version 2 of the patch – put in sync with current trunk
        Hide
        Simone Leo added a comment -

        Updated for current trunk

        Show
        Simone Leo added a comment - Updated for current trunk
        Hide
        Simone Leo added a comment -

        Updated for current trunk

        Show
        Simone Leo added a comment - Updated for current trunk
        Hide
        Tom White added a comment -

        This has fallen out of date. Can you sync with trunk then hopefully a committer familiar with pipes will review and commit. Thanks.

        Show
        Tom White added a comment - This has fallen out of date. Can you sync with trunk then hopefully a committer familiar with pipes will review and commit. Thanks.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12456393/MAPREDUCE-1125.patch
        against trunk revision 1075216.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 7 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/92//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12456393/MAPREDUCE-1125.patch against trunk revision 1075216. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/92//console This message is automatically generated.
        Hide
        Simone Leo added a comment -

        As suggested on the mapreduce-dev mailing list, I'm attaching results from my local test-patch run:

        [exec] +1 overall.
        [exec]
        [exec] +1 @author. The patch does not contain any @author tags.
        [exec]
        [exec] +1 tests included. The patch appears to include 7 new or modified tests.
        [exec]
        [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
        [exec]
        [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
        [exec]
        [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
        [exec]
        [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
        [exec]
        [exec] +1 system tests framework. The patch passed system tests framework compile.

        Show
        Simone Leo added a comment - As suggested on the mapreduce-dev mailing list, I'm attaching results from my local test-patch run: [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 7 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system tests framework. The patch passed system tests framework compile.
        Hide
        Simone Leo added a comment -

        Attaching patch for current trunk

        Show
        Simone Leo added a comment - Attaching patch for current trunk
        Hide
        Simone Leo added a comment -

        Patch for current trunk@1004113

        Show
        Simone Leo added a comment - Patch for current trunk@1004113
        Hide
        Greg Roelofs added a comment -

        Is this still an issue? I'm not familiar with the C++ code in question--where does it live?

        The patch looks fine as far as it goes (other than the lack of path info in the filenames), but I'm kind of amazed it ever compiled in the first place. Is SerialUtils.hh not actually included in SerialUtils.cc? Does Hudson even look at this stuff, wherever it lives?

        Show
        Greg Roelofs added a comment - Is this still an issue? I'm not familiar with the C++ code in question--where does it live? The patch looks fine as far as it goes (other than the lack of path info in the filenames), but I'm kind of amazed it ever compiled in the first place. Is SerialUtils.hh not actually included in SerialUtils.cc? Does Hudson even look at this stuff, wherever it lives?
        Hide
        Simone Leo added a comment -

        Attaching proposed patch

        Show
        Simone Leo added a comment - Attaching proposed patch

          People

          • Assignee:
            Simone Leo
            Reporter:
            Simone Leo
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:

              Development