Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1125

SerialUtils.cc: deserializeFloat is out of sync with SerialUtils.hh

    Details

    • Type: Bug Bug
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.24.0
    • Component/s: pipes
    • Labels:
      None
    • Tags:
      utils

      Description

      *** SerialUtils.hh ***
        float deserializeFloat(InStream& stream);
      
      *** SerialUtils.cc ***
        void deserializeFloat(float& t, InStream& stream)
        {
          char buf[sizeof(float)];
          stream.read(buf, sizeof(float));
          XDR xdrs;
          xdrmem_create(&xdrs, buf, sizeof(float), XDR_DECODE);
          xdr_float(&xdrs, &t);
        }
      
      1. MAPREDUCE-1125-2.patch
        4 kB
        Simone Leo
      2. MAPREDUCE-1125-3.patch
        4 kB
        Simone Leo

        Activity

        Hide
        Simone Leo added a comment -

        Attaching proposed patch

        Show
        Simone Leo added a comment - Attaching proposed patch
        Hide
        Greg Roelofs added a comment -

        Is this still an issue? I'm not familiar with the C++ code in question--where does it live?

        The patch looks fine as far as it goes (other than the lack of path info in the filenames), but I'm kind of amazed it ever compiled in the first place. Is SerialUtils.hh not actually included in SerialUtils.cc? Does Hudson even look at this stuff, wherever it lives?

        Show
        Greg Roelofs added a comment - Is this still an issue? I'm not familiar with the C++ code in question--where does it live? The patch looks fine as far as it goes (other than the lack of path info in the filenames), but I'm kind of amazed it ever compiled in the first place. Is SerialUtils.hh not actually included in SerialUtils.cc? Does Hudson even look at this stuff, wherever it lives?
        Hide
        Simone Leo added a comment -

        Patch for current trunk@1004113

        Show
        Simone Leo added a comment - Patch for current trunk@1004113
        Hide
        Simone Leo added a comment -

        Attaching patch for current trunk

        Show
        Simone Leo added a comment - Attaching patch for current trunk
        Hide
        Simone Leo added a comment -

        As suggested on the mapreduce-dev mailing list, I'm attaching results from my local test-patch run:

        [exec] +1 overall.
        [exec]
        [exec] +1 @author. The patch does not contain any @author tags.
        [exec]
        [exec] +1 tests included. The patch appears to include 7 new or modified tests.
        [exec]
        [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
        [exec]
        [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
        [exec]
        [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
        [exec]
        [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
        [exec]
        [exec] +1 system tests framework. The patch passed system tests framework compile.

        Show
        Simone Leo added a comment - As suggested on the mapreduce-dev mailing list, I'm attaching results from my local test-patch run: [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 7 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system tests framework. The patch passed system tests framework compile.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12456393/MAPREDUCE-1125.patch
        against trunk revision 1075216.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 7 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/92//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12456393/MAPREDUCE-1125.patch against trunk revision 1075216. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/92//console This message is automatically generated.
        Hide
        Tom White added a comment -

        This has fallen out of date. Can you sync with trunk then hopefully a committer familiar with pipes will review and commit. Thanks.

        Show
        Tom White added a comment - This has fallen out of date. Can you sync with trunk then hopefully a committer familiar with pipes will review and commit. Thanks.
        Hide
        Simone Leo added a comment -

        Updated for current trunk

        Show
        Simone Leo added a comment - Updated for current trunk
        Hide
        Simone Leo added a comment -

        Updated for current trunk

        Show
        Simone Leo added a comment - Updated for current trunk
        Hide
        Simone Leo added a comment -

        Attached version 2 of the patch – put in sync with current trunk

        Show
        Simone Leo added a comment - Attached version 2 of the patch – put in sync with current trunk
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12472943/MAPREDUCE-1125-2.patch
        against trunk revision 1079072.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 5 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/130//testReport/
        Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/130//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/130//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12472943/MAPREDUCE-1125-2.patch against trunk revision 1079072. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/130//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/130//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/130//console This message is automatically generated.
        Hide
        Simone Leo added a comment -

        Looks like some tests failed due to timeout. Those tests have nothing to do with this patch, anyway.

        Show
        Simone Leo added a comment - Looks like some tests failed due to timeout. Those tests have nothing to do with this patch, anyway.
        Hide
        Todd Lipcon added a comment -

        This is a sort of incompatible change - some user might be explicitly declaring the extern void deserializeFloat(float &t, InStream &stream), right?

        Do we need to leave the old version around for a version? Perhaps someone who makes use of SerialUtils can comment - I never have worked on significant amounts of C++ code with Hadoop

        Show
        Todd Lipcon added a comment - This is a sort of incompatible change - some user might be explicitly declaring the extern void deserializeFloat(float &t, InStream &stream), right? Do we need to leave the old version around for a version? Perhaps someone who makes use of SerialUtils can comment - I never have worked on significant amounts of C++ code with Hadoop
        Hide
        Simone Leo added a comment -

        In Pydoop we are currently patching SerialUtils.cc at compile time, before building the Python extension with Boost.

        Show
        Simone Leo added a comment - In Pydoop we are currently patching SerialUtils.cc at compile time, before building the Python extension with Boost.
        Hide
        Arun C Murthy added a comment -

        Sorry to come in late, the patch has gone stale. Can you please rebase? Thanks.

        Show
        Arun C Murthy added a comment - Sorry to come in late, the patch has gone stale. Can you please rebase? Thanks.
        Hide
        Simone Leo added a comment -

        rebased for current trunk

        Show
        Simone Leo added a comment - rebased for current trunk
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12494006/MAPREDUCE-1125-3.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 5 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12494006/MAPREDUCE-1125-3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/702//console This message is automatically generated.

          People

          • Assignee:
            Simone Leo
            Reporter:
            Simone Leo
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development