Hadoop Common
  1. Hadoop Common
  2. HADOOP-8722

Update BUILDING.txt with latest snappy info

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.2-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: documentation
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      HADOOP-8620 changed the default of snappy.lib from snappy.prefix/lib to empty. This, in turn, means that you can't use -Dbundle.snappy without setting -Dsnappy.lib

      1. HADOOP-8722.002.patch
        1 kB
        Colin Patrick McCabe
      2. HADOOP-8722.003.patch
        2 kB
        Colin Patrick McCabe
      3. HADOOP-8722.004.patch
        2 kB
        Colin Patrick McCabe
      4. hadoop-8722.txt
        2 kB
        Eli Collins

        Activity

        Hide
        Eli Collins added a comment -

        Maybe just update BUILDING.txt to indicate snappy.lib is required? Seems like with this patch we could end up copying two snappy lib files and think it's fine to just doc the required dirs.

        Show
        Eli Collins added a comment - Maybe just update BUILDING.txt to indicate snappy.lib is required? Seems like with this patch we could end up copying two snappy lib files and think it's fine to just doc the required dirs.
        Hide
        Colin Patrick McCabe added a comment -

        Looking at our code again, I think it requires the library to be dynamic, so I can just skip the .a stuff. We use dlopen, and I don't think that works with .a (static) libraries.

        At that point, there no danger of making two copies of the .so file, since the second copy would overwrite the first. If the developer puts two different copies of libsnappy.so in ${snappy.lib} or ${snappy.prefix}, I think we can safely discount that as user error.

        Show
        Colin Patrick McCabe added a comment - Looking at our code again, I think it requires the library to be dynamic, so I can just skip the .a stuff. We use dlopen , and I don't think that works with .a (static) libraries. At that point, there no danger of making two copies of the .so file, since the second copy would overwrite the first. If the developer puts two different copies of libsnappy.so in ${snappy.lib } or ${snappy.prefix }, I think we can safely discount that as user error.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12542061/HADOOP-8722.002.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1349//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1349//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542061/HADOOP-8722.002.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +0 tests included. The patch appears to be a documentation patch that doesn't require tests. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1349//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1349//console This message is automatically generated.
        Hide
        Colin Patrick McCabe added a comment -

        After thinking about it, I think I agree with Eli's suggestion here. Let's just update the docs. It's the easiest option for everyone.

        The docs need to be updated anyway because they're missing a few of the newer options.

        Show
        Colin Patrick McCabe added a comment - After thinking about it, I think I agree with Eli's suggestion here. Let's just update the docs. It's the easiest option for everyone. The docs need to be updated anyway because they're missing a few of the newer options.
        Hide
        Colin Patrick McCabe added a comment -

        slightly improved version

        Show
        Colin Patrick McCabe added a comment - slightly improved version
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12542988/HADOOP-8722.003.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1383//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1383//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542988/HADOOP-8722.003.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +0 tests included. The patch appears to be a documentation patch that doesn't require tests. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1383//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1383//console This message is automatically generated.
        Hide
        Eli Collins added a comment -

        Looks good, minor nits:

        • "-Dsnappy.prefix or -Dsnappy.lib" should be "and"
        • "if Snappy is missing" should be "and snappy is not installed on your host"
        • "Most of the time, these options should not be necessary" should be something like "These options are not required if the snappy libraries and headers are installed in the system lib and includes directories."
        Show
        Eli Collins added a comment - Looks good, minor nits: "-Dsnappy.prefix or -Dsnappy.lib" should be "and" "if Snappy is missing" should be "and snappy is not installed on your host" "Most of the time, these options should not be necessary" should be something like "These options are not required if the snappy libraries and headers are installed in the system lib and includes directories."
        Hide
        Colin Patrick McCabe added a comment -

        I reworded this a bit. Thinking about it, I realized the snappy options might look better in a single section. Also there should be some intro text about what the heck snappy is before launching into these details.

        "-Dsnappy.prefix or -Dsnappy.lib" should be "and"

        Seemed better to give each option its own section, I got rid of the 'or' completely...

        "Most of the time, these options should not be necessary" should be something like "These options are not required if the snappy libraries and headers are installed in the system lib and includes directories."

        I replaced this with "You do not need this option if you have installed snappy through... a package manager"

        Show
        Colin Patrick McCabe added a comment - I reworded this a bit. Thinking about it, I realized the snappy options might look better in a single section. Also there should be some intro text about what the heck snappy is before launching into these details. "-Dsnappy.prefix or -Dsnappy.lib" should be "and" Seemed better to give each option its own section, I got rid of the 'or' completely... "Most of the time, these options should not be necessary" should be something like "These options are not required if the snappy libraries and headers are installed in the system lib and includes directories." I replaced this with "You do not need this option if you have installed snappy through... a package manager"
        Hide
        Eli Collins added a comment -

        +1 looks great. Attaching a slight updated version with some small tweaks.

        Show
        Eli Collins added a comment - +1 looks great. Attaching a slight updated version with some small tweaks.
        Hide
        Eli Collins added a comment -

        I've committed this, thanks Colin!

        Show
        Eli Collins added a comment - I've committed this, thanks Colin!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1151 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1151/)
        HADOOP-8722. Update BUILDING.txt with latest snappy info. Contributed by Colin Patrick McCabe (Revision 1379280)

        Result = FAILURE
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379280
        Files :

        • /hadoop/common/trunk/BUILDING.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1151 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1151/ ) HADOOP-8722 . Update BUILDING.txt with latest snappy info. Contributed by Colin Patrick McCabe (Revision 1379280) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379280 Files : /hadoop/common/trunk/BUILDING.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1182 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1182/)
        HADOOP-8722. Update BUILDING.txt with latest snappy info. Contributed by Colin Patrick McCabe (Revision 1379280)

        Result = SUCCESS
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379280
        Files :

        • /hadoop/common/trunk/BUILDING.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1182 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1182/ ) HADOOP-8722 . Update BUILDING.txt with latest snappy info. Contributed by Colin Patrick McCabe (Revision 1379280) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379280 Files : /hadoop/common/trunk/BUILDING.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt

          People

          • Assignee:
            Colin Patrick McCabe
            Reporter:
            Eli Collins
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development