Details

    • Type: Task Task
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: documentation
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Native Libraries Guide - Update

      Updated/edited guide for Hadoop 0.21 release.
      Some passages/statements were not clear (user confusion).

      Note: Setting priority to Blocker to make sure these changes get into Hadoop 0.21.0 release (and to pave the way for future updates/additions to this guide)

      1. Hadoo-6292-2.patch
        12 kB
        Corinne Chandel
      2. Hadoop-6292.patch
        12 kB
        Corinne Chandel
      3. Hadoop-6292-3.patch
        13 kB
        Corinne Chandel
      4. Native_Libraries_Guide_UPDATE-2.pdf
        19 kB
        Corinne Chandel
      5. Native_Libraries_Guide_UPDATE-3.pdf
        19 kB
        Corinne Chandel
      6. Native_Libraries_Guide_UPDATE-4.pdf
        20 kB
        Corinne Chandel
      7. Native-Lib-4-Branch-021.patch
        13 kB
        Corinne Chandel
      8. Native-Lib-4-Trunk.patch
        13 kB
        Corinne Chandel
      9. Native Libraries Guide-UPDATE.docx
        18 kB
        Corinne Chandel

        Activity

        Hide
        Corinne Chandel added a comment -

        (1) Hadoop-6292.patch - patch file

        (2) Native-Libraries-Guide-UPDATE.docs - file showing updated guide

        Show
        Corinne Chandel added a comment - (1) Hadoop-6292.patch - patch file (2) Native-Libraries-Guide-UPDATE.docs - file showing updated guide
        Hide
        Corinne Chandel added a comment -

        Apply this patch to: http://svn.apache.org/repos/asf/hadoop/common/trunk

        Note: No new test code required; change to documentation only.

        Show
        Corinne Chandel added a comment - Apply this patch to: http://svn.apache.org/repos/asf/hadoop/common/trunk Note: No new test code required; change to documentation only.
        Hide
        Allen Wittenauer added a comment -

        One fix that needs to be made is that not all of the native libraries are written in ANSI C. Some use GNU extensions and may require special flags and/or code changes to work on ANSI compilers.

        Show
        Allen Wittenauer added a comment - One fix that needs to be made is that not all of the native libraries are written in ANSI C. Some use GNU extensions and may require special flags and/or code changes to work on ANSI compilers.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12420943/Native+Libraries+Guide-UPDATE.docx
        against trunk revision 820094.

        +1 @author. The patch does not contain any @author tags.

        +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

        -1 patch. The patch command could not apply the patch.

        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/67/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12420943/Native+Libraries+Guide-UPDATE.docx against trunk revision 820094. +1 @author. The patch does not contain any @author tags. +0 tests included. The patch appears to be a documentation patch that doesn't require tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/67/console This message is automatically generated.
        Hide
        Corinne Chandel added a comment -

        Thanks Allen. I will make change, get review, and submit another patch. /C

        Show
        Corinne Chandel added a comment - Thanks Allen. I will make change, get review, and submit another patch. /C
        Hide
        Arun C Murthy added a comment -

        One fix that needs to be made is that not all of the native libraries are written in ANSI C. Some use GNU extensions and may require special flags and/or code changes to work on ANSI compilers.

        Allen - can you please clarify whether you are talking about libhadoop.so here or the compression codecs (lzo, gzip, bzip) it relies on?

        Show
        Arun C Murthy added a comment - One fix that needs to be made is that not all of the native libraries are written in ANSI C. Some use GNU extensions and may require special flags and/or code changes to work on ANSI compilers. Allen - can you please clarify whether you are talking about libhadoop.so here or the compression codecs (lzo, gzip, bzip) it relies on?
        Hide
        Allen Wittenauer added a comment -

        native compression appears to be ANSI. HADOOP-6262 and HADOOP-6263 documents two of the problems I hit with libhadoop. MAPREDUCE-991 documents issues with pipes.

        Show
        Allen Wittenauer added a comment - native compression appears to be ANSI. HADOOP-6262 and HADOOP-6263 documents two of the problems I hit with libhadoop. MAPREDUCE-991 documents issues with pipes.
        Hide
        Arun C Murthy added a comment -

        Thanks for clarifying Allen - HADOOP-6262 and HADOOP-6263 are both related to hadoop-pipes, nothing to do with libhadoop.so which is only native compression for now.

        Show
        Arun C Murthy added a comment - Thanks for clarifying Allen - HADOOP-6262 and HADOOP-6263 are both related to hadoop-pipes, nothing to do with libhadoop.so which is only native compression for now.
        Hide
        Corinne Chandel added a comment -

        Arun - Thanks for your help.

        Owen - Please proceed, using the attached patch (hadoop-6292.patch).

        Allen - If you would like to add a "pipes" section to this doc, please open a new JIRA and provide content.

        Show
        Corinne Chandel added a comment - Arun - Thanks for your help. Owen - Please proceed, using the attached patch (hadoop-6292.patch). Allen - If you would like to add a "pipes" section to this doc, please open a new JIRA and provide content.
        Hide
        Allen Wittenauer added a comment -

        To me:

        "Native Libraries" == all .so's I have to compile

        "Native Compression" == all .so's I have to compile related specifically to compression

        I think we have a terminology problem here. I'm sure I'm not the only one confused.

        Show
        Allen Wittenauer added a comment - To me: "Native Libraries" == all .so's I have to compile "Native Compression" == all .so's I have to compile related specifically to compression I think we have a terminology problem here. I'm sure I'm not the only one confused.
        Hide
        Corinne Chandel added a comment -

        No. You are not the only one who is confused.

        Show
        Corinne Chandel added a comment - No. You are not the only one who is confused.
        Hide
        Corinne Chandel added a comment -

        In the edited version (attached to this JIRA):

        Section: Download Native Hadoop Libraries
        This section is talking specifically about libhadoop.so

        Section: Building Native Hadoop Libraries
        This section is talking specifically about libhadoop.so

        Section: Loading Native Shared Libraries
        This section is talking about any native shared library ...

        As noted, this document probably needs to be expanded to include more sections ...

        Show
        Corinne Chandel added a comment - In the edited version (attached to this JIRA): Section: Download Native Hadoop Libraries This section is talking specifically about libhadoop.so Section: Building Native Hadoop Libraries This section is talking specifically about libhadoop.so Section: Loading Native Shared Libraries This section is talking about any native shared library ... As noted, this document probably needs to be expanded to include more sections ...
        Hide
        Corinne Chandel added a comment -

        Updates -

        patch #2

        doc-update #2

        Show
        Corinne Chandel added a comment - Updates - patch #2 doc-update #2
        Hide
        Corinne Chandel added a comment -

        Added note about *.so terminology to introduction.

        Made revisions to 2 section titles.

        Owen - Please use patch #2 (hadoo-6292-2.patch)

        Show
        Corinne Chandel added a comment - Added note about *.so terminology to introduction. Made revisions to 2 section titles. Owen - Please use patch #2 (hadoo-6292-2.patch)
        Hide
        Chris Douglas added a comment -

        The bzip2 codec is implemented in Java and is not required to build or deploy the native libs. Other than that, this looks good

        Show
        Chris Douglas added a comment - The bzip2 codec is implemented in Java and is not required to build or deploy the native libs. Other than that, this looks good
        Hide
        Corinne Chandel added a comment -

        Patch #3 - please use this patch.
        PDF #3 - shows latest changes.

        Note: Removed references to bzip2; reorganized sections again.

        Thanks Chris!

        Show
        Corinne Chandel added a comment - Patch #3 - please use this patch. PDF #3 - shows latest changes. Note: Removed references to bzip2; reorganized sections again. Thanks Chris!
        Hide
        Corinne Chandel added a comment -

        Updated guide again (#4).

        (1) PDF #4 shows latest changse.

        (2) Patch for trunk.

        Apply: http://svn.apache.org/repos/asf/hadoop/common/trunk

        (3) Patch for branch-0.21

        Apply: http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.21

        Note: No new test code required; changes to documentation only.

        Show
        Corinne Chandel added a comment - Updated guide again (#4). (1) PDF #4 shows latest changse. (2) Patch for trunk. Apply: http://svn.apache.org/repos/asf/hadoop/common/trunk (3) Patch for branch-0.21 Apply: http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.21 Note: No new test code required; changes to documentation only.
        Hide
        Chris Douglas added a comment -
             [exec] +1 overall.  
             [exec] 
             [exec]     +1 @author.  The patch does not contain any @author tags.
             [exec] 
             [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
             [exec]                         Please justify why no new tests are needed for this patch.
             [exec]                         Also please list what manual steps were performed to verify this patch.
             [exec] 
             [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
             [exec] 
             [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
             [exec] 
             [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
             [exec] 
             [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
        
        Show
        Chris Douglas added a comment - [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no new tests are needed for this patch. [exec] Also please list what manual steps were performed to verify this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
        Hide
        Chris Douglas added a comment -

        +1

        I committed this. Thanks, Corinne!

        Show
        Chris Douglas added a comment - +1 I committed this. Thanks, Corinne!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #65 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/65/)
        . Update native libraries guide. Contributed by Corinne Chandel

        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #65 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/65/ ) . Update native libraries guide. Contributed by Corinne Chandel
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk #134 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/134/)
        . Update native libraries guide. Contributed by Corinne Chandel

        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk #134 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/134/ ) . Update native libraries guide. Contributed by Corinne Chandel

          People

          • Assignee:
            Corinne Chandel
            Reporter:
            Corinne Chandel
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development