Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.18.0
    • Fix Version/s: 0.18.2
    • Component/s: documentation
    • Labels:
      None

      Description

      After reviewing bugs 3505 & 3668, following changes are required in HOD documentation.

      1. HOD user guide :
      ===============
      Term HDFS should be used instead of DFS under following sections:
      3.9. Capturing HOD exit codes in Torque
      4.3. hod Fails With an error code and error message

      2. HOD admin guide:
      ================
      'Torque' should be a hyperlink in Pre-requisites section of hod admin guide.

      1. HADOOP-3786
        10 kB
        Vinod Kumar Vavilapalli

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk #636 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/636/)
          . Use HDFS instead of DFS in all docs and hyperlink to Torque. Contributed by Vinod Kumar Vavilapalli.

          Show
          Hudson added a comment - Integrated in Hadoop-trunk #636 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/636/ ) . Use HDFS instead of DFS in all docs and hyperlink to Torque. Contributed by Vinod Kumar Vavilapalli.
          Hide
          Arun C Murthy added a comment -

          I just committed this. Thanks, Vinod!

          Show
          Arun C Murthy added a comment - I just committed this. Thanks, Vinod!
          Hide
          Arun C Murthy added a comment -

          Never mind - Nigel pointed out that -Djava5.home doesn't work for branch-0.18 ...

          Show
          Arun C Murthy added a comment - Never mind - Nigel pointed out that -Djava5.home doesn't work for branch-0.18 ...
          Hide
          Arun C Murthy added a comment -

          I committed this to trunk and branch-0.19, however when I try running 'ant docs' on branch-0.18 it fails with:

              [exec] BUILD FAILED
               [exec] /Users/arunc/Installs/apache-forrest-0.8/main/targets/validate.xml:158: java.lang.NullPointerException
               [exec] 
               [exec] Total time: 5 seconds
          
          Show
          Arun C Murthy added a comment - I committed this to trunk and branch-0.19, however when I try running 'ant docs' on branch-0.18 it fails with: [exec] BUILD FAILED [exec] /Users/arunc/Installs/apache-forrest-0.8/main/targets/validate.xml:158: java.lang.NullPointerException [exec] [exec] Total time: 5 seconds
          Hide
          Hemanth Yamijala added a comment -

          Request to committers. Can you please modify src/contrib/hod/CHANGES.txt when committing this ? It has the same format as the main CHANGES.txt file.

          Show
          Hemanth Yamijala added a comment - Request to committers. Can you please modify src/contrib/hod/CHANGES.txt when committing this ? It has the same format as the main CHANGES.txt file.
          Hide
          Hemanth Yamijala added a comment -

          +1 for the patch. Core test failure is not related to this HOD documentation fix.

          Show
          Hemanth Yamijala added a comment - +1 for the patch. Core test failure is not related to this HOD documentation fix.
          Hide
          Devaraj Das added a comment -

          Please mark it as a blocker for 0.19 if required

          Show
          Devaraj Das added a comment - Please mark it as a blocker for 0.19 if required
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12390906/HADOOP-3786
          against trunk revision 698721.

          +1 @author. The patch does not contain any @author tags.

          +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3369/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3369/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3369/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3369/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12390906/HADOOP-3786 against trunk revision 698721. +1 @author. The patch does not contain any @author tags. +0 tests included. The patch appears to be a documentation patch that doesn't require tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3369/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3369/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3369/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3369/console This message is automatically generated.
          Hide
          Craig Macdonald added a comment -

          +1 for the changes relating to Torque.

          Show
          Craig Macdonald added a comment - +1 for the changes relating to Torque.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Attaching documentation fixes.

          Show
          Vinod Kumar Vavilapalli added a comment - Attaching documentation fixes.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Also add the documentation for HADOOP-3959 as part of this JIRA.

          Show
          Vinod Kumar Vavilapalli added a comment - Also add the documentation for HADOOP-3959 as part of this JIRA.
          Hide
          Craig Macdonald added a comment -

          Can you update the HOD documentation to record that PBS env vars can be using the hodrc?

          For instance, our cluster automatically allocates local space on each node for a job. This allows us to use the following configuration for HOD, and it works nicely!:

          [ringmaster]
          temp-dir                        = /scratch/pbstmp.$PBS_JOBID
          work-dirs                       = /scratch/pbstmp.$PBS_JOBID/1,/scratch/pbstmp.$PBS_JOBID/2
          [hodring]
          temp-dir                        = /scratch/pbstmp.$PBS_JOBID
          

          If documentation is updated, HADOOP-2862 can be closed as a duplicate.

          C

          Show
          Craig Macdonald added a comment - Can you update the HOD documentation to record that PBS env vars can be using the hodrc? For instance, our cluster automatically allocates local space on each node for a job. This allows us to use the following configuration for HOD, and it works nicely!: [ringmaster] temp-dir = /scratch/pbstmp.$PBS_JOBID work-dirs = /scratch/pbstmp.$PBS_JOBID/1,/scratch/pbstmp.$PBS_JOBID/2 [hodring] temp-dir = /scratch/pbstmp.$PBS_JOBID If documentation is updated, HADOOP-2862 can be closed as a duplicate. C
          Hide
          Hemanth Yamijala added a comment -

          Seems a reasonable thing to document. We'll take this up.

          Show
          Hemanth Yamijala added a comment - Seems a reasonable thing to document. We'll take this up.
          Hide
          Karam Singh added a comment -

          Some times after successful hod allocation using dynamic dfs. if we immediately fire dfs -put put. User will get messages like -: file replicated 0 out 3 times and after 4 out of 5 retries dfs -put command starts working. On investigation, it is found that datanodes were still initializing.
          So if we use hdfs command after 10 secs of hod allocation there is no problem
          Should we add this hod user guide?

          Show
          Karam Singh added a comment - Some times after successful hod allocation using dynamic dfs. if we immediately fire dfs -put put. User will get messages like -: file replicated 0 out 3 times and after 4 out of 5 retries dfs -put command starts working. On investigation, it is found that datanodes were still initializing. So if we use hdfs command after 10 secs of hod allocation there is no problem Should we add this hod user guide?

            People

            • Assignee:
              Vinod Kumar Vavilapalli
              Reporter:
              Suman Sehgal
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development