Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.15.0
    • Component/s: util
    • Labels:
      None
    • Environment:

      All

      Description

      ToolBase which has been superceded by ToolRunner doesn't pass in an existing configuration object and therefore won't pick up initial configuration resources. One consequence of this is the nutch default and site.xml files are ignored.

      1. HADOOP-2012-2.patch
        2 kB
        Enis Soztutar
      2. NUTCH-2102-1.patch
        0.9 kB
        Dennis Kubes

        Activity

        Hide
        Dennis Kubes added a comment -

        Simple patch, sets the configuration in ToolBase to the passed in configuration.

        Show
        Dennis Kubes added a comment - Simple patch, sets the configuration in ToolBase to the passed in configuration.
        Hide
        Dennis Kubes added a comment -

        Don't know why, first one got named as a sql file. Removed and re-uploaded as patch file.

        Show
        Dennis Kubes added a comment - Don't know why, first one got named as a sql file. Removed and re-uploaded as patch file.
        Hide
        Enis Soztutar added a comment -

        Dennis, we can just change :

        public final int doMain(Configuration conf, String[] args) throws Exception {
          return ToolRunner.run(conf, this, args);
        }
        

        I will apply the patch, but I think its better if nutch give up using ToolBase in the first place, since it will be removed soon.

        Show
        Enis Soztutar added a comment - Dennis, we can just change : public final int doMain(Configuration conf, String [] args) throws Exception { return ToolRunner.run(conf, this , args); } I will apply the patch, but I think its better if nutch give up using ToolBase in the first place, since it will be removed soon.
        Hide
        Enis Soztutar added a comment -

        Marking this as blocker for 0.15, since for this release we just deprecate ToolBase, there will be classes depending on correct behavior of it.

        Show
        Enis Soztutar added a comment - Marking this as blocker for 0.15, since for this release we just deprecate ToolBase, there will be classes depending on correct behavior of it.
        Hide
        Enis Soztutar added a comment -

        updated patch.

        Show
        Enis Soztutar added a comment - updated patch.
        Hide
        Dennis Kubes added a comment -

        Ah, I like your change better. If I had bothered to read ToolRunner I probably would have gone that route. We are going to stop using ToolBase for Nutch. Those changes will probably go into our current patch sets for upgrading to 0.15 hadoop. Just didn't want anything current to break upon upgrade.

        Show
        Dennis Kubes added a comment - Ah, I like your change better. If I had bothered to read ToolRunner I probably would have gone that route. We are going to stop using ToolBase for Nutch. Those changes will probably go into our current patch sets for upgrading to 0.15 hadoop. Just didn't want anything current to break upon upgrade.
        Hide
        Arun C Murthy added a comment -

        +1

        Show
        Arun C Murthy added a comment - +1
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12368485/HADOOP-2012-2.patch
        against trunk revision r588778.

        @author +1. The patch does not contain any @author tags.

        javadoc +1. The javadoc tool did not generate any warning messages.

        javac +1. The applied patch does not generate any new compiler warnings.

        findbugs +1. The patch does not introduce any new Findbugs warnings.

        core tests -1. The patch failed core unit tests.

        contrib tests +1. The patch passed contrib unit tests.

        Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1014/testReport/
        Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1014/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1014/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1014/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12368485/HADOOP-2012-2.patch against trunk revision r588778. @author +1. The patch does not contain any @author tags. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new compiler warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests -1. The patch failed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1014/testReport/ Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1014/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1014/artifact/trunk/build/test/checkstyle-errors.html Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1014/console This message is automatically generated.
        Hide
        Devaraj Das added a comment -

        The core-tests failed due to HADOOP-2112. I think this patch can be committed ignoring the testcase failure.

        Show
        Devaraj Das added a comment - The core-tests failed due to HADOOP-2112 . I think this patch can be committed ignoring the testcase failure.
        Hide
        Enis Soztutar added a comment -

        So, me Arun and Devaraj agree that this is a blocker. I will commit this one to trunk and 0.15 unless any objections.

        Show
        Enis Soztutar added a comment - So, me Arun and Devaraj agree that this is a blocker. I will commit this one to trunk and 0.15 unless any objections.
        Hide
        Owen O'Malley added a comment -

        +1

        Show
        Owen O'Malley added a comment - +1
        Hide
        Arun C Murthy added a comment -

        I just committed this. Thanks, Dennis and Enis!

        Show
        Arun C Murthy added a comment - I just committed this. Thanks, Dennis and Enis!
        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Hadoop-Nightly #286 (See http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/286/ )

          People

          • Assignee:
            Dennis Kubes
            Reporter:
            Dennis Kubes
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development