Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: nutchgora
    • Fix Version/s: nutchgora
    • Component/s: build
    • Labels:
      None

      Description

      As mentioned in NUTCH-950, we can slightly improve the Ivy configuration to help setup the Gora backend more easily.
      If the user does not want to stick with default HSQL database, other alternatives exist, such as MySQL and HBase.

      org.restlet and xercesImpl versions should be changed as well.

        Activity

        Hide
        Alexis added a comment - - edited

        In the patch, the required dependencies for MySQL and HBase are included in the Ivy config, but commented out as suggested in Julien's comment. It's up to the user to use his own backend to store the data.

        Following 3 points are minor issues but the fixes allow to play more nicely under Eclipse:

        • The call to "nutch.root" property set in build.xml for ant should be replaced in src/plugin/protocol-sftp/ivy.xml by the built-in "basedir" ivy property.
        • The 2.0.1 version of restlet dependency does not exist in the maven repository, so you want to manually change it to 2.0.0.
        • The xerces (XML parser) implementation needs to be upgraded from 2.6.2 to 2.9.1, otherwise you'll see exceptions while running a JUnit test.
        Show
        Alexis added a comment - - edited In the patch, the required dependencies for MySQL and HBase are included in the Ivy config, but commented out as suggested in Julien's comment. It's up to the user to use his own backend to store the data. Following 3 points are minor issues but the fixes allow to play more nicely under Eclipse: The call to "nutch.root" property set in build.xml for ant should be replaced in src/plugin/protocol-sftp/ivy.xml by the built-in "basedir" ivy property. The 2.0.1 version of restlet dependency does not exist in the maven repository, so you want to manually change it to 2.0.0. The xerces (XML parser) implementation needs to be upgraded from 2.6.2 to 2.9.1, otherwise you'll see exceptions while running a JUnit test.
        Hide
        Alexis added a comment -

        Sorry please disregard the nutch.root first bullet in the previous comment and in the patch. This would break the build: basedir variable holds the plugin's base directory ("Nutch2.0/src/plugin/protocol-sftp"). I get an error in the build saying ivy/ivy-configurations.xml is not found with this patch.

        I need to figure out how to load this nutch.root variable in the Ivy plugin in Eclipse.

        Show
        Alexis added a comment - Sorry please disregard the nutch.root first bullet in the previous comment and in the patch. This would break the build: basedir variable holds the plugin's base directory ("Nutch2.0/src/plugin/protocol-sftp"). I get an error in the build saying ivy/ivy-configurations.xml is not found with this patch. I need to figure out how to load this nutch.root variable in the Ivy plugin in Eclipse.
        Hide
        Andrzej Bialecki added a comment -

        Committed with a tweak in rev. 1079770. Thanks!

        Show
        Andrzej Bialecki added a comment - Committed with a tweak in rev. 1079770. Thanks!

          People

          • Assignee:
            Andrzej Bialecki
            Reporter:
            Alexis
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development