Solr
  1. Solr
  2. SOLR-6855

bin/solr -e dih launches, but has some path cruft issues preventing some of the imports don't work

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0
    • Component/s: None
    • Labels:
      None

      Description

      While trying to update this ref guide page...
      https://cwiki.apache.org/confluence/display/solr/Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler

      I encountered a bunch of problems when running bin/solr -e dih:

      1. every collection in example/example-DIH/solr started getting _rest_managed* and _schema_analysis* files created in it
        • either we should commit these empty files into the example, or pair down the schema's in these collections not to use these fieldTypes
      2. a server/example-DIH directory got created containing some hsqldb logs & properties
      3. at least 2 of the full import commands don't seem to work
        • the DB probably isn't working because the path to the hsqldb may not be correct anymore - hence the problem above as well (JDBC probably relative to CWD at the moment? need sys prop to be relative to solr home?)
        • the tika import doesn't seem to work either - probably another relative path problem
      4. the example/example-DIH/README.txt file still needs updated to refer to bin/solr -e dih

        Activity

        Hide
        Hoss Man added a comment -

        first start at a patch...

        • fixes the READEME.txt
        • fixes the hsqldb & tika paths by using \${solr.install.dir}
        • removes the (unused) managed_en fieldType from all the schemas

        When commiting this:

        • these files can be removed and added to svn:ignores for this dir (should have never been commited in the first place):
          • solr/example/example-DIH/hsqldb/ex.properties
          • solr/example/example-DIH/hsqldb/ex.log
        • all of the ./conf dirs should have "dataimport.properties" added t o svn:ignore

        Things that still need work:

        • _rest_managed.json files are still created for every collection - i thought these were only created if/when something wanted to use the resource manager, but i may be wrong – if there is no way to avoid them, we should commit empty copies of them

        Probably should be defered to a distinct issue but...

        these example configs are still big and bloated and carry over a lot of stuff that isn't needed to make them work - i really think that the dih solrconfig.xml and schema.xml should be as small as freaking possible to still demonstrate the dih features they use, with comments directing people to basic_configs & the sample_techproducts_configs to see other features of solr.

        ie: we don't bother having dih configured in the techproducs example because it's not needed – why do we have a bunch of tech product cruft that's not needed configured in the dih examples?

        Show
        Hoss Man added a comment - first start at a patch... fixes the READEME.txt fixes the hsqldb & tika paths by using \${solr.install.dir} removes the (unused) managed_en fieldType from all the schemas When commiting this: these files can be removed and added to svn:ignores for this dir (should have never been commited in the first place): solr/example/example-DIH/hsqldb/ex.properties solr/example/example-DIH/hsqldb/ex.log all of the ./conf dirs should have "dataimport.properties" added t o svn:ignore Things that still need work: _rest_managed.json files are still created for every collection - i thought these were only created if/when something wanted to use the resource manager, but i may be wrong – if there is no way to avoid them, we should commit empty copies of them Probably should be defered to a distinct issue but... these example configs are still big and bloated and carry over a lot of stuff that isn't needed to make them work - i really think that the dih solrconfig.xml and schema.xml should be as small as freaking possible to still demonstrate the dih features they use, with comments directing people to basic_configs & the sample_techproducts_configs to see other features of solr. ie: we don't bother having dih configured in the techproducs example because it's not needed – why do we have a bunch of tech product cruft that's not needed configured in the dih examples?
        Hide
        Timothy Potter added a comment -

        _rest_managed.json files are still created for every collection

        Sorry - that's my bad ... The easiest would be to just commit the _rest_managed.json files but I'd like to see if there's a clean way to just make the RestManager initialize lazily the first time it's invoked, which will mean any cores that don't need it will never have this file created.

        Show
        Timothy Potter added a comment - _rest_managed.json files are still created for every collection Sorry - that's my bad ... The easiest would be to just commit the _rest_managed.json files but I'd like to see if there's a clean way to just make the RestManager initialize lazily the first time it's invoked, which will mean any cores that don't need it will never have this file created.
        Hide
        ASF subversion and git services added a comment -

        Commit 1647825 from Timothy Potter in branch 'dev/trunk'
        [ https://svn.apache.org/r1647825 ]

        SOLR-6855: bin/solr -e dih launches, but has some path cruft issues preventing some of the imports don't work

        Show
        ASF subversion and git services added a comment - Commit 1647825 from Timothy Potter in branch 'dev/trunk' [ https://svn.apache.org/r1647825 ] SOLR-6855 : bin/solr -e dih launches, but has some path cruft issues preventing some of the imports don't work
        Hide
        ASF subversion and git services added a comment -

        Commit 1647843 from Timothy Potter in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1647843 ]

        SOLR-6855: bin/solr -e dih launches, but has some path cruft issues preventing some of the imports don't work

        Show
        ASF subversion and git services added a comment - Commit 1647843 from Timothy Potter in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1647843 ] SOLR-6855 : bin/solr -e dih launches, but has some path cruft issues preventing some of the imports don't work
        Hide
        Anshum Gupta added a comment -

        Bulk close after 5.0 release.

        Show
        Anshum Gupta added a comment - Bulk close after 5.0 release.

          People

          • Assignee:
            Timothy Potter
            Reporter:
            Hoss Man
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development