Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-7301

HdfsDirectoryFactory does not support maprfs

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Hadoop Integration
    • Labels:
      None

      Description

      when map-reduce index generator was run, an exception was thrown as
      2015-03-24 11:06:04,569 WARN org.apache.hadoop.mapred.Child: Error running child
      java.lang.IllegalStateException: Failed to initialize record writer for MapReduceSolrIndex, attempt_201503171620_12558_r_000000_0
      at org.apache.solr.hadoop.SolrRecordWriter.<init>(SolrRecordWriter.java:127)
      at org.apache.solr.hadoop.SolrOutputFormat.getRecordWriter(SolrOutputFormat.java:164)
      at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:605)
      at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:456)
      at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
      at org.apache.hadoop.mapred.Child.main(Child.java:264)
      Caused by: org.apache.solr.common.SolrException: Unable to create core [core1]
      at org.apache.solr.core.CoreContainer.create(CoreContainer.java:507)
      at org.apache.solr.core.CoreContainer.create(CoreContainer.java:466)
      at org.apache.solr.hadoop.SolrRecordWriter.createEmbeddedSolrServer(SolrRecordWriter.java:172)
      at org.apache.solr.hadoop.SolrRecordWriter.<init>(SolrRecordWriter.java:120)
      ... 8 more
      Caused by: org.apache.solr.common.SolrException: You must set the HdfsDirectoryFactory param solr.hdfs.home for relative dataDir paths to work
      at org.apache.solr.core.HdfsDirectoryFactory.getDataHome(HdfsDirectoryFactory.java:271)
      at org.apache.solr.core.SolrCore.<init>(SolrCore.java:699)
      at org.apache.solr.core.SolrCore.<init>(SolrCore.java:646)
      at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491)
      ... 11 more

      after investigation, I found the class HdfsDirectoryFactory hardcoded "hdfs:/".

      a patch is provided in the attachment.

        Attachments

        1. fix_support_maprfs.patch
          0.9 kB
          Shenghua Wan

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                wanshenghua Shenghua Wan
              • Votes:
                2 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: