[SOLR-7301] HdfsDirectoryFactory does not support maprfs

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Component/s: Hadoop Integration
• Assignee: Unassigned
• Reporter: Shenghua Wan
• Created: 24/Mar/15 23:59
• Updated: 02/Feb/19 20:45
• Resolved: 02/Feb/19 20:45

Description

when map-reduce index generator was run, an exception was thrown as
2015-03-24 11:06:04,569 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.IllegalStateException: Failed to initialize record writer for MapReduceSolrIndex, attempt_201503171620_12558_r_000000_0
at org.apache.solr.hadoop.SolrRecordWriter.<init>(SolrRecordWriter.java:127)
at org.apache.solr.hadoop.SolrOutputFormat.getRecordWriter(SolrOutputFormat.java:164)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:605)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:456)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: org.apache.solr.common.SolrException: Unable to create core [core1]
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:507)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:466)
at org.apache.solr.hadoop.SolrRecordWriter.createEmbeddedSolrServer(SolrRecordWriter.java:172)
at org.apache.solr.hadoop.SolrRecordWriter.<init>(SolrRecordWriter.java:120)
... 8 more
Caused by: org.apache.solr.common.SolrException: You must set the HdfsDirectoryFactory param solr.hdfs.home for relative dataDir paths to work
at org.apache.solr.core.HdfsDirectoryFactory.getDataHome(HdfsDirectoryFactory.java:271)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:699)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:646)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491)
... 11 more

after investigation, I found the class HdfsDirectoryFactory hardcoded "hdfs:/".

a patch is provided in the attachment.

Issue Links

• is related to: SOLR-11473 Make HDFSDirectoryFactory support other prefixes (besides hdfs:/)