Details
-
Bug
-
Status: Resolved
-
Trivial
-
Resolution: Cannot Reproduce
-
1.5.0
-
None
-
None
-
None
Description
Problem:
I can use a custom iterator using the setiter command but the same iterator does not work using the setscaniter or setshelliter commands.
References:
https://blogs.apache.org/accumulo/entry/the_accumulo_classloader
http://accumulo.apache.org/1.5/examples/classpath.html
Description:
I am using my https://github.com/medined/D4M_Schema project to start Accumulo. So the environment that I am using can be duplicated exactly if needed. I am using
Accumulo: 1.5.0
Hadoop: 1.2.1
The classpath settings in accumulo-site.xml are the following (which I think are the default):
<property>
<name>general.classpaths</name>
<value>
$ACCUMULO_HOME/server/target/classes/,
$ACCUMULO_HOME/core/target/classes/,
$ACCUMULO_HOME/start/target/classes/,
$ACCUMULO_HOME/examples/target/classes/,
$ACCUMULO_HOME/lib/[^.].$ACCUMULO_VERSION.jar,
$ACCUMULO_HOME/lib/[^.].*.jar,
$ZOOKEEPER_HOME/zookeeper[^.].*.jar,
$HADOOP_HOME/conf,
$HADOOP_HOME/[^.].*.jar,
$HADOOP_HOME/lib/[^.].*.jar,
</value>
<description>Classpaths that accumulo checks for updates and class files.
When using the Security Manager, please remove the ".../target/classes/" values.
</description>
</property>
I can load my iterator using setiter but not with setscaniter or setshelliter.
Here is my do-nothing iterator:
public class MyIterator extends WrappingIterator implements OptionDescriber {
@Override
public IteratorOptions describeOptions()
@Override
public boolean validateOptions(Map<String, String> options)
}
I copy the jar file out to HDFS:
hadoop fs -mkdir /user/vagrant/d4m/classpath
hadoop fs -put /vagrant/schema/target/d4m_schema-0.0.1-SNAPSHOT.jar /user/vagrant/classpath
I set the table-specific classpath context:
createtable atest
table atest
insert row cf cq value
config -s general.vfs.context.classpath.d4m=hdfs://affy-master:9000/user/vagrant/classpath
config -t atest -s table.classpath.context=d4m
Now I can configure the iterator and scan over the single row without a problem:
setiter -n MyIterator -p 10 -scan -minc -majc -class com.codebits.d4m.iterator.MyIterator
scan
deleteiter -n MyIterator -scan -minc -majc
However, the setscaniter commands fails:
root@instance atest> setscaniter -n MyIterator -p 10 -class com.codebits.d4m.iterator.MyIterator
2014-06-15 02:54:14,098 [shell.Shell] WARN : Deprecated, use setshelliter
Dummy Description
2014-06-15 02:54:14,126 [shell.Shell] ERROR: org.apache.accumulo.core.util.shell.ShellCommandException: Command could not be initialized (Unable to load com.codebits.d4m.iterator.MyIterator)
As does the setshelliter:
root@instance atest> setshelliter -pn d4m -n MyIterator -p 10 -class com.codebits.d4m.iterator.MyIterator
Dummy Description
2014-06-15 02:55:07,025 [shell.Shell] ERROR: org.apache.accumulo.core.util.shell.ShellCommandException: Command could not be initialized (Unable to load com.codebits.d4m.iterator.MyIterator)
I don't see any messages in the log files.
Attachments
Issue Links
- relates to
-
ACCUMULO-3093 Shell scan command does not find formatter
- Resolved