Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-10078

Dynamic Filter - Not using DynamicClassLoader when using FilterList

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.94.13
    • Fix Version/s: 0.94.16
    • Component/s: Filters
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      I've tried to use dynamic jar load (https://issues.apache.org/jira/browse/HBASE-1936) but seems to have an issue with FilterList.
      Here is some log from my app where i send a Get with a FilterList containing AFilter and other with BFilter.

      2013-12-02 13:55:42,564 DEBUG org.apache.hadoop.hbase.util.DynamicClassLoader: Class d.p.AFilter not found - using dynamical class loader
      2013-12-02 13:55:42,564 DEBUG org.apache.hadoop.hbase.util.DynamicClassLoader: Finding class: d.p.AFilter
      2013-12-02 13:55:42,564 DEBUG org.apache.hadoop.hbase.util.DynamicClassLoader: Loading new jar files, if any
      2013-12-02 13:55:42,677 DEBUG org.apache.hadoop.hbase.util.DynamicClassLoader: Finding class again: d.p.AFilter
      2013-12-02 13:55:43,004 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable: Can't find class d.p.BFilter
      java.lang.ClassNotFoundException: d.p.BFilter
      	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
      	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
      	at java.lang.Class.forName0(Native Method)
      	at java.lang.Class.forName(Class.java:247)
      	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
      	at org.apache.hadoop.hbase.io.HbaseObjectWritable.getClassByName(HbaseObjectWritable.java:792)
      	at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:679)
      	at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:594)
      	at org.apache.hadoop.hbase.filter.FilterList.readFields(FilterList.java:324)
      	at org.apache.hadoop.hbase.client.Get.readFields(Get.java:405)
      	at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:690)
      	at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:594)
      	at org.apache.hadoop.hbase.client.Action.readFields(Action.java:101)
      	at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:690)
      	at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:594)
      	at org.apache.hadoop.hbase.client.MultiAction.readFields(MultiAction.java:116)
      	at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:690)
      	at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:126)
      	at org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1311)
      	at org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1226)
      	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
      	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
      	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      	at java.lang.Thread.run(Thread.java:662)
      

      AFilter is not found so it tries with DynamicClassLoader, but when it tries to load AFilter, it uses URLClassLoader and fails without checking out for dynamic jars.

      I think the issue is releated to FilterList#readFields

      FilterList.java
       
       public void readFields(final DataInput in) throws IOException {
          byte opByte = in.readByte();
          operator = Operator.values()[opByte];
          int size = in.readInt();
          if (size > 0) {
            filters = new ArrayList<Filter>(size);
            for (int i = 0; i < size; i++) {
              Filter filter = (Filter)HbaseObjectWritable.readObject(in, conf);
              filters.add(filter);
            }
          }
        }
      

      HbaseObjectWritable#readObject uses a conf (created by calling HBaseConfiguration.create()) which i suppose doesn't include a DynamicClassLoader instance.

        Attachments

        1. 0.94-10078_v2.patch
          12 kB
          Jimmy Xiang
        2. hbase-10078.patch
          7 kB
          Jimmy Xiang
        3. 0.94-10078.patch
          12 kB
          Jimmy Xiang

          Issue Links

            Activity

              People

              • Assignee:
                jxiang Jimmy Xiang
                Reporter:
                fgaule Federico Gaule
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: