Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-21699

Create table failed when using SPLITS_FILE => 'splits.txt'

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 2.0.0, 2.0.1, 2.1.1, 2.0.2, 2.0.3, 2.1.2, 2.0.4
    • 3.0.0-alpha-1, 2.2.0, 2.1.3, 2.0.5, 2.3.0
    • Client, shell
    • None
    • Patch

    Description

      Hi all:

       When I ran 

      create 't1', 'f1', SPLITS_FILE => 'splits.txt'
      

      on HBase2.0.0, it failed, and no detailed error info, just like below:

      ERROR: 
      
      Creates a table. Pass a table name, and a set of column family
      specifications (at least one), and, optionally, table configuration.
      Column specification can be a simple string (name), or a dictionary
      (dictionaries are described below in main help output), necessarily
      including NAME attribute.
      Examples:
      

      So I opened the debug:

      hbase shell -d
      

      and 

      ERROR: 
      Backtrace: org.apache.hadoop.hbase.util.Bytes.toBytes(org/apache/hadoop/hbase/util/Bytes.java:732)
      org.apache.hadoop.hbase.HTableDescriptor.setValue(org/apache/hadoop/hbase/HTableDescriptor.java:190)

      But it works on branch 1.2.0.

      so I view the source code, I find the issue is because the below code:

      // admin.rb
      if arg.key?(SPLITS_FILE)
        splits_file = arg.delete(SPLITS_FILE)
        unless File.exist?(splits_file)
          raise(ArgumentError, "Splits file #{splits_file} doesn't exist")
        end
        arg[SPLITS] = []
        File.foreach(splits_file) do |line|
          arg[SPLITS].push(line.chomp)
        end
        htd.setValue(SPLITS_FILE, arg[SPLITS_FILE])
      end
      
      // HTableDescriptor part
      public HTableDescriptor setValue(String key, String value) {
        getDelegateeForModification().setValue(Bytes.toBytes(key), Bytes.toBytes(value));
        return this;
      }
      
      // Bytes part
      public static byte[] toBytes(String s) {
        try {
          return s.getBytes(UTF8_CSN);
        } catch (UnsupportedEncodingException e) {
          // should never happen!
          throw new IllegalArgumentException("UTF8 decoding is not supported", e);
        }
      }
      

      Call flow is:

      admin.rb ---> htd.setValue(SPLITS_FILE, arg[SPLITS_FILE]) ---> Bytes.toBytes(key) && Bytes.toBytes(value) 

      from Bytes.toBytes, if s is null, the function will throw NullPointerException, but HTableDescriptor.setValue(String key, String value) does not check key and value.

      in admin.rb, it use arg.delete(SPLITS_FILE) to get the value, but this means, after using arg.delete(SPLITS_FILE),  arg[SPLITS_FILE] will return nil. so HTableDescriptor.setValue(String key, String value) does not check key and value is root cause.

      why branch below 2.0.0 works fine, because old code is :

      public HTableDescriptor setValue(String key, String value) {
      if (value == null) {
      remove(key);
      } else {
      setValue(Bytes.toBytes(key), Bytes.toBytes(value));
      }
      return this;
      }
      

      it check the value.

       since branch 2.0.0, HBase add new function called 'TableDescriptorBuilder'

      it included:

      public ModifyableTableDescriptor setValue(String key, String value) {
        return setValue(toBytesOrNull(key, Bytes::toBytes),
                toBytesOrNull(value, Bytes::toBytes));
      }
      

      it checked key and value, but HTableDescriptor.setValue(String key, String value)  does not call it, so

      just change HTableDescriptor.setValue(String key, String value) to call it, it will works

      Attachments

        1. hbase-21699.001.patch
          2 kB
          huan
        2. HBASE-21699.master.006.patch
          7 kB
          Guanghao Zhang
        3. HBase-21699.v2.patch
          5 kB
          huan
        4. HBase-21699.v3.patch
          7 kB
          huan
        5. HBase-21699.v4.patch
          6 kB
          huan
        6. HBase-21699.v5.patch
          6 kB
          huan

        Activity

          People

            rebornHuan huan
            rebornHuan huan
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: