[HBASE-27246] RSGroupMappingScript#getRSGroup has thread safety problem

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
• Component/s: rsgroup
• Hadoop Flags: Reviewed
• Assignee: Yutong Xiao
• Reporter: Yutong Xiao
• Created: 27/Jul/22 09:34
• Updated: 25/Aug/22 05:54
• Resolved: 24/Aug/22 08:59

Description

We are using version 1.4.12 and met a problem in table creation phase some time. The master branch also has this problem. The error message is:

2022-07-26 19:26:20.122 [http-nio-8078-exec-24,d2ad4b13b542b6fb] ERROR HBaseServiceImpl - hbase create table: xxx:xxxx failed. (HBaseServiceImpl.java:116)
java.lang.RuntimeException: org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Default RSGroup (default
default) for this table's namespace does not exist.

The rsgroup here should be one 'default' but not two consecutive 'default'. The code to get RSGroup from a mapping script is:

String getRSGroup(String namespace, String tablename) {
      if (rsgroupMappingScript == null) {
        return null;
      }
      String[] exec = rsgroupMappingScript.getExecString();
      exec[1] = namespace;
      exec[2] = tablename;
      try {
        rsgroupMappingScript.execute();
      } catch (IOException e) {
        // This exception may happen, like process doesn't have permission to run this script.
        LOG.error("{}, placing {} back to default rsgroup", e.getMessage(),
          TableName.valueOf(namespace, tablename));
        return RSGroupInfo.DEFAULT_GROUP;
      }
      return rsgroupMappingScript.getOutput().trim();
    }

here the rsgourpMappingScript could be executed by multi-threads.

To test it is a multi-thread issue, I ran a piece of code locally and found that the hadoop ShellCommandExecutor is not thread-safe (I run the code with hadoop 2.10.0 and 3.3.2). So that we should make this method synchronized. Besides, I found that this issue is retained in master branch also.

The test code is attached and my rsgroup mapping script is very simple:

#!/bin/bash

namespace=$1
tablename=$2
echo default

The reproduced screenshot is also attached.

Issue Links

• links to: GitHub Pull Request #4657