GitHub user dlmarion opened a pull request:

https://github.com/apache/accumulo/pull/83

ACCUMULO-4173: Host Regex Table Load Balancer

A table load balancer that passes a subset of the tserver hosts to the configured tablet balancer for a table. Tservers that do not match any regex are part of a default pool. Tables with configured regex (or those that are in error) are balanced within the default pool of tservers.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/accumulo ACCUMULO-4173

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/accumulo/pull/83.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #83

commit 594ecc3c6a876922b1a47301a619551e32c31e7a
Author: Dave Marion <dlmarion@apache.org>
Date: 2016-03-29T14:33:41Z

Squashed commit of the following:

commit 2afa820763ece3a1299ac9e8889532fa4966acff
Author: Dave Marion <dlmarion@apache.org>
Date: Fri Mar 25 14:36:18 2016 -0400

Added tests where tablets were either all assigned or partially assigned

commit 9c08258b7907ad3195227d21a782ffe7545fa696
Author: Dave Marion <dlmarion@apache.org>
Date: Fri Mar 25 14:26:57 2016 -0400

Added test for out of bounds tablets

commit be07ba3d20e378d88c71f77366e1b3f2dd325c73
Author: Dave Marion <dlmarion@apache.org>
Date: Fri Mar 25 13:38:35 2016 -0400

Added more tests

commit dc842305d4fda77d2271ae8a111031bfb1eda9f7
Author: Dave Marion <dlmarion@apache.org>
Date: Fri Mar 25 11:31:20 2016 -0400

Initial work on Host based regex master balancer

Github user dlmarion commented on the pull request:

https://github.com/apache/accumulo/pull/83#issuecomment-203039636

Jenkins failure is not due to the patch.

Github user brianloss commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57927227

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,314 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ String tableName = getPoolNameForHost(e.getKey().host());
+ if (!newPools.containsKey(tableName)) {
+ newPools.put(tableName, new TreeMap<TServerInstance,TabletServerStatus>());
— End diff –

Is there a need to ensure the TreeMap objects you create here have the same comparator as that used by current (see SortedMap.comparator())? If the incoming map isn't sorted based on the natural ordering, then this could be a problem.

Github user brianloss commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57928954

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,314 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ String tableName = getPoolNameForHost(e.getKey().host());
+ if (!newPools.containsKey(tableName))

+ newPools.get(tableName).put(e.getKey(), e.getValue());
+ }
+ // Ensure that no host is in more than one pool
+ // TODO: I'm not sure that I need to check for disjoint as the call to getPoolNameForHost checks for more than one match
+ boolean error = false;
+ for (SortedMap<TServerInstance,TabletServerStatus> s1 : newPools.values()) {
— End diff –

This nested loop is doing a lot more disjoint checks than need to be done. You could change the inner loop to iterate over a tailMap starting from the current outer loop value to get rid of the dups.
That being said, I can also think of cases where you might actually want disjoint pools. For example, one pool could be for "archive" data which is large and not accessed very often, so you want to spread it across all tservers. Then other pools, for hot tables would be on subsets of the tservers. To support this would mean some re-architecting of the code since the check would need to be get pool for host and table name.

Github user brianloss commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57929241

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,314 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ String tableName = getPoolNameForHost(e.getKey().host());
+ if (!newPools.containsKey(tableName))

+ newPools.get(tableName).put(e.getKey(), e.getValue());
+ }
+ // Ensure that no host is in more than one pool
+ // TODO: I'm not sure that I need to check for disjoint as the call to getPoolNameForHost checks for more than one match
+ boolean error = false;
+ for (SortedMap<TServerInstance,TabletServerStatus> s1 : newPools.values()) {
+ for (SortedMap<TServerInstance,TabletServerStatus> s2 : newPools.values()) {
+ if (s1 == s2)

+ if (!Collections.disjoint(s1.keySet(), s2.keySet())) {
+ LOG.error("Pools are not disjoint: {}, there is a problem with your regular expressions. Putting all servers in default pool", newPools);
+ error = true;
+ }
+ }
+ }
+ if (error) {
+ // Put all servers into the default pool
+ newPools.clear();
+ newPools.put(DEFAULT_POOL, new TreeMap<TServerInstance,TabletServerStatus>());
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
— End diff –

Nitpick, so ignore if you want, but you could replace this loop with newPools.putAll(current). (Same comment as above about the comparator also applies.)

Github user brianloss commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57929457

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,314 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ String tableName = getPoolNameForHost(e.getKey().host());
+ if (!newPools.containsKey(tableName))

+ newPools.get(tableName).put(e.getKey(), e.getValue());
+ }
+ // Ensure that no host is in more than one pool
+ // TODO: I'm not sure that I need to check for disjoint as the call to getPoolNameForHost checks for more than one match
+ boolean error = false;
+ for (SortedMap<TServerInstance,TabletServerStatus> s1 : newPools.values()) {
+ for (SortedMap<TServerInstance,TabletServerStatus> s2 : newPools.values()) {
+ if (s1 == s2)

+ if (!Collections.disjoint(s1.keySet(), s2.keySet())) {
+ LOG.error("Pools are not disjoint: {}, there is a problem with your regular expressions. Putting all servers in default pool", newPools);
+ error = true;
+ }
+ }
+ }
+ if (error) {
+ // Put all servers into the default pool
+ newPools.clear();
+ newPools.put(DEFAULT_POOL, new TreeMap<TServerInstance,TabletServerStatus>());
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet())

+ }
+ pools = newPools;
+ this.lastPoolRecheck = System.currentTimeMillis();
+ }
+ return pools;
+ }
+
+ /**
+ * Matches host against the regexes and returns the matching pool name
+ *
+ * @param host
+ * tablet server host
+ * @return name of pool. will return default pool if host matches more than one regex
+ */
+ protected String getPoolNameForHost(String host) {
+ String test = host;
+ String table = DEFAULT_POOL;
— End diff –

Another nitpick, but I think this variable should be called pool not table.

Github user brianloss commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57930356

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,314 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ String tableName = getPoolNameForHost(e.getKey().host());
+ if (!newPools.containsKey(tableName))

+ newPools.get(tableName).put(e.getKey(), e.getValue());
+ }
+ // Ensure that no host is in more than one pool
+ // TODO: I'm not sure that I need to check for disjoint as the call to getPoolNameForHost checks for more than one match
+ boolean error = false;
+ for (SortedMap<TServerInstance,TabletServerStatus> s1 : newPools.values()) {
+ for (SortedMap<TServerInstance,TabletServerStatus> s2 : newPools.values()) {
+ if (s1 == s2)

+ if (!Collections.disjoint(s1.keySet(), s2.keySet())) {
+ LOG.error("Pools are not disjoint: {}, there is a problem with your regular expressions. Putting all servers in default pool", newPools);
+ error = true;
+ }
+ }
+ }
+ if (error) {
+ // Put all servers into the default pool
+ newPools.clear();
+ newPools.put(DEFAULT_POOL, new TreeMap<TServerInstance,TabletServerStatus>());
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet())

+ }
+ pools = newPools;
+ this.lastPoolRecheck = System.currentTimeMillis();
+ }
+ return pools;
+ }
+
+ /**
+ * Matches host against the regexes and returns the matching pool name
+ *
+ * @param host
+ * tablet server host
+ * @return name of pool. will return default pool if host matches more than one regex
+ */
+ protected String getPoolNameForHost(String host) {
+ String test = host;
+ String table = DEFAULT_POOL;
+ if (!isIpBasedRegex) {
+ try

catch (UnknownHostException e1)

+ }
+ for (Entry<String,Pattern> e : poolNameToRegexPattern.entrySet()) {
+ if (e.getValue().matcher(test).matches()) {
+ if (!table.equals(DEFAULT_POOL)) {
+ LOG.warn("host {} matches more than one regex, assigning to default pool", host);
+ return DEFAULT_POOL;
+ }
+ table = e.getKey();
+ }
+ }
+ return table;
+ }
+
+ protected String getNameFromIp(String hostIp) throws UnknownHostException

+
+ /**
+ * Matches table name against pool names, returns matching pool name or DEFAULT_POOL.
+ *
+ * @param tableName
+ * name of table
+ * @return tablet server pool name (table name or DEFAULT_POOL)
+ */
+ protected String getPoolNameForTable(String tableName) {
+ if (null == tableName)

+ return poolNameToRegexPattern.containsKey(tableName) ? tableName : DEFAULT_POOL;
+ }
+
+ /**
+ * Parse configuration and extract properties
+ *
+ * @param conf
+ * server configuration
+ */
+ protected void parseConfiguration(ServerConfiguration conf) {
+ tableIdToTableName = new HashMap<>();
+ poolNameToRegexPattern = new HashMap<>();
+ for (Entry<String,String> table : getTableOperations().tableIdMap().entrySet()) {
— End diff –

What happens if a new table is created after the balance is initialized, and the properties are updated to define a pool for that table? Also, what happens if a table is renamed?

Github user dlmarion commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57930648

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,314 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ String tableName = getPoolNameForHost(e.getKey().host());
+ if (!newPools.containsKey(tableName))

+ newPools.get(tableName).put(e.getKey(), e.getValue());
+ }
+ // Ensure that no host is in more than one pool
+ // TODO: I'm not sure that I need to check for disjoint as the call to getPoolNameForHost checks for more than one match
+ boolean error = false;
+ for (SortedMap<TServerInstance,TabletServerStatus> s1 : newPools.values()) {
— End diff –

I have no issue removing this check. However, getPoolNameForHost only returns one pool name; either the name of the pool that matches one regex, or the DEFAULT. I believe to support this example, getPoolNameForHost would have to change to return all matching pool names, and return DEFAULT when it matches none.

Github user brianloss commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57931945

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,314 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ String tableName = getPoolNameForHost(e.getKey().host());
+ if (!newPools.containsKey(tableName))

+ newPools.get(tableName).put(e.getKey(), e.getValue());
+ }
+ // Ensure that no host is in more than one pool
+ // TODO: I'm not sure that I need to check for disjoint as the call to getPoolNameForHost checks for more than one match
+ boolean error = false;
+ for (SortedMap<TServerInstance,TabletServerStatus> s1 : newPools.values()) {
+ for (SortedMap<TServerInstance,TabletServerStatus> s2 : newPools.values()) {
+ if (s1 == s2)

+ String tid = getTableOperations().tableIdMap().get(pool);
— End diff –

If a table is renamed, this will likely cause problems.

Github user brianloss commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57933756

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,314 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ String tableName = getPoolNameForHost(e.getKey().host());
+ if (!newPools.containsKey(tableName))

+ newPools.get(tableName).put(e.getKey(), e.getValue());
+ }
+ // Ensure that no host is in more than one pool
+ // TODO: I'm not sure that I need to check for disjoint as the call to getPoolNameForHost checks for more than one match
+ boolean error = false;
+ for (SortedMap<TServerInstance,TabletServerStatus> s1 : newPools.values()) {
+ for (SortedMap<TServerInstance,TabletServerStatus> s2 : newPools.values()) {
+ if (s1 == s2)

+ String tid = getTableOperations().tableIdMap().get(pool);
+ try {
+ List<TabletStats> outOfBoundsTablets = getOnlineTabletsForTable(e.getKey(), tid);
+ for (TabletStats ts : outOfBoundsTablets) {
+ LOG.info("Tablet {} is currently outside the bounds of the regex, reassigning", ts.toString());
+ unassigned.put(new KeyExtent(ts.getExtent()), e.getKey());
— End diff –

getAssignments is for making assignments where there isn't one currently. Is it safe to take a tablet that is already assigned, and generate a new assignment here? It might be totally fine--but it's something to verify.

Github user dlmarion commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57934586

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,314 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ String tableName = getPoolNameForHost(e.getKey().host());
+ if (!newPools.containsKey(tableName))

+ newPools.get(tableName).put(e.getKey(), e.getValue());
+ }
+ // Ensure that no host is in more than one pool
+ // TODO: I'm not sure that I need to check for disjoint as the call to getPoolNameForHost checks for more than one match
+ boolean error = false;
+ for (SortedMap<TServerInstance,TabletServerStatus> s1 : newPools.values()) {
+ for (SortedMap<TServerInstance,TabletServerStatus> s2 : newPools.values()) {
+ if (s1 == s2)

+ if (!Collections.disjoint(s1.keySet(), s2.keySet())) {
+ LOG.error("Pools are not disjoint: {}, there is a problem with your regular expressions. Putting all servers in default pool", newPools);
+ error = true;
+ }
+ }
+ }
+ if (error) {
+ // Put all servers into the default pool
+ newPools.clear();
+ newPools.put(DEFAULT_POOL, new TreeMap<TServerInstance,TabletServerStatus>());
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet())

+ }
+ pools = newPools;
+ this.lastPoolRecheck = System.currentTimeMillis();
+ }
+ return pools;
+ }
+
+ /**
+ * Matches host against the regexes and returns the matching pool name
+ *
+ * @param host
+ * tablet server host
+ * @return name of pool. will return default pool if host matches more than one regex
+ */
+ protected String getPoolNameForHost(String host) {
+ String test = host;
+ String table = DEFAULT_POOL;
+ if (!isIpBasedRegex) {
+ try

catch (UnknownHostException e1)

+ }
+ for (Entry<String,Pattern> e : poolNameToRegexPattern.entrySet()) {
+ if (e.getValue().matcher(test).matches()) {
+ if (!table.equals(DEFAULT_POOL)) {
+ LOG.warn("host {} matches more than one regex, assigning to default pool", host);
+ return DEFAULT_POOL;
+ }
+ table = e.getKey();
+ }
+ }
+ return table;
+ }
+
+ protected String getNameFromIp(String hostIp) throws UnknownHostException

+
+ /**
+ * Matches table name against pool names, returns matching pool name or DEFAULT_POOL.
+ *
+ * @param tableName
+ * name of table
+ * @return tablet server pool name (table name or DEFAULT_POOL)
+ */
+ protected String getPoolNameForTable(String tableName) {
+ if (null == tableName)

+ return poolNameToRegexPattern.containsKey(tableName) ? tableName : DEFAULT_POOL;
+ }
+
+ /**
+ * Parse configuration and extract properties
+ *
+ * @param conf
+ * server configuration
+ */
+ protected void parseConfiguration(ServerConfiguration conf) {
+ tableIdToTableName = new HashMap<>();
+ poolNameToRegexPattern = new HashMap<>();
+ for (Entry<String,String> table : getTableOperations().tableIdMap().entrySet()) {
— End diff –

I think you would need to restart the master

Github user brianloss commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57934799

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,314 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ String tableName = getPoolNameForHost(e.getKey().host());
+ if (!newPools.containsKey(tableName))

+ newPools.get(tableName).put(e.getKey(), e.getValue());
+ }
+ // Ensure that no host is in more than one pool
+ // TODO: I'm not sure that I need to check for disjoint as the call to getPoolNameForHost checks for more than one match
+ boolean error = false;
+ for (SortedMap<TServerInstance,TabletServerStatus> s1 : newPools.values()) {
+ for (SortedMap<TServerInstance,TabletServerStatus> s2 : newPools.values()) {
+ if (s1 == s2)

+ String tid = getTableOperations().tableIdMap().get(pool);
+ try {
+ List<TabletStats> outOfBoundsTablets = getOnlineTabletsForTable(e.getKey(), tid);
+ for (TabletStats ts : outOfBoundsTablets) {
+ LOG.info("Tablet {} is currently outside the bounds of the regex, reassigning", ts.toString());
+ unassigned.put(new KeyExtent(ts.getExtent()), e.getKey());
— End diff –

Assuming this is safe for now, you are taking an entry from current, adding it to unassigned, but leaving it in current. Then the results get passed to the table-specific balancer below. Not sure if this will cause any problems for the table-specific balancers. It's probably worth removing the assignment from current.

Github user brianloss commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57935222

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,314 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ String tableName = getPoolNameForHost(e.getKey().host());
+ if (!newPools.containsKey(tableName))

+ newPools.get(tableName).put(e.getKey(), e.getValue());
+ }
+ // Ensure that no host is in more than one pool
+ // TODO: I'm not sure that I need to check for disjoint as the call to getPoolNameForHost checks for more than one match
+ boolean error = false;
+ for (SortedMap<TServerInstance,TabletServerStatus> s1 : newPools.values()) {
+ for (SortedMap<TServerInstance,TabletServerStatus> s2 : newPools.values()) {
+ if (s1 == s2)

+ String tid = getTableOperations().tableIdMap().get(pool);
+ try {
+ List<TabletStats> outOfBoundsTablets = getOnlineTabletsForTable(e.getKey(), tid);
+ for (TabletStats ts : outOfBoundsTablets) {
+ LOG.info("Tablet {} is currently outside the bounds of the regex, reassigning", ts.toString());
+ unassigned.put(new KeyExtent(ts.getExtent()), e.getKey());
+ }
+ } catch (TException e1) {
+ LOG.error("Error in OOB check getting tablets for table {} from server {}", tid, e.getKey().host(), e);
+ }
+ }
+ }
+ this.oobCheckMillis = System.currentTimeMillis();
+ }
+
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = splitCurrentByRegex(current);
+ // separate the unassigned into tables
+ Map<String,Map<KeyExtent,TServerInstance>> groupedUnassigned = new HashMap<String,Map<KeyExtent,TServerInstance>>();
+ for (Entry<KeyExtent,TServerInstance> e : unassigned.entrySet()) {
+ Map<KeyExtent,TServerInstance> tableUnassigned = groupedUnassigned.get(e.getKey().getTableId());
+ if (tableUnassigned == null)

+ tableUnassigned.put(e.getKey(), e.getValue());
+ }
+ // Send a view of the current servers to the tables tablet balancer
+ for (Entry<String,Map<KeyExtent,TServerInstance>> e : groupedUnassigned.entrySet())

+ }
+
+ @Override
+ public long balance(SortedMap<TServerInstance,TabletServerStatus> current, Set<KeyExtent> migrations, List<TabletMigration> migrationsOut) {
+ long minBalanceTime = 5 * 1000;
+ // Iterate over the tables and balance each of them
+ TableOperations t = getTableOperations();
+ if (t == null)
— End diff –

You call getTableOperations() in other places in this code, and assume the response won't be null. Why is it treated differently here? Either this check is superfluous, or the other places need to be fixed.

Github user ivakegg commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57935942

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,294 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
— End diff –

typo: ppol

Github user dlmarion commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57936092

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,314 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ String tableName = getPoolNameForHost(e.getKey().host());
+ if (!newPools.containsKey(tableName))

+ newPools.get(tableName).put(e.getKey(), e.getValue());
+ }
+ // Ensure that no host is in more than one pool
+ // TODO: I'm not sure that I need to check for disjoint as the call to getPoolNameForHost checks for more than one match
+ boolean error = false;
+ for (SortedMap<TServerInstance,TabletServerStatus> s1 : newPools.values()) {
+ for (SortedMap<TServerInstance,TabletServerStatus> s2 : newPools.values()) {
+ if (s1 == s2)

+ String tid = getTableOperations().tableIdMap().get(pool);
+ try {
+ List<TabletStats> outOfBoundsTablets = getOnlineTabletsForTable(e.getKey(), tid);
+ for (TabletStats ts : outOfBoundsTablets) {
+ LOG.info("Tablet {} is currently outside the bounds of the regex, reassigning", ts.toString());
+ unassigned.put(new KeyExtent(ts.getExtent()), e.getKey());
+ }
+ } catch (TException e1) {
+ LOG.error("Error in OOB check getting tablets for table {} from server {}", tid, e.getKey().host(), e);
+ }
+ }
+ }
+ this.oobCheckMillis = System.currentTimeMillis();
+ }
+
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = splitCurrentByRegex(current);
+ // separate the unassigned into tables
+ Map<String,Map<KeyExtent,TServerInstance>> groupedUnassigned = new HashMap<String,Map<KeyExtent,TServerInstance>>();
+ for (Entry<KeyExtent,TServerInstance> e : unassigned.entrySet()) {
+ Map<KeyExtent,TServerInstance> tableUnassigned = groupedUnassigned.get(e.getKey().getTableId());
+ if (tableUnassigned == null)

+ tableUnassigned.put(e.getKey(), e.getValue());
+ }
+ // Send a view of the current servers to the tables tablet balancer
+ for (Entry<String,Map<KeyExtent,TServerInstance>> e : groupedUnassigned.entrySet())

+ }
+
+ @Override
+ public long balance(SortedMap<TServerInstance,TabletServerStatus> current, Set<KeyExtent> migrations, List<TabletMigration> migrationsOut) {
+ long minBalanceTime = 5 * 1000;
+ // Iterate over the tables and balance each of them
+ TableOperations t = getTableOperations();
+ if (t == null)
— End diff –

Good catch. This is a copy/paste from TableLoadBalancer.balance(). It might be superfluous there.

Github user joshelser commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57936424

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,314 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ String tableName = getPoolNameForHost(e.getKey().host());
+ if (!newPools.containsKey(tableName))

+ newPools.get(tableName).put(e.getKey(), e.getValue());
+ }
+ // Ensure that no host is in more than one pool
+ // TODO: I'm not sure that I need to check for disjoint as the call to getPoolNameForHost checks for more than one match
+ boolean error = false;
+ for (SortedMap<TServerInstance,TabletServerStatus> s1 : newPools.values()) {
+ for (SortedMap<TServerInstance,TabletServerStatus> s2 : newPools.values()) {
+ if (s1 == s2)

+ if (!Collections.disjoint(s1.keySet(), s2.keySet())) {
+ LOG.error("Pools are not disjoint: {}, there is a problem with your regular expressions. Putting all servers in default pool", newPools);
+ error = true;
+ }
+ }
+ }
+ if (error) {
+ // Put all servers into the default pool
+ newPools.clear();
+ newPools.put(DEFAULT_POOL, new TreeMap<TServerInstance,TabletServerStatus>());
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet())

+ }
+ pools = newPools;
+ this.lastPoolRecheck = System.currentTimeMillis();
+ }
+ return pools;
+ }
+
+ /**
+ * Matches host against the regexes and returns the matching pool name
+ *
+ * @param host
+ * tablet server host
+ * @return name of pool. will return default pool if host matches more than one regex
+ */
+ protected String getPoolNameForHost(String host) {
+ String test = host;
+ String table = DEFAULT_POOL;
+ if (!isIpBasedRegex) {
+ try

catch (UnknownHostException e1)

+ }
+ for (Entry<String,Pattern> e : poolNameToRegexPattern.entrySet()) {
+ if (e.getValue().matcher(test).matches()) {
+ if (!table.equals(DEFAULT_POOL)) {
+ LOG.warn("host {} matches more than one regex, assigning to default pool", host);
+ return DEFAULT_POOL;
+ }
+ table = e.getKey();
+ }
+ }
+ return table;
+ }
+
+ protected String getNameFromIp(String hostIp) throws UnknownHostException

+
+ /**
+ * Matches table name against pool names, returns matching pool name or DEFAULT_POOL.
+ *
+ * @param tableName
+ * name of table
+ * @return tablet server pool name (table name or DEFAULT_POOL)
+ */
+ protected String getPoolNameForTable(String tableName) {
+ if (null == tableName)

+ return poolNameToRegexPattern.containsKey(tableName) ? tableName : DEFAULT_POOL;
+ }
+
+ /**
+ * Parse configuration and extract properties
+ *
+ * @param conf
+ * server configuration
+ */
+ protected void parseConfiguration(ServerConfiguration conf) {
+ tableIdToTableName = new HashMap<>();
+ poolNameToRegexPattern = new HashMap<>();
+ for (Entry<String,String> table : getTableOperations().tableIdMap().entrySet()) {
— End diff –

There's logic to handle configuration updates via a Callback. Seems logical that active Balancer could also tie into that callback to process configuration updates. (could be follow-on work since it already doesn't do that)

Github user ivakegg commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57936464

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,294 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ List<String> poolNames = getPoolNamesForHost(e.getKey().host());
+ for (String pool : poolNames) {
+ if (!newPools.containsKey(pool)) {
— End diff –

more efficient to do newPools.get(pool) and check result for null. Avoids additional get down below when adding to the sorted map

Github user brianloss commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57938201

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,294 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ List<String> poolNames = getPoolNamesForHost(e.getKey().host());
+ for (String pool : poolNames) {
+ if (!newPools.containsKey(pool))

+ newPools.get(pool).put(e.getKey(), e.getValue());
+ }
+ }
+ pools = newPools;
+ this.lastPoolRecheck = System.currentTimeMillis();
+ }
+ return pools;
+ }
+
+ /**
+ * Matches host against the regexes and returns the matching pool names
+ *
+ * @param host
+ * tablet server host
+ * @return pool names, will return default pool if host matches more no regex
+ */
+ protected List<String> getPoolNamesForHost(String host) {
+ String test = host;
+ if (!isIpBasedRegex) {
+ try

catch (UnknownHostException e1)

+ }
+ List<String> pools = new ArrayList<>();
+ for (Entry<String,Pattern> e : poolNameToRegexPattern.entrySet()) {
+ if (e.getValue().matcher(test).matches())

+ }
+ if (pools.size() == 0)

+ return pools;
+ }
+
+ protected String getNameFromIp(String hostIp) throws UnknownHostException

+
+ /**
+ * Matches table name against pool names, returns matching pool name or DEFAULT_POOL.
+ *
+ * @param tableName
+ * name of table
+ * @return tablet server pool name (table name or DEFAULT_POOL)
+ */
+ protected String getPoolNameForTable(String tableName) {
+ if (null == tableName)

+ return poolNameToRegexPattern.containsKey(tableName) ? tableName : DEFAULT_POOL;
+ }
+
+ /**
+ * Parse configuration and extract properties
+ *
+ * @param conf
+ * server configuration
+ */
+ protected void parseConfiguration(ServerConfiguration conf) {
+ tableIdToTableName = new HashMap<>();
+ poolNameToRegexPattern = new HashMap<>();
+ for (Entry<String,String> table : getTableOperations().tableIdMap().entrySet()) {
+ tableIdToTableName.put(table.getValue(), table.getKey());
+ Map<String,String> customProps = conf.getTableConfiguration(table.getValue()).getAllPropertiesWithPrefix(Property.TABLE_ARBITRARY_PROP_PREFIX);
+ if (null != customProps && customProps.size() > 0) {
+ for (Entry<String,String> customProp : customProps.entrySet())

+ }
+ }
+ String oobProperty = conf.getConfiguration().get(HOST_BALANCER_OOB_CHECK);
+ if (null != oobProperty)

+ String poolRecheckProperty = conf.getConfiguration().get(HOST_BALANCER_POOL_RECHECK_KEY);
+ if (null != poolRecheckProperty)

+ String ipBased = conf.getConfiguration().get(HOST_BALANCER_REGEX_USING_IPS);
+ if (null != ipBased)

+ }
+
+ public Map<String,String> getTableIdToTableName()

+
+ public Map<String,Pattern> getPoolNameToRegexPattern()

+
+ public long getOobCheckMillis()

+
+ public long getPoolRecheckMillis()

+
+ public boolean isIpBasedRegex()

+
+ @Override
+ public void init(ServerConfiguration conf)

+
+ @Override
+ public void getAssignments(SortedMap<TServerInstance,TabletServerStatus> current, Map<KeyExtent,TServerInstance> unassigned,
+ Map<KeyExtent,TServerInstance> assignments) {
+
+ if ((System.currentTimeMillis() - this.lastOOBCheck) > this.oobCheckMillis) {
+ // Check to see if a tablet is assigned outside the bounds of the pool. If so, migrate it.
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ for (String assignedPool : getPoolNamesForHost(e.getKey().host())) {
+ for (String pool : poolNameToRegexPattern.keySet()) {
+ if (assignedPool.equals(pool) || pool.equals(DEFAULT_POOL))

+ String tid = getTableOperations().tableIdMap().get(pool);
+ try {
+ List<TabletStats> outOfBoundsTablets = getOnlineTabletsForTable(e.getKey(), tid);
+ for (TabletStats ts : outOfBoundsTablets) {
+ LOG.info("Tablet {} is currently outside the bounds of the regex, reassigning", ts.toString());
+ unassigned.put(new KeyExtent(ts.getExtent()), e.getKey());
+ }
+ } catch (TException e1) {
+ LOG.error("Error in OOB check getting tablets for table {} from server {}", tid, e.getKey().host(), e);
+ }
+ }
+ }
+ }
+ this.oobCheckMillis = System.currentTimeMillis();
+ }
+
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = splitCurrentByRegex(current);
+ // separate the unassigned into tables
+ Map<String,Map<KeyExtent,TServerInstance>> groupedUnassigned = new HashMap<String,Map<KeyExtent,TServerInstance>>();
+ for (Entry<KeyExtent,TServerInstance> e : unassigned.entrySet()) {
+ Map<KeyExtent,TServerInstance> tableUnassigned = groupedUnassigned.get(e.getKey().getTableId());
+ if (tableUnassigned == null)

+ tableUnassigned.put(e.getKey(), e.getValue());
+ }
+ // Send a view of the current servers to the tables tablet balancer
+ for (Entry<String,Map<KeyExtent,TServerInstance>> e : groupedUnassigned.entrySet()) {
+ Map<KeyExtent,TServerInstance> newAssignments = new HashMap<KeyExtent,TServerInstance>();
+ String tableName = tableIdToTableName.get(e.getKey());
+ String poolName = getPoolNameForTable(tableName);
+ SortedMap<TServerInstance,TabletServerStatus> currentView = pools.get(poolName);
+ getBalancerForTable(e.getKey()).getAssignments(currentView, e.getValue(), newAssignments);
— End diff –

Many balancers will assign to the last known location (the entry part of the unassigned map). This code isn't doing any verification to ensure that the unassigned value is a tserver that corresponds to the pool for the table. That is, when you add to unassigned in the oob check above and when you group the unassigned, you should check to make sure the last value is valid (and I guess clear it if it is a server that doesn't belong to the target pool).

Github user ivakegg commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57940694

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,294 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ List<String> poolNames = getPoolNamesForHost(e.getKey().host());
+ for (String pool : poolNames) {
+ if (!newPools.containsKey(pool))

+ newPools.get(pool).put(e.getKey(), e.getValue());
+ }
+ }
+ pools = newPools;
+ this.lastPoolRecheck = System.currentTimeMillis();
+ }
+ return pools;
+ }
+
+ /**
+ * Matches host against the regexes and returns the matching pool names
+ *
+ * @param host
+ * tablet server host
+ * @return pool names, will return default pool if host matches more no regex
+ */
+ protected List<String> getPoolNamesForHost(String host) {
+ String test = host;
+ if (!isIpBasedRegex) {
+ try

catch (UnknownHostException e1)

+ }
+ List<String> pools = new ArrayList<>();
+ for (Entry<String,Pattern> e : poolNameToRegexPattern.entrySet()) {
+ if (e.getValue().matcher(test).matches())

+ }
+ if (pools.size() == 0)

+ return pools;
+ }
+
+ protected String getNameFromIp(String hostIp) throws UnknownHostException

+
+ /**
+ * Matches table name against pool names, returns matching pool name or DEFAULT_POOL.
+ *
+ * @param tableName
+ * name of table
+ * @return tablet server pool name (table name or DEFAULT_POOL)
+ */
+ protected String getPoolNameForTable(String tableName) {
+ if (null == tableName)

+ return poolNameToRegexPattern.containsKey(tableName) ? tableName : DEFAULT_POOL;
+ }
+
+ /**
+ * Parse configuration and extract properties
+ *
+ * @param conf
+ * server configuration
+ */
+ protected void parseConfiguration(ServerConfiguration conf) {
+ tableIdToTableName = new HashMap<>();
+ poolNameToRegexPattern = new HashMap<>();
+ for (Entry<String,String> table : getTableOperations().tableIdMap().entrySet()) {
+ tableIdToTableName.put(table.getValue(), table.getKey());
+ Map<String,String> customProps = conf.getTableConfiguration(table.getValue()).getAllPropertiesWithPrefix(Property.TABLE_ARBITRARY_PROP_PREFIX);
+ if (null != customProps && customProps.size() > 0) {
+ for (Entry<String,String> customProp : customProps.entrySet())

+ }
+ }
+ String oobProperty = conf.getConfiguration().get(HOST_BALANCER_OOB_CHECK);
+ if (null != oobProperty)

+ String poolRecheckProperty = conf.getConfiguration().get(HOST_BALANCER_POOL_RECHECK_KEY);
+ if (null != poolRecheckProperty)

+ String ipBased = conf.getConfiguration().get(HOST_BALANCER_REGEX_USING_IPS);
+ if (null != ipBased)

+ }
+
+ public Map<String,String> getTableIdToTableName()

+
+ public Map<String,Pattern> getPoolNameToRegexPattern()

+
+ public long getOobCheckMillis()

+
+ public long getPoolRecheckMillis()

+
+ public boolean isIpBasedRegex()

+
+ @Override
+ public void init(ServerConfiguration conf)

+
+ @Override
+ public void getAssignments(SortedMap<TServerInstance,TabletServerStatus> current, Map<KeyExtent,TServerInstance> unassigned,
+ Map<KeyExtent,TServerInstance> assignments) {
+
+ if ((System.currentTimeMillis() - this.lastOOBCheck) > this.oobCheckMillis) {
+ // Check to see if a tablet is assigned outside the bounds of the pool. If so, migrate it.
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ for (String assignedPool : getPoolNamesForHost(e.getKey().host())) {
+ for (String pool : poolNameToRegexPattern.keySet()) {
— End diff –

rename "pool" to "table" or "tableName" here and add a comment that the poolNames in poolNameToRegexPattern are the same as table names.

Github user dlmarion commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r57950095

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,294 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ List<String> poolNames = getPoolNamesForHost(e.getKey().host());
+ for (String pool : poolNames) {
+ if (!newPools.containsKey(pool))

+ newPools.get(pool).put(e.getKey(), e.getValue());
+ }
+ }
+ pools = newPools;
+ this.lastPoolRecheck = System.currentTimeMillis();
+ }
+ return pools;
+ }
+
+ /**
+ * Matches host against the regexes and returns the matching pool names
+ *
+ * @param host
+ * tablet server host
+ * @return pool names, will return default pool if host matches more no regex
+ */
+ protected List<String> getPoolNamesForHost(String host) {
+ String test = host;
+ if (!isIpBasedRegex) {
+ try

catch (UnknownHostException e1)

+ }
+ List<String> pools = new ArrayList<>();
+ for (Entry<String,Pattern> e : poolNameToRegexPattern.entrySet()) {
+ if (e.getValue().matcher(test).matches())

+ }
+ if (pools.size() == 0)

+ return pools;
+ }
+
+ protected String getNameFromIp(String hostIp) throws UnknownHostException

+
+ /**
+ * Matches table name against pool names, returns matching pool name or DEFAULT_POOL.
+ *
+ * @param tableName
+ * name of table
+ * @return tablet server pool name (table name or DEFAULT_POOL)
+ */
+ protected String getPoolNameForTable(String tableName) {
+ if (null == tableName)

+ return poolNameToRegexPattern.containsKey(tableName) ? tableName : DEFAULT_POOL;
+ }
+
+ /**
+ * Parse configuration and extract properties
+ *
+ * @param conf
+ * server configuration
+ */
+ protected void parseConfiguration(ServerConfiguration conf) {
+ tableIdToTableName = new HashMap<>();
+ poolNameToRegexPattern = new HashMap<>();
+ for (Entry<String,String> table : getTableOperations().tableIdMap().entrySet()) {
+ tableIdToTableName.put(table.getValue(), table.getKey());
+ Map<String,String> customProps = conf.getTableConfiguration(table.getValue()).getAllPropertiesWithPrefix(Property.TABLE_ARBITRARY_PROP_PREFIX);
+ if (null != customProps && customProps.size() > 0) {
+ for (Entry<String,String> customProp : customProps.entrySet())

+ }
+ }
+ String oobProperty = conf.getConfiguration().get(HOST_BALANCER_OOB_CHECK);
+ if (null != oobProperty)

+ String poolRecheckProperty = conf.getConfiguration().get(HOST_BALANCER_POOL_RECHECK_KEY);
+ if (null != poolRecheckProperty)

+ String ipBased = conf.getConfiguration().get(HOST_BALANCER_REGEX_USING_IPS);
+ if (null != ipBased)

+ }
+
+ public Map<String,String> getTableIdToTableName()

+
+ public Map<String,Pattern> getPoolNameToRegexPattern()

+
+ public long getOobCheckMillis()

+
+ public long getPoolRecheckMillis()

+
+ public boolean isIpBasedRegex()

+
+ @Override
+ public void init(ServerConfiguration conf)

+
+ @Override
+ public void getAssignments(SortedMap<TServerInstance,TabletServerStatus> current, Map<KeyExtent,TServerInstance> unassigned,
+ Map<KeyExtent,TServerInstance> assignments) {
+
+ if ((System.currentTimeMillis() - this.lastOOBCheck) > this.oobCheckMillis) {
+ // Check to see if a tablet is assigned outside the bounds of the pool. If so, migrate it.
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ for (String assignedPool : getPoolNamesForHost(e.getKey().host())) {
+ for (String pool : poolNameToRegexPattern.keySet()) {
+ if (assignedPool.equals(pool) || pool.equals(DEFAULT_POOL))

+ String tid = getTableOperations().tableIdMap().get(pool);
+ try {
+ List<TabletStats> outOfBoundsTablets = getOnlineTabletsForTable(e.getKey(), tid);
+ for (TabletStats ts : outOfBoundsTablets) {
+ LOG.info("Tablet {} is currently outside the bounds of the regex, reassigning", ts.toString());
+ unassigned.put(new KeyExtent(ts.getExtent()), e.getKey());
+ }
+ } catch (TException e1) {
+ LOG.error("Error in OOB check getting tablets for table {} from server {}", tid, e.getKey().host(), e);
+ }
+ }
+ }
+ }
+ this.oobCheckMillis = System.currentTimeMillis();
+ }
+
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = splitCurrentByRegex(current);
+ // separate the unassigned into tables
+ Map<String,Map<KeyExtent,TServerInstance>> groupedUnassigned = new HashMap<String,Map<KeyExtent,TServerInstance>>();
+ for (Entry<KeyExtent,TServerInstance> e : unassigned.entrySet()) {
+ Map<KeyExtent,TServerInstance> tableUnassigned = groupedUnassigned.get(e.getKey().getTableId());
+ if (tableUnassigned == null)

+ tableUnassigned.put(e.getKey(), e.getValue());
+ }
+ // Send a view of the current servers to the tables tablet balancer
+ for (Entry<String,Map<KeyExtent,TServerInstance>> e : groupedUnassigned.entrySet()) {
+ Map<KeyExtent,TServerInstance> newAssignments = new HashMap<KeyExtent,TServerInstance>();
+ String tableName = tableIdToTableName.get(e.getKey());
+ String poolName = getPoolNameForTable(tableName);
+ SortedMap<TServerInstance,TabletServerStatus> currentView = pools.get(poolName);
+ getBalancerForTable(e.getKey()).getAssignments(currentView, e.getValue(), newAssignments);
— End diff –

I think I have addressed this now, see above lines 263-278

Github user dlmarion commented on the pull request:

https://github.com/apache/accumulo/pull/83#issuecomment-208920076

Update PR based on changes from actually testing this on a test cluster. Any final comments?

Github user joshelser commented on the pull request:

https://github.com/apache/accumulo/pull/83#issuecomment-208921993

> Any final comments?

I'll take a look again since you asked

Github user joshelser commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r59381530

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
— End diff –

Nit: this is more about how it is implemented than what it does. "This balancer creates groups of tabletservers, using user-provided regular expressions over the tabletservers' hostnames, and then delegates to another TableBalancer for each group of tabletserver" (or something).

Github user joshelser commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r59382019

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. All tablet servers that do not match a regex are grouped into a default pool.<br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private volatile long lastOOBCheck = System.currentTimeMillis();
+ private volatile long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default pool.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
— End diff –

This is probably fine when the defaults are on the order of minutes, but could be problematic if someone reduces the period. Maybe worthy of a comment to state the expectation that `poolRecheckMillis` is commonly order-minutes.

Github user joshelser commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r59383136

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. All tablet servers that do not match a regex are grouped into a default pool.<br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private volatile long lastOOBCheck = System.currentTimeMillis();
+ private volatile long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default pool.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ LOG.debug("Performing pool recheck - regrouping tablet servers into pools based on regex");
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ List<String> poolNames = getPoolNamesForHost(e.getKey().host());
+ for (String pool : poolNames) {
+ SortedMap<TServerInstance,TabletServerStatus> np = newPools.get(pool);
+ if (null == np)

+ np.put(e.getKey(), e.getValue());
+ }
+ }
+ pools = newPools;
+ this.lastPoolRecheck = System.currentTimeMillis();
+ }
+ return pools;
+ }
+
+ /**
+ * Matches host against the regexes and returns the matching pool names
+ *
+ * @param host
+ * tablet server host
+ * @return pool names, will return default pool if host matches more no regex
+ */
+ protected List<String> getPoolNamesForHost(String host) {
+ String test = host;
+ if (!isIpBasedRegex) {
+ try

catch (UnknownHostException e1)

+ }
+ List<String> pools = new ArrayList<>();
+ for (Entry<String,Pattern> e : poolNameToRegexPattern.entrySet()) {
+ if (e.getValue().matcher(test).matches())

+ }
+ if (pools.size() == 0)

+ return pools;
+ }
+
+ protected String getNameFromIp(String hostIp) throws UnknownHostException

+
+ /**
+ * Matches table name against pool names, returns matching pool name or DEFAULT_POOL.
+ *
+ * @param tableName
+ * name of table
+ * @return tablet server pool name (table name or DEFAULT_POOL)
+ */
+ protected String getPoolNameForTable(String tableName) {
+ if (null == tableName)

+ return poolNameToRegexPattern.containsKey(tableName) ? tableName : DEFAULT_POOL;
+ }
+
+ /**
+ * Parse configuration and extract properties
+ *
+ * @param conf
+ * server configuration
+ */
+ protected void parseConfiguration(ServerConfiguration conf) {
+ tableIdToTableName = new HashMap<>();
+ poolNameToRegexPattern = new HashMap<>();
+ for (Entry<String,String> table : getTableOperations().tableIdMap().entrySet()) {
+ tableIdToTableName.put(table.getValue(), table.getKey());
+ Map<String,String> customProps = conf.getTableConfiguration(table.getValue()).getAllPropertiesWithPrefix(Property.TABLE_ARBITRARY_PROP_PREFIX);
+ if (null != customProps && customProps.size() > 0) {
+ for (Entry<String,String> customProp : customProps.entrySet()) {
+ if (customProp.getKey().startsWith(HOST_BALANCER_PREFIX))

+ }
+ }
+ }
+ String oobProperty = conf.getConfiguration().get(HOST_BALANCER_OOB_CHECK);
+ if (null != oobProperty)

+ String poolRecheckProperty = conf.getConfiguration().get(HOST_BALANCER_POOL_RECHECK_KEY);
+ if (null != poolRecheckProperty)

+ String ipBased = conf.getConfiguration().get(HOST_BALANCER_REGEX_USING_IPS);
+ if (null != ipBased)

+ }
+
+ public Map<String,String> getTableIdToTableName()

+
+ public Map<String,Pattern> getPoolNameToRegexPattern()

+
+ public long getOobCheckMillis()

+
+ public long getPoolRecheckMillis()

+
+ public boolean isIpBasedRegex()

+
+ @Override
+ public void init(ServerConfiguration conf)

+
+ @Override
+ public void getAssignments(SortedMap<TServerInstance,TabletServerStatus> current, Map<KeyExtent,TServerInstance> unassigned,
+ Map<KeyExtent,TServerInstance> assignments) {
+
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = splitCurrentByRegex(current);
— End diff –

This seems like it would be problematic. I would have to wait a default of 5minutes for tablets to be migrated to a new server in a pool (because the pools would not be recomputed). Did I miss something?

Github user joshelser commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r59384715

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. All tablet servers that do not match a regex are grouped into a default pool.<br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private volatile long lastOOBCheck = System.currentTimeMillis();
+ private volatile long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default pool.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ LOG.debug("Performing pool recheck - regrouping tablet servers into pools based on regex");
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ List<String> poolNames = getPoolNamesForHost(e.getKey().host());
+ for (String pool : poolNames) {
+ SortedMap<TServerInstance,TabletServerStatus> np = newPools.get(pool);
+ if (null == np)

+ np.put(e.getKey(), e.getValue());
+ }
+ }
+ pools = newPools;
+ this.lastPoolRecheck = System.currentTimeMillis();
+ }
+ return pools;
+ }
+
+ /**
+ * Matches host against the regexes and returns the matching pool names
+ *
+ * @param host
+ * tablet server host
+ * @return pool names, will return default pool if host matches more no regex
+ */
+ protected List<String> getPoolNamesForHost(String host) {
+ String test = host;
+ if (!isIpBasedRegex) {
+ try

catch (UnknownHostException e1)

+ }
+ List<String> pools = new ArrayList<>();
+ for (Entry<String,Pattern> e : poolNameToRegexPattern.entrySet()) {
+ if (e.getValue().matcher(test).matches())

+ }
+ if (pools.size() == 0)

+ return pools;
+ }
+
+ protected String getNameFromIp(String hostIp) throws UnknownHostException

+
+ /**
+ * Matches table name against pool names, returns matching pool name or DEFAULT_POOL.
+ *
+ * @param tableName
+ * name of table
+ * @return tablet server pool name (table name or DEFAULT_POOL)
+ */
+ protected String getPoolNameForTable(String tableName) {
+ if (null == tableName)

+ return poolNameToRegexPattern.containsKey(tableName) ? tableName : DEFAULT_POOL;
+ }
+
+ /**
+ * Parse configuration and extract properties
+ *
+ * @param conf
+ * server configuration
+ */
+ protected void parseConfiguration(ServerConfiguration conf) {
+ tableIdToTableName = new HashMap<>();
+ poolNameToRegexPattern = new HashMap<>();
+ for (Entry<String,String> table : getTableOperations().tableIdMap().entrySet()) {
+ tableIdToTableName.put(table.getValue(), table.getKey());
+ Map<String,String> customProps = conf.getTableConfiguration(table.getValue()).getAllPropertiesWithPrefix(Property.TABLE_ARBITRARY_PROP_PREFIX);
+ if (null != customProps && customProps.size() > 0) {
+ for (Entry<String,String> customProp : customProps.entrySet()) {
+ if (customProp.getKey().startsWith(HOST_BALANCER_PREFIX))

+ }
+ }
+ }
+ String oobProperty = conf.getConfiguration().get(HOST_BALANCER_OOB_CHECK);
+ if (null != oobProperty)

+ String poolRecheckProperty = conf.getConfiguration().get(HOST_BALANCER_POOL_RECHECK_KEY);
+ if (null != poolRecheckProperty)

+ String ipBased = conf.getConfiguration().get(HOST_BALANCER_REGEX_USING_IPS);
+ if (null != ipBased)

+ }
+
+ public Map<String,String> getTableIdToTableName()

+
+ public Map<String,Pattern> getPoolNameToRegexPattern()

+
+ public long getOobCheckMillis()

+
+ public long getPoolRecheckMillis()

+
+ public boolean isIpBasedRegex()

+
+ @Override
+ public void init(ServerConfiguration conf)

+
+ @Override
+ public void getAssignments(SortedMap<TServerInstance,TabletServerStatus> current, Map<KeyExtent,TServerInstance> unassigned,
+ Map<KeyExtent,TServerInstance> assignments) {
+
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = splitCurrentByRegex(current);
+ // group the unassigned into tables
+ Map<String,Map<KeyExtent,TServerInstance>> groupedUnassigned = new HashMap<String,Map<KeyExtent,TServerInstance>>();
+ for (Entry<KeyExtent,TServerInstance> e : unassigned.entrySet()) {
+ Map<KeyExtent,TServerInstance> tableUnassigned = groupedUnassigned.get(e.getKey().getTableId());
+ if (tableUnassigned == null)

+ tableUnassigned.put(e.getKey(), e.getValue());
+ }
+ // Send a view of the current servers to the tables tablet balancer
+ for (Entry<String,Map<KeyExtent,TServerInstance>> e : groupedUnassigned.entrySet()) {
+ Map<KeyExtent,TServerInstance> newAssignments = new HashMap<KeyExtent,TServerInstance>();
+ String tableName = tableIdToTableName.get(e.getKey());
+ String poolName = getPoolNameForTable(tableName);
+ SortedMap<TServerInstance,TabletServerStatus> currentView = pools.get(poolName);
+ if (null == currentView || currentView.size() == 0) {
+ LOG.warn("No tablet servers online for table {}, assigning within default pool", tableName);
+ currentView = pools.get(DEFAULT_POOL);
+ if (null == currentView) {
+ LOG.error("No tablet servers exist in the default pool, unable to assign tablets for table {}", tableName);
— End diff –

Findbugs caught a bug here. You're likely missing a `continue`?. When `null == currentView`, you'll fall through and call `keySet()` on it.

Github user dlmarion commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r59385418

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. All tablet servers that do not match a regex are grouped into a default pool.<br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private volatile long lastOOBCheck = System.currentTimeMillis();
+ private volatile long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default pool.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ LOG.debug("Performing pool recheck - regrouping tablet servers into pools based on regex");
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ List<String> poolNames = getPoolNamesForHost(e.getKey().host());
+ for (String pool : poolNames) {
+ SortedMap<TServerInstance,TabletServerStatus> np = newPools.get(pool);
+ if (null == np)

+ np.put(e.getKey(), e.getValue());
+ }
+ }
+ pools = newPools;
+ this.lastPoolRecheck = System.currentTimeMillis();
+ }
+ return pools;
+ }
+
+ /**
+ * Matches host against the regexes and returns the matching pool names
+ *
+ * @param host
+ * tablet server host
+ * @return pool names, will return default pool if host matches more no regex
+ */
+ protected List<String> getPoolNamesForHost(String host) {
+ String test = host;
+ if (!isIpBasedRegex) {
+ try

catch (UnknownHostException e1)

+ }
+ List<String> pools = new ArrayList<>();
+ for (Entry<String,Pattern> e : poolNameToRegexPattern.entrySet()) {
+ if (e.getValue().matcher(test).matches())

+ }
+ if (pools.size() == 0)

+ return pools;
+ }
+
+ protected String getNameFromIp(String hostIp) throws UnknownHostException

+
+ /**
+ * Matches table name against pool names, returns matching pool name or DEFAULT_POOL.
+ *
+ * @param tableName
+ * name of table
+ * @return tablet server pool name (table name or DEFAULT_POOL)
+ */
+ protected String getPoolNameForTable(String tableName) {
+ if (null == tableName)

+ return poolNameToRegexPattern.containsKey(tableName) ? tableName : DEFAULT_POOL;
+ }
+
+ /**
+ * Parse configuration and extract properties
+ *
+ * @param conf
+ * server configuration
+ */
+ protected void parseConfiguration(ServerConfiguration conf) {
+ tableIdToTableName = new HashMap<>();
+ poolNameToRegexPattern = new HashMap<>();
+ for (Entry<String,String> table : getTableOperations().tableIdMap().entrySet()) {
+ tableIdToTableName.put(table.getValue(), table.getKey());
+ Map<String,String> customProps = conf.getTableConfiguration(table.getValue()).getAllPropertiesWithPrefix(Property.TABLE_ARBITRARY_PROP_PREFIX);
+ if (null != customProps && customProps.size() > 0) {
+ for (Entry<String,String> customProp : customProps.entrySet()) {
+ if (customProp.getKey().startsWith(HOST_BALANCER_PREFIX))

+ }
+ }
+ }
+ String oobProperty = conf.getConfiguration().get(HOST_BALANCER_OOB_CHECK);
+ if (null != oobProperty)

+ String poolRecheckProperty = conf.getConfiguration().get(HOST_BALANCER_POOL_RECHECK_KEY);
+ if (null != poolRecheckProperty)

+ String ipBased = conf.getConfiguration().get(HOST_BALANCER_REGEX_USING_IPS);
+ if (null != ipBased)

+ }
+
+ public Map<String,String> getTableIdToTableName()

+
+ public Map<String,Pattern> getPoolNameToRegexPattern()

+
+ public long getOobCheckMillis()

+
+ public long getPoolRecheckMillis()

+
+ public boolean isIpBasedRegex()

+
+ @Override
+ public void init(ServerConfiguration conf)

+
+ @Override
+ public void getAssignments(SortedMap<TServerInstance,TabletServerStatus> current, Map<KeyExtent,TServerInstance> unassigned,
+ Map<KeyExtent,TServerInstance> assignments) {
+
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = splitCurrentByRegex(current);
+ // group the unassigned into tables
+ Map<String,Map<KeyExtent,TServerInstance>> groupedUnassigned = new HashMap<String,Map<KeyExtent,TServerInstance>>();
+ for (Entry<KeyExtent,TServerInstance> e : unassigned.entrySet()) {
+ Map<KeyExtent,TServerInstance> tableUnassigned = groupedUnassigned.get(e.getKey().getTableId());
+ if (tableUnassigned == null)

+ tableUnassigned.put(e.getKey(), e.getValue());
+ }
+ // Send a view of the current servers to the tables tablet balancer
+ for (Entry<String,Map<KeyExtent,TServerInstance>> e : groupedUnassigned.entrySet()) {
+ Map<KeyExtent,TServerInstance> newAssignments = new HashMap<KeyExtent,TServerInstance>();
+ String tableName = tableIdToTableName.get(e.getKey());
+ String poolName = getPoolNameForTable(tableName);
+ SortedMap<TServerInstance,TabletServerStatus> currentView = pools.get(poolName);
+ if (null == currentView || currentView.size() == 0) {
+ LOG.warn("No tablet servers online for table {}, assigning within default pool", tableName);
+ currentView = pools.get(DEFAULT_POOL);
+ if (null == currentView) {
+ LOG.error("No tablet servers exist in the default pool, unable to assign tablets for table {}", tableName);
— End diff –

Good catch. I didn't hit it in my testing. I'll fix.

Github user dlmarion commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r59388944

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. All tablet servers that do not match a regex are grouped into a default pool.<br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private volatile long lastOOBCheck = System.currentTimeMillis();
+ private volatile long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default pool.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ LOG.debug("Performing pool recheck - regrouping tablet servers into pools based on regex");
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ List<String> poolNames = getPoolNamesForHost(e.getKey().host());
+ for (String pool : poolNames) {
+ SortedMap<TServerInstance,TabletServerStatus> np = newPools.get(pool);
+ if (null == np)

+ np.put(e.getKey(), e.getValue());
+ }
+ }
+ pools = newPools;
+ this.lastPoolRecheck = System.currentTimeMillis();
+ }
+ return pools;
+ }
+
+ /**
+ * Matches host against the regexes and returns the matching pool names
+ *
+ * @param host
+ * tablet server host
+ * @return pool names, will return default pool if host matches more no regex
+ */
+ protected List<String> getPoolNamesForHost(String host) {
+ String test = host;
+ if (!isIpBasedRegex) {
+ try

catch (UnknownHostException e1)

+ }
+ List<String> pools = new ArrayList<>();
+ for (Entry<String,Pattern> e : poolNameToRegexPattern.entrySet()) {
+ if (e.getValue().matcher(test).matches())

+ }
+ if (pools.size() == 0)

+ return pools;
+ }
+
+ protected String getNameFromIp(String hostIp) throws UnknownHostException

+
+ /**
+ * Matches table name against pool names, returns matching pool name or DEFAULT_POOL.
+ *
+ * @param tableName
+ * name of table
+ * @return tablet server pool name (table name or DEFAULT_POOL)
+ */
+ protected String getPoolNameForTable(String tableName) {
+ if (null == tableName)

+ return poolNameToRegexPattern.containsKey(tableName) ? tableName : DEFAULT_POOL;
+ }
+
+ /**
+ * Parse configuration and extract properties
+ *
+ * @param conf
+ * server configuration
+ */
+ protected void parseConfiguration(ServerConfiguration conf) {
+ tableIdToTableName = new HashMap<>();
+ poolNameToRegexPattern = new HashMap<>();
+ for (Entry<String,String> table : getTableOperations().tableIdMap().entrySet()) {
+ tableIdToTableName.put(table.getValue(), table.getKey());
+ Map<String,String> customProps = conf.getTableConfiguration(table.getValue()).getAllPropertiesWithPrefix(Property.TABLE_ARBITRARY_PROP_PREFIX);
+ if (null != customProps && customProps.size() > 0) {
+ for (Entry<String,String> customProp : customProps.entrySet()) {
+ if (customProp.getKey().startsWith(HOST_BALANCER_PREFIX))

+ }
+ }
+ }
+ String oobProperty = conf.getConfiguration().get(HOST_BALANCER_OOB_CHECK);
+ if (null != oobProperty)

+ String poolRecheckProperty = conf.getConfiguration().get(HOST_BALANCER_POOL_RECHECK_KEY);
+ if (null != poolRecheckProperty)

+ String ipBased = conf.getConfiguration().get(HOST_BALANCER_REGEX_USING_IPS);
+ if (null != ipBased)

+ }
+
+ public Map<String,String> getTableIdToTableName()

+
+ public Map<String,Pattern> getPoolNameToRegexPattern()

+
+ public long getOobCheckMillis()

+
+ public long getPoolRecheckMillis()

+
+ public boolean isIpBasedRegex()

+
+ @Override
+ public void init(ServerConfiguration conf)

+
+ @Override
+ public void getAssignments(SortedMap<TServerInstance,TabletServerStatus> current, Map<KeyExtent,TServerInstance> unassigned,
+ Map<KeyExtent,TServerInstance> assignments) {
+
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = splitCurrentByRegex(current);
— End diff –

No, thats correct. You can set the pool recheck time to a low value, 1 to 2 minutes would seem reasonable, but then you pay the price of the recalculation.

Github user joshelser commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r59391317

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. All tablet servers that do not match a regex are grouped into a default pool.<br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private volatile long lastOOBCheck = System.currentTimeMillis();
+ private volatile long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default pool.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ LOG.debug("Performing pool recheck - regrouping tablet servers into pools based on regex");
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ List<String> poolNames = getPoolNamesForHost(e.getKey().host());
+ for (String pool : poolNames) {
+ SortedMap<TServerInstance,TabletServerStatus> np = newPools.get(pool);
+ if (null == np)

+ np.put(e.getKey(), e.getValue());
+ }
+ }
+ pools = newPools;
+ this.lastPoolRecheck = System.currentTimeMillis();
+ }
+ return pools;
+ }
+
+ /**
+ * Matches host against the regexes and returns the matching pool names
+ *
+ * @param host
+ * tablet server host
+ * @return pool names, will return default pool if host matches more no regex
+ */
+ protected List<String> getPoolNamesForHost(String host) {
+ String test = host;
+ if (!isIpBasedRegex) {
+ try

catch (UnknownHostException e1)

+ }
+ List<String> pools = new ArrayList<>();
+ for (Entry<String,Pattern> e : poolNameToRegexPattern.entrySet()) {
+ if (e.getValue().matcher(test).matches())

+ }
+ if (pools.size() == 0)

+ return pools;
+ }
+
+ protected String getNameFromIp(String hostIp) throws UnknownHostException

+
+ /**
+ * Matches table name against pool names, returns matching pool name or DEFAULT_POOL.
+ *
+ * @param tableName
+ * name of table
+ * @return tablet server pool name (table name or DEFAULT_POOL)
+ */
+ protected String getPoolNameForTable(String tableName) {
+ if (null == tableName)

+ return poolNameToRegexPattern.containsKey(tableName) ? tableName : DEFAULT_POOL;
+ }
+
+ /**
+ * Parse configuration and extract properties
+ *
+ * @param conf
+ * server configuration
+ */
+ protected void parseConfiguration(ServerConfiguration conf) {
+ tableIdToTableName = new HashMap<>();
+ poolNameToRegexPattern = new HashMap<>();
+ for (Entry<String,String> table : getTableOperations().tableIdMap().entrySet()) {
+ tableIdToTableName.put(table.getValue(), table.getKey());
+ Map<String,String> customProps = conf.getTableConfiguration(table.getValue()).getAllPropertiesWithPrefix(Property.TABLE_ARBITRARY_PROP_PREFIX);
+ if (null != customProps && customProps.size() > 0) {
+ for (Entry<String,String> customProp : customProps.entrySet()) {
+ if (customProp.getKey().startsWith(HOST_BALANCER_PREFIX))

+ }
+ }
+ }
+ String oobProperty = conf.getConfiguration().get(HOST_BALANCER_OOB_CHECK);
+ if (null != oobProperty)

+ String poolRecheckProperty = conf.getConfiguration().get(HOST_BALANCER_POOL_RECHECK_KEY);
+ if (null != poolRecheckProperty)

+ String ipBased = conf.getConfiguration().get(HOST_BALANCER_REGEX_USING_IPS);
+ if (null != ipBased)

+ }
+
+ public Map<String,String> getTableIdToTableName()

+
+ public Map<String,Pattern> getPoolNameToRegexPattern()

+
+ public long getOobCheckMillis()

+
+ public long getPoolRecheckMillis()

+
+ public boolean isIpBasedRegex()

+
+ @Override
+ public void init(ServerConfiguration conf)

+
+ @Override
+ public void getAssignments(SortedMap<TServerInstance,TabletServerStatus> current, Map<KeyExtent,TServerInstance> unassigned,
+ Map<KeyExtent,TServerInstance> assignments) {
+
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = splitCurrentByRegex(current);
— End diff –

5 minutes seems high as a default then. Considering the average person will have 10's of servers, maybe 100's, waiting 5 minutes by default seems extraordinary. Hypothetically, grouping 100 hostnames every minute seems fine to me (esp in the master). Maybe the scope of the feature itself isn't a worry for people with 1s-10s of servers? WDYT?

Github user dlmarion commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r59392425

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. All tablet servers that do not match a regex are grouped into a default pool.<br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private volatile long lastOOBCheck = System.currentTimeMillis();
+ private volatile long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default pool.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ LOG.debug("Performing pool recheck - regrouping tablet servers into pools based on regex");
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ List<String> poolNames = getPoolNamesForHost(e.getKey().host());
+ for (String pool : poolNames) {
+ SortedMap<TServerInstance,TabletServerStatus> np = newPools.get(pool);
+ if (null == np)

+ np.put(e.getKey(), e.getValue());
+ }
+ }
+ pools = newPools;
+ this.lastPoolRecheck = System.currentTimeMillis();
+ }
+ return pools;
+ }
+
+ /**
+ * Matches host against the regexes and returns the matching pool names
+ *
+ * @param host
+ * tablet server host
+ * @return pool names, will return default pool if host matches more no regex
+ */
+ protected List<String> getPoolNamesForHost(String host) {
+ String test = host;
+ if (!isIpBasedRegex) {
+ try

catch (UnknownHostException e1)

+ }
+ List<String> pools = new ArrayList<>();
+ for (Entry<String,Pattern> e : poolNameToRegexPattern.entrySet()) {
+ if (e.getValue().matcher(test).matches())

+ }
+ if (pools.size() == 0)

+ return pools;
+ }
+
+ protected String getNameFromIp(String hostIp) throws UnknownHostException

+
+ /**
+ * Matches table name against pool names, returns matching pool name or DEFAULT_POOL.
+ *
+ * @param tableName
+ * name of table
+ * @return tablet server pool name (table name or DEFAULT_POOL)
+ */
+ protected String getPoolNameForTable(String tableName) {
+ if (null == tableName)

+ return poolNameToRegexPattern.containsKey(tableName) ? tableName : DEFAULT_POOL;
+ }
+
+ /**
+ * Parse configuration and extract properties
+ *
+ * @param conf
+ * server configuration
+ */
+ protected void parseConfiguration(ServerConfiguration conf) {
+ tableIdToTableName = new HashMap<>();
+ poolNameToRegexPattern = new HashMap<>();
+ for (Entry<String,String> table : getTableOperations().tableIdMap().entrySet()) {
+ tableIdToTableName.put(table.getValue(), table.getKey());
+ Map<String,String> customProps = conf.getTableConfiguration(table.getValue()).getAllPropertiesWithPrefix(Property.TABLE_ARBITRARY_PROP_PREFIX);
+ if (null != customProps && customProps.size() > 0) {
+ for (Entry<String,String> customProp : customProps.entrySet()) {
+ if (customProp.getKey().startsWith(HOST_BALANCER_PREFIX))

+ }
+ }
+ }
+ String oobProperty = conf.getConfiguration().get(HOST_BALANCER_OOB_CHECK);
+ if (null != oobProperty)

+ String poolRecheckProperty = conf.getConfiguration().get(HOST_BALANCER_POOL_RECHECK_KEY);
+ if (null != poolRecheckProperty)

+ String ipBased = conf.getConfiguration().get(HOST_BALANCER_REGEX_USING_IPS);
+ if (null != ipBased)

+ }
+
+ public Map<String,String> getTableIdToTableName()

+
+ public Map<String,Pattern> getPoolNameToRegexPattern()

+
+ public long getOobCheckMillis()

+
+ public long getPoolRecheckMillis()

+
+ public boolean isIpBasedRegex()

+
+ @Override
+ public void init(ServerConfiguration conf)

+
+ @Override
+ public void getAssignments(SortedMap<TServerInstance,TabletServerStatus> current, Map<KeyExtent,TServerInstance> unassigned,
+ Map<KeyExtent,TServerInstance> assignments) {
+
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = splitCurrentByRegex(current);
— End diff –

I'm fine with lowering it. I'm about to push another commit that addresses the other comments as well.

Github user joshelser commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r59392771

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. All tablet servers that do not match a regex are grouped into a default pool.<br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private volatile long lastOOBCheck = System.currentTimeMillis();
+ private volatile long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default pool.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ LOG.debug("Performing pool recheck - regrouping tablet servers into pools based on regex");
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ List<String> poolNames = getPoolNamesForHost(e.getKey().host());
+ for (String pool : poolNames) {
+ SortedMap<TServerInstance,TabletServerStatus> np = newPools.get(pool);
+ if (null == np)

+ np.put(e.getKey(), e.getValue());
+ }
+ }
+ pools = newPools;
+ this.lastPoolRecheck = System.currentTimeMillis();
+ }
+ return pools;
+ }
+
+ /**
+ * Matches host against the regexes and returns the matching pool names
+ *
+ * @param host
+ * tablet server host
+ * @return pool names, will return default pool if host matches more no regex
+ */
+ protected List<String> getPoolNamesForHost(String host) {
+ String test = host;
+ if (!isIpBasedRegex) {
+ try

catch (UnknownHostException e1)

+ }
+ List<String> pools = new ArrayList<>();
+ for (Entry<String,Pattern> e : poolNameToRegexPattern.entrySet()) {
+ if (e.getValue().matcher(test).matches())

+ }
+ if (pools.size() == 0)

+ return pools;
+ }
+
+ protected String getNameFromIp(String hostIp) throws UnknownHostException

+
+ /**
+ * Matches table name against pool names, returns matching pool name or DEFAULT_POOL.
+ *
+ * @param tableName
+ * name of table
+ * @return tablet server pool name (table name or DEFAULT_POOL)
+ */
+ protected String getPoolNameForTable(String tableName) {
+ if (null == tableName)

+ return poolNameToRegexPattern.containsKey(tableName) ? tableName : DEFAULT_POOL;
+ }
+
+ /**
+ * Parse configuration and extract properties
+ *
+ * @param conf
+ * server configuration
+ */
+ protected void parseConfiguration(ServerConfiguration conf) {
+ tableIdToTableName = new HashMap<>();
+ poolNameToRegexPattern = new HashMap<>();
+ for (Entry<String,String> table : getTableOperations().tableIdMap().entrySet()) {
+ tableIdToTableName.put(table.getValue(), table.getKey());
+ Map<String,String> customProps = conf.getTableConfiguration(table.getValue()).getAllPropertiesWithPrefix(Property.TABLE_ARBITRARY_PROP_PREFIX);
+ if (null != customProps && customProps.size() > 0) {
+ for (Entry<String,String> customProp : customProps.entrySet()) {
+ if (customProp.getKey().startsWith(HOST_BALANCER_PREFIX))

+ }
+ }
+ }
+ String oobProperty = conf.getConfiguration().get(HOST_BALANCER_OOB_CHECK);
+ if (null != oobProperty)

+ String poolRecheckProperty = conf.getConfiguration().get(HOST_BALANCER_POOL_RECHECK_KEY);
+ if (null != poolRecheckProperty)

+ String ipBased = conf.getConfiguration().get(HOST_BALANCER_REGEX_USING_IPS);
+ if (null != ipBased)

+ }
+
+ public Map<String,String> getTableIdToTableName()

+
+ public Map<String,Pattern> getPoolNameToRegexPattern()

+
+ public long getOobCheckMillis()

+
+ public long getPoolRecheckMillis()

+
+ public boolean isIpBasedRegex()

+
+ @Override
+ public void init(ServerConfiguration conf)

+
+ @Override
+ public void getAssignments(SortedMap<TServerInstance,TabletServerStatus> current, Map<KeyExtent,TServerInstance> unassigned,
+ Map<KeyExtent,TServerInstance> assignments) {
+
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = splitCurrentByRegex(current);
— End diff –

Cool, thanks Dave.

Github user dlmarion commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r59392931

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. All tablet servers that do not match a regex are grouped into a default pool.<br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private volatile long lastOOBCheck = System.currentTimeMillis();
+ private volatile long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default pool.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
— End diff –

Added a comment to the method javadoc.

Github user joshelser commented on the pull request:

https://github.com/apache/accumulo/pull/83#issuecomment-208953363

I think I'm good with this. FWIW, I did ask Eric if he would have any time to take a look, and he mentioned that, if he does, it wouldn't be until after normal working hours. Up to you if you want to wait for him to give a pass over it, of course.

Github user dlmarion commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r59393002

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. All tablet servers that do not match a regex are grouped into a default pool.<br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private volatile long lastOOBCheck = System.currentTimeMillis();
+ private volatile long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default pool.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ LOG.debug("Performing pool recheck - regrouping tablet servers into pools based on regex");
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ List<String> poolNames = getPoolNamesForHost(e.getKey().host());
+ for (String pool : poolNames) {
+ SortedMap<TServerInstance,TabletServerStatus> np = newPools.get(pool);
+ if (null == np)

+ np.put(e.getKey(), e.getValue());
+ }
+ }
+ pools = newPools;
+ this.lastPoolRecheck = System.currentTimeMillis();
+ }
+ return pools;
+ }
+
+ /**
+ * Matches host against the regexes and returns the matching pool names
+ *
+ * @param host
+ * tablet server host
+ * @return pool names, will return default pool if host matches more no regex
+ */
+ protected List<String> getPoolNamesForHost(String host) {
+ String test = host;
+ if (!isIpBasedRegex) {
+ try

catch (UnknownHostException e1)

+ }
+ List<String> pools = new ArrayList<>();
+ for (Entry<String,Pattern> e : poolNameToRegexPattern.entrySet()) {
+ if (e.getValue().matcher(test).matches())

+ }
+ if (pools.size() == 0)

+ return pools;
+ }
+
+ protected String getNameFromIp(String hostIp) throws UnknownHostException

+
+ /**
+ * Matches table name against pool names, returns matching pool name or DEFAULT_POOL.
+ *
+ * @param tableName
+ * name of table
+ * @return tablet server pool name (table name or DEFAULT_POOL)
+ */
+ protected String getPoolNameForTable(String tableName) {
+ if (null == tableName)

+ return poolNameToRegexPattern.containsKey(tableName) ? tableName : DEFAULT_POOL;
+ }
+
+ /**
+ * Parse configuration and extract properties
+ *
+ * @param conf
+ * server configuration
+ */
+ protected void parseConfiguration(ServerConfiguration conf) {
+ tableIdToTableName = new HashMap<>();
+ poolNameToRegexPattern = new HashMap<>();
+ for (Entry<String,String> table : getTableOperations().tableIdMap().entrySet()) {
+ tableIdToTableName.put(table.getValue(), table.getKey());
+ Map<String,String> customProps = conf.getTableConfiguration(table.getValue()).getAllPropertiesWithPrefix(Property.TABLE_ARBITRARY_PROP_PREFIX);
+ if (null != customProps && customProps.size() > 0) {
+ for (Entry<String,String> customProp : customProps.entrySet()) {
+ if (customProp.getKey().startsWith(HOST_BALANCER_PREFIX))

+ }
+ }
+ }
+ String oobProperty = conf.getConfiguration().get(HOST_BALANCER_OOB_CHECK);
+ if (null != oobProperty)

+ String poolRecheckProperty = conf.getConfiguration().get(HOST_BALANCER_POOL_RECHECK_KEY);
+ if (null != poolRecheckProperty)

+ String ipBased = conf.getConfiguration().get(HOST_BALANCER_REGEX_USING_IPS);
+ if (null != ipBased)

+ }
+
+ public Map<String,String> getTableIdToTableName()

+
+ public Map<String,Pattern> getPoolNameToRegexPattern()

+
+ public long getOobCheckMillis()

+
+ public long getPoolRecheckMillis()

+
+ public boolean isIpBasedRegex()

+
+ @Override
+ public void init(ServerConfiguration conf)

+
+ @Override
+ public void getAssignments(SortedMap<TServerInstance,TabletServerStatus> current, Map<KeyExtent,TServerInstance> unassigned,
+ Map<KeyExtent,TServerInstance> assignments) {
+
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = splitCurrentByRegex(current);
— End diff –

Lowered to 1m

Github user dlmarion commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r59393036

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. All tablet servers that do not match a regex are grouped into a default pool.<br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private volatile long lastOOBCheck = System.currentTimeMillis();
+ private volatile long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default pool.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ LOG.debug("Performing pool recheck - regrouping tablet servers into pools based on regex");
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ List<String> poolNames = getPoolNamesForHost(e.getKey().host());
+ for (String pool : poolNames) {
+ SortedMap<TServerInstance,TabletServerStatus> np = newPools.get(pool);
+ if (null == np)

+ np.put(e.getKey(), e.getValue());
+ }
+ }
+ pools = newPools;
+ this.lastPoolRecheck = System.currentTimeMillis();
+ }
+ return pools;
+ }
+
+ /**
+ * Matches host against the regexes and returns the matching pool names
+ *
+ * @param host
+ * tablet server host
+ * @return pool names, will return default pool if host matches more no regex
+ */
+ protected List<String> getPoolNamesForHost(String host) {
+ String test = host;
+ if (!isIpBasedRegex) {
+ try

catch (UnknownHostException e1)

+ }
+ List<String> pools = new ArrayList<>();
+ for (Entry<String,Pattern> e : poolNameToRegexPattern.entrySet()) {
+ if (e.getValue().matcher(test).matches())

+ }
+ if (pools.size() == 0)

+ return pools;
+ }
+
+ protected String getNameFromIp(String hostIp) throws UnknownHostException

+
+ /**
+ * Matches table name against pool names, returns matching pool name or DEFAULT_POOL.
+ *
+ * @param tableName
+ * name of table
+ * @return tablet server pool name (table name or DEFAULT_POOL)
+ */
+ protected String getPoolNameForTable(String tableName) {
+ if (null == tableName)

+ return poolNameToRegexPattern.containsKey(tableName) ? tableName : DEFAULT_POOL;
+ }
+
+ /**
+ * Parse configuration and extract properties
+ *
+ * @param conf
+ * server configuration
+ */
+ protected void parseConfiguration(ServerConfiguration conf) {
+ tableIdToTableName = new HashMap<>();
+ poolNameToRegexPattern = new HashMap<>();
+ for (Entry<String,String> table : getTableOperations().tableIdMap().entrySet()) {
+ tableIdToTableName.put(table.getValue(), table.getKey());
+ Map<String,String> customProps = conf.getTableConfiguration(table.getValue()).getAllPropertiesWithPrefix(Property.TABLE_ARBITRARY_PROP_PREFIX);
+ if (null != customProps && customProps.size() > 0) {
+ for (Entry<String,String> customProp : customProps.entrySet()) {
+ if (customProp.getKey().startsWith(HOST_BALANCER_PREFIX))

+ }
+ }
+ }
+ String oobProperty = conf.getConfiguration().get(HOST_BALANCER_OOB_CHECK);
+ if (null != oobProperty)

+ String poolRecheckProperty = conf.getConfiguration().get(HOST_BALANCER_POOL_RECHECK_KEY);
+ if (null != poolRecheckProperty)

+ String ipBased = conf.getConfiguration().get(HOST_BALANCER_REGEX_USING_IPS);
+ if (null != ipBased)

+ }
+
+ public Map<String,String> getTableIdToTableName()

+
+ public Map<String,Pattern> getPoolNameToRegexPattern()

+
+ public long getOobCheckMillis()

+
+ public long getPoolRecheckMillis()

+
+ public boolean isIpBasedRegex()

+
+ @Override
+ public void init(ServerConfiguration conf)

+
+ @Override
+ public void getAssignments(SortedMap<TServerInstance,TabletServerStatus> current, Map<KeyExtent,TServerInstance> unassigned,
+ Map<KeyExtent,TServerInstance> assignments) {
+
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = splitCurrentByRegex(current);
+ // group the unassigned into tables
+ Map<String,Map<KeyExtent,TServerInstance>> groupedUnassigned = new HashMap<String,Map<KeyExtent,TServerInstance>>();
+ for (Entry<KeyExtent,TServerInstance> e : unassigned.entrySet()) {
+ Map<KeyExtent,TServerInstance> tableUnassigned = groupedUnassigned.get(e.getKey().getTableId());
+ if (tableUnassigned == null)

+ tableUnassigned.put(e.getKey(), e.getValue());
+ }
+ // Send a view of the current servers to the tables tablet balancer
+ for (Entry<String,Map<KeyExtent,TServerInstance>> e : groupedUnassigned.entrySet()) {
+ Map<KeyExtent,TServerInstance> newAssignments = new HashMap<KeyExtent,TServerInstance>();
+ String tableName = tableIdToTableName.get(e.getKey());
+ String poolName = getPoolNameForTable(tableName);
+ SortedMap<TServerInstance,TabletServerStatus> currentView = pools.get(poolName);
+ if (null == currentView || currentView.size() == 0) {
+ LOG.warn("No tablet servers online for table {}, assigning within default pool", tableName);
+ currentView = pools.get(DEFAULT_POOL);
+ if (null == currentView) {
+ LOG.error("No tablet servers exist in the default pool, unable to assign tablets for table {}", tableName);
— End diff –

Fixed.

Github user dlmarion commented on a diff in the pull request:

https://github.com/apache/accumulo/pull/83#discussion_r59393062

— Diff: server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java —
@@ -0,0 +1,314 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.master.balancer;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.regex.Pattern;
+
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.server.conf.ServerConfiguration;
+import org.apache.accumulo.server.master.state.TServerInstance;
+import org.apache.accumulo.server.master.state.TabletMigration;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This balancer will create pools of tablet servers by grouping tablet servers that match a regex into the same pool and calling the balancer set on the table
+ * to balance within the set of matching tablet servers. <br>
+ * Regex properties for this balancer are specified as:<br>
+ * <b>table.custom.balancer.host.regex.<tablename>=<regex></b><br>
+ * Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration.
+ * If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while
+ * the tablet servers are up. To change the out of bounds check time period, set the following property:<br>
+ * <b>table.custom.balancer.host.regex.oob.period=5m</b><br>
+ * Periodically (default 5m) this balancer will regroup the set of current tablet servers into pools based on regexes applied to the tserver host names. This
+ * would cover the case of tservers dying or coming online. To change the host pool check time period, set the following property: <br>
+ * <b>table.custom.balancer.host.regex.pool.check=5m</b><br>
+ * Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server
+ * IP address, then set the following property:<br>
+ * <b>table.custom.balancer.host.regex.is.ip=true</b>
+ *
+ */
+public class HostRegexTableLoadBalancer extends TableLoadBalancer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(HostRegexTableLoadBalancer.class);
+ public static final String HOST_BALANCER_PREFIX = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.";
+ public static final String HOST_BALANCER_OOB_CHECK = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.oob.period";
+ private static final String HOST_BALANCER_OOB_DEFAULT = "5m";
+ public static final String HOST_BALANCER_POOL_RECHECK_KEY = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.pool.check";
+ private static final String HOST_BALANCER_POOL_RECHECK_DEFAULT = "5m";
+ public static final String HOST_BALANCER_REGEX_USING_IPS = Property.TABLE_ARBITRARY_PROP_PREFIX.getKey() + "balancer.host.regex.is.ip";
+ protected static final String DEFAULT_POOL = "HostTableLoadBalancer.ALL";
+
+ protected long oobCheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_OOB_DEFAULT);
+ protected long poolRecheckMillis = AccumuloConfiguration.getTimeInMillis(HOST_BALANCER_POOL_RECHECK_DEFAULT);
+
+ private Map<String,String> tableIdToTableName = null;
+ private Map<String,Pattern> poolNameToRegexPattern = null;
+ private long lastOOBCheck = System.currentTimeMillis();
+ private long lastPoolRecheck = 0;
+ private boolean isIpBasedRegex = false;
+ private Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+
+ /**
+ * Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default ppol.
+ *
+ * @param current
+ * map of current tservers
+ * @return current servers grouped by pool name, if not a match it is put into a default pool.
+ */
+ protected synchronized Map<String,SortedMap<TServerInstance,TabletServerStatus>> splitCurrentByRegex(SortedMap<TServerInstance,TabletServerStatus> current) {
+ if ((System.currentTimeMillis() - lastPoolRecheck) > poolRecheckMillis) {
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> newPools = new HashMap<String,SortedMap<TServerInstance,TabletServerStatus>>();
+ for (Entry<TServerInstance,TabletServerStatus> e : current.entrySet()) {
+ String tableName = getPoolNameForHost(e.getKey().host());
+ if (!newPools.containsKey(tableName))

+ newPools.get(tableName).put(e.getKey(), e.getValue());
+ }
+ // Ensure that no host is in more than one pool
+ // TODO: I'm not sure that I need to check for disjoint as the call to getPoolNameForHost checks for more than one match
+ boolean error = false;
+ for (SortedMap<TServerInstance,TabletServerStatus> s1 : newPools.values()) {
+ for (SortedMap<TServerInstance,TabletServerStatus> s2 : newPools.values()) {
+ if (s1 == s2)

+ String tid = getTableOperations().tableIdMap().get(pool);
+ try {
+ List<TabletStats> outOfBoundsTablets = getOnlineTabletsForTable(e.getKey(), tid);
+ for (TabletStats ts : outOfBoundsTablets) {
+ LOG.info("Tablet {} is currently outside the bounds of the regex, reassigning", ts.toString());
+ unassigned.put(new KeyExtent(ts.getExtent()), e.getKey());
+ }
+ } catch (TException e1) {
+ LOG.error("Error in OOB check getting tablets for table {} from server {}", tid, e.getKey().host(), e);
+ }
+ }
+ }
+ this.oobCheckMillis = System.currentTimeMillis();
+ }
+
+ Map<String,SortedMap<TServerInstance,TabletServerStatus>> pools = splitCurrentByRegex(current);
+ // separate the unassigned into tables
+ Map<String,Map<KeyExtent,TServerInstance>> groupedUnassigned = new HashMap<String,Map<KeyExtent,TServerInstance>>();
+ for (Entry<KeyExtent,TServerInstance> e : unassigned.entrySet()) {
+ Map<KeyExtent,TServerInstance> tableUnassigned = groupedUnassigned.get(e.getKey().getTableId());
+ if (tableUnassigned == null)

+ tableUnassigned.put(e.getKey(), e.getValue());
+ }
+ // Send a view of the current servers to the tables tablet balancer
+ for (Entry<String,Map<KeyExtent,TServerInstance>> e : groupedUnassigned.entrySet())

+ }
+
+ @Override
+ public long balance(SortedMap<TServerInstance,TabletServerStatus> current, Set<KeyExtent> migrations, List<TabletMigration> migrationsOut) {
+ long minBalanceTime = 5 * 1000;
+ // Iterate over the tables and balance each of them
+ TableOperations t = getTableOperations();
+ if (t == null)
— End diff –

Fixed in other location

Github user dlmarion commented on the pull request:

https://github.com/apache/accumulo/pull/83#issuecomment-208955560

I would normally wait, but I have tested this and its working and other have laid eyes on it and commented. I plan on manually applying this to 1.6 and merging up. I think I will do that and commit it and address anything he raises in another issue.

Github user dlmarion closed the pull request at:

https://github.com/apache/accumulo/pull/83