[HBASE-25665] Disable reverse DNS lookup for SASL Kerberos client connection

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha-1, 1.4.13, 2.4.1
• Fix Version/s: 3.0.0-alpha-1, 2.5.0
• Hadoop Flags: Reviewed

Release Note

New client side configuration `hbase.unsafe.client.kerberos.hostname.disable.reversedns` is added.

This configuration is advanced for experts and you shouldn't specify unless you really what is this configuration and doing.
HBase secure client using SASL Kerberos performs DNS reverse lookup to get hostname for server principal using InetAddress.getCanonicalHostName by default (false for this configuration).
If you set true for this configuration, HBase client doen't perform DNS reverse lookup for server principal and use InetAddress.getHostName which is sent by HBase cluster instead.
This helps your client application deploy under unusual network environment which DNS doesn't provide reverse lookup.
Check the description of the JIRA ticket, HBASE-25665 carefully and check that this configuration fits your environment and deployment actually before enable this configuration.

• Assignee: Shinya Yoshida
• Reporter: Shinya Yoshida
• Created: 15/Mar/21 09:19
• Updated: 30/Mar/21 15:27
• Resolved: 17/Mar/21 04:07

Description

In some unusual network environment that forward DNS lookup is supported, but revers isn't,

we can configure the HBase cluster by deploying/etc/hosts which support reverse lookup for all nodes in the cluster or hbase.unsafe.regionserver.hostname.disable.master.reversedns=true which is introduced in HBASE-18226(See also HBASE-12954, HBASE-24667).

Our network environment is also unusual and doesn't provide a reverse lookup.
Thus, we configure the HBase cluster by deploying /etc/hosts.

Assume our DNS setup is

master1.example.com A 12.34.56.1
master2.example.com A 12.34.56.2
master3.example.com A 12.34.56.3

regionserver1.example.com A 12.34.56.4
regionserver2.example.com A 12.34.56.5
regionserver3.example.com A 12.34.56.6

We deploy the following /etc/hosts for the HBase cluster nodes by our deployment system.

12.34.56.1 master1.example.com master1
12.34.56.2 master2.example.com master2
12.34.56.3 master3.example.com master3

12.34.56.4 regionserver1.example.com regionserver1
12.34.56.5 regionserver2.example.com regionserver2
12.34.56.6 regionserver3.example.com regionserver3

(We don't use hbase.unsafe.regionserver.hostname.disable.master.reversedns=true for now)

So all nodes in the cluster have stable reverse lookup for the IPs in the cluster, and the HBase cluster deployed in this way is quite stable and we can expand the cluster easily without any modification on the client-side.

Now we need to introduce Kerberos SASL secured cluster for security reasons.

We tried to construct in the same way as is, i.e. deploy /etc/hosts for the HBase cluster nodes.
However, this won't work well because the HBase client does a reverse lookup to get principal for Kerberos.
(hbase.unsafe.regionserver.hostname.disable.master.reversedns=true won't work as well as)

Thus we need to deploy /etc/hosts to all application servers, which contains all nodes of the HBase cluster to be connected.

This is quite terrible for our cluster operation and application server setup.

We, the HBase cluster manager, need to take care of application server setup and deployment.
We must provide all master and region server lists.
It's much more complicated when applications access multiple HBase clusters...
We, the HBase cluster manager, cannot expand the cluster unless the latest /etc/hosts are deployed to all application servers.
If we expand the cluster before deployment, the application is unable to connect and got an error

Assume their own Kerberos principal is their FQDN i.e. master1.example.com for example, and the cluster is aware of their FQDN.
So all clients can connect cluster nodes using the FQDN for Kerberos principal.

Could we provide an advanced unsafe option to disable DNS reverse lookup for clients using Kerberos SASL like hbase.unsafe.regionserver.hostname.disable.master.reversedns and other config?

Let's say `hbase.unsafe.client.kerberos.hostname.disable.reversedns` and if this is true, client uses InetAddress.getHostname() for Kerberos principal instead of InetAddress.getCanonicalHostName().

Issue Links

• relates to: HBASE-12954 Ability impaired using HBase on multihomed hosts
• relates to: HBASE-18226 Disable reverse DNS lookup at HMaster and use the hostname provided by RegionServer
• relates to: HBASE-24667 Rename configs that support atypical DNS set ups to put them in hbase.unsafe
• links to: GitHub Pull Request #3051