[HDFS-16842] DatanodeHttpServer cannot get the new configuration of the new nameservice even after refreshNamenodes is run

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.1.2
• Fix Version/s: 3.1.2
• Component/s: datanode
• Labels: patch-available
• Assignee: Unassigned
• Reporter: Jihye Seo
• Created: 14/Nov/22 12:16
• Updated: 14/Nov/22 12:18

Description

'refreshNamenodes' should be run when a new nameservice is added.
However, a 'DatanodeHttpServer' cannot get the new configuration of the new nameservice even after 'refreshNamenodes' is run.
Therefore, when a client tries to access the new namespace's data with a swebhdfs or REST API request, an error occurs as follows.

Does not contain a valid host:port authority: NEW_NAMESPACE

When 'refreshNamenodes' is executed, a Datanode object reads the new configuration file and updates its configuration with 'setConf(new Configuration)'.

public void refreshNamenodes() throws IOException {
  checkSuperuserPrivilege();
  setConf(new Configuration());
  refreshNamenodes(getConf());
}

On the other hand, the configuration of a DatanodeHttpServer object remains unchanged after the DatanodeHttpServer object is created.
When a Datanode object is created, a DatanodeHttpServer is created by copying the Datanode's configuration. Since then, there's no way to update its configuration.

public DatanodeHttpServer(final Configuration conf,
      final DataNode datanode,
      final ServerSocketChannel externalHttpChannel)
    throws IOException {
    this.restCsrfPreventionFilter = createRestCsrfPreventionFilter(conf);
    this.conf = conf;
    ...
} 

This causes a DatanodeHttpServer to be unable to interpret a request for the new name service.

Therefore, a DatanodeHttpServer also needs to update its configuration to handle a request for a new name service.
I attached a patch based on the trunk.

 

Thanks!