[BEAM-9123] HadoopResourceId returns wrong directory name

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.17.0
• Fix Version/s: 2.19.0
• Component/s: io-java-hadoop-file-system
• Assignee: Marek Simunek
• Reporter: Marek Simunek
• Created: 15/Jan/20 14:20
• Updated: 22/Jan/20 18:40
• Resolved: 16/Jan/20 13:07

Description

 HadoopResourceId returns for directory uri wrong fileName() which doesnt conform ResourceId interface javadoc

Returns the name of the file or directory..

String getFilename();

 

URI hdfsClusterBaseUri = new URI(configuration.get("fs.defaultFS") + "/");
HadoopResourceId hadoopResourceId =
new HadoopResourceId(hdfsClusterBaseUri.resolve("/dirA/file1"));

Expected behavior: 

hadoopResourceId.getCurrentDirectory().getFilename() returns dirA

Actual behavior:
hadoopResourceId.getCurrentDirectory().getFilename() returns ""

The problem is that uri for folder is ending with '/' and  getFileName() is using hadoop Path.getName() which returns everything after last slash.

 

Issue Links

• links to: GitHub Pull Request #10597
• links to: GitHub Pull Request #10625
• links to: GitHub Pull Request #10626
• links to: GitHub Pull Request #10658