[HDDS-6700] NPE with distcp on ofs with non-privileged user

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 1.3.0
• Component/s: Ozone Datanode
• Assignee: Saketa Chalamchala
• Reporter: Soumitra Sulav
• Created: 04/May/22 17:25
• Updated: 10/Jul/23 18:25
• Resolved: 10/Jul/23 18:25

Description

DistCp fails with NPE with ofs scheme when a user doesn't have the privilege to read the volume.

[root@nightly-71x-eu-1 ~]# kinit -kt /cdep/keytabs/systest.keytab systest
[root@nightly-71x-eu-1 ~]# /opt/cloudera/parcels/CDH/bin/hadoop distcp  -i -pp -skipcrccheck /tmp/testDir ofs://ozone1/test/test/test
22/05/04 17:15:10 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=true, overwrite=false, append=false, useDiff=false, useRdiff=false, fromSnapshot=null, toSnapshot=null, skipCRC=true, blocking=true, numListstatusThreads=0, maxMaps=20, mapBandwidth=0.0, copyStrategy='uniformsize', preserveStatus=[PERMISSION], atomicWorkPath=null, logPath=null, sourceFileListing=null, sourcePaths=[/tmp/testDir], targetPath=ofs://ozone1/test/test/test, filtersFile='null', blocksPerChunk=0, copyBufferSize=8192, verboseLog=false, directWrite=false}, sourcePaths=[/tmp/testDir], targetPathExists=false, preserveRawXattrsfalse
22/05/04 17:15:11 INFO client.RMProxy: Connecting to ResourceManager at nightly-71x-eu-2.nightly-71x-eu.root.hwx.site/172.27.11.70:8032
22/05/04 17:15:12 ERROR tools.DistCp: Exception encountered
java.lang.NullPointerException
    at org.apache.hadoop.tools.SimpleCopyListing.validatePaths(SimpleCopyListing.java:134)
    at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:85)
    at org.apache.hadoop.tools.GlobbedCopyListing.doBuildListing(GlobbedCopyListing.java:89)
    at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:86)
    at org.apache.hadoop.tools.DistCp.createInputFileListing(DistCp.java:379)
    at org.apache.hadoop.tools.DistCp.prepareFileListing(DistCp.java:96)
    at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:216)
    at org.apache.hadoop.tools.DistCp.execute(DistCp.java:193)
    at org.apache.hadoop.tools.DistCp.run(DistCp.java:155)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
    at org.apache.hadoop.tools.DistCp.main(DistCp.java:443) 

Even on adding acls for systest as all permissions same NPE was seen.
Works only when the volume owner was changed to the test user i.e. systest
ozone sh volume update test --user=systest

Post Edit :
Worked on ranger policy add and sync.
But still, we can add a better exception message than NPE.