Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-6700

NPE with distcp on ofs with non-owner user

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Critical
    • Resolution: Unresolved
    • 1.3.0
    • None
    • Ozone Datanode
    • None

    Description

      DistCp fails with NPE with ofs scheme when a user doesn't have the privilege to read the volume.

      [root@nightly-71x-eu-1 ~]# /opt/cloudera/parcels/CDH/bin/hadoop distcp  -i -pp -skipcrccheck /tmp/testDir ofs://ozone1/test/test/test
      22/05/04 17:15:10 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=true, overwrite=false, append=false, useDiff=false, useRdiff=false, fromSnapshot=null, toSnapshot=null, skipCRC=true, blocking=true, numListstatusThreads=0, maxMaps=20, mapBandwidth=0.0, copyStrategy='uniformsize', preserveStatus=[PERMISSION], atomicWorkPath=null, logPath=null, sourceFileListing=null, sourcePaths=[/tmp/testDir], targetPath=ofs://ozone1/test/test/test, filtersFile='null', blocksPerChunk=0, copyBufferSize=8192, verboseLog=false, directWrite=false}, sourcePaths=[/tmp/testDir], targetPathExists=false, preserveRawXattrsfalse
      22/05/04 17:15:11 INFO client.RMProxy: Connecting to ResourceManager at nightly-71x-eu-2.nightly-71x-eu.root.hwx.site/172.27.11.70:8032
      22/05/04 17:15:12 ERROR tools.DistCp: Exception encountered
      java.lang.NullPointerException
          at org.apache.hadoop.tools.SimpleCopyListing.validatePaths(SimpleCopyListing.java:134)
          at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:85)
          at org.apache.hadoop.tools.GlobbedCopyListing.doBuildListing(GlobbedCopyListing.java:89)
          at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:86)
          at org.apache.hadoop.tools.DistCp.createInputFileListing(DistCp.java:379)
          at org.apache.hadoop.tools.DistCp.prepareFileListing(DistCp.java:96)
          at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:216)
          at org.apache.hadoop.tools.DistCp.execute(DistCp.java:193)
          at org.apache.hadoop.tools.DistCp.run(DistCp.java:155)
          at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
          at org.apache.hadoop.tools.DistCp.main(DistCp.java:443) 

      Even on adding acls for systest as all permissions same NPE was seen.
      Works only when the volume owner was changed to the test user i.e. systest

      # ozone sh volume update test --user=systest 

      Attachments

        Activity

          People

            Unassigned Unassigned
            ssulav Soumitra Sulav
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: