Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Not A Problem
-
None
-
None
-
None
-
None
Description
The following problem was discovered in the Streaming Branch (HDDS-4454). The same problem can also be reproduced in the master branch (without using Streaming).
The ozone MR Acceptance fails on the streaming branch if ozone.fs.datastream.enable is set to true.
The test basically runs a yarn wordcount job and the observation here is that the job doesn’t release the Application master (AM container ) as soon as the job is done and the job counters are displayed which is why the second job waits for a long time ~10 min for the AM container to be released.
Tried with enabling streaming and disabling streaming and verified that the problem occurs only when ozone.fs.datastream.enable=true
yarn jar /opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar pi -D fs.defaultFS=o3fs://bucket1.volume1.om/ -D ozone.fs.datastream.enable=true 3 3