Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-4454 Ozone Streaming Write Pipeline
  3. HDDS-6126

MR Acceptance test failure on Streaming Branch (HDDS-4454)

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      The ozone MR Acceptance fails on the streaming branch if ozone.fs.datastream.enable is set to true.

      The test basically runs a yarn wordcount job and  the observation here is that the job doesn’t release the Application master (AM container ) as soon as the job is done and the job counters are displayed which is why the second job waits for a long time ~10 min for the AM container to be released. 

      Tried with enabling streaming and disabling streaming and verified that the problem occurs only when  ozone.fs.datastream.enable=true 

      yarn jar /opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar pi -D   fs.defaultFS=o3fs://bucket1.volume1.om/  -D ozone.fs.datastream.enable=true 3 3

      Attachments

        1. NM-jstack.txt
          77 kB
          Sadanand Shenoy

        Activity

          People

            sadanand_shenoy Sadanand Shenoy
            sadanand_shenoy Sadanand Shenoy
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: