Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2810

RM job was stuck when running with oozie

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 4.3.0
    • 4.3.0
    • action, core, HA, workflow
    • hadoop2.7.2,centos7*3

    Description

      I'm running a mapreduce wordcount job task on oozie. 2 jobs were submitted to the yarn, and then the monitoring tasks running upto 99% were stuck. Wordcount job has been 0%.

      When I kill off the monitor job, wordcount job runs smoothly.

      I use a cluster of 3 virtual machines, configuration is as follows:

      Profile per VM: cores=2 memory=2048MB reserved=0GB usableMem=0GB disks=1
      Num Container=3
      Container Ram=640MB
      Used Ram=1GB
      Unused Ram=0GB
      yarn.scheduler.minimum-allocation-mb=640
      yarn.scheduler.maximum-allocation-mb=1920
      yarn.nodemanager.resource.memory-mb=1920
      mapreduce.map.memory.mb=640
      mapreduce.map.java.opts=-Xmx512m
      mapreduce.reduce.memory.mb=1280
      mapreduce.reduce.java.opts=-Xmx1024m
      yarn.app.mapreduce.am.resource.mb=640
      yarn.app.mapreduce.am.command-opts=-Xmx512m
      mapreduce.task.io.sort.mb=256
      Is there any way to solve this, to make two jobs run smoothly and finish at the same time ?

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            amendla yangsongjie

            Dates

              Created:
              Updated:

              Time Tracking

                Estimated:
                Original Estimate - 168h
                168h
                Remaining:
                Remaining Estimate - 168h
                168h
                Logged:
                Time Spent - Not Specified
                Not Specified

                Slack

                  Issue deployment