Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: bsp core
    • Labels:
      None
    1. HAMA-429.patch
      3 kB
      ChiaHung Lin

      Activity

      Hide
      ChiaHung Lin added a comment -

      After checking out the source from svn trunk, the implementation shows that the doReport() is only done after bsp peer finishes its execution by GroomServer. This looks better than original way which GroomServer periodically executes doReport() back to bsp master. The only weird thing is sometime the system is kept overloaded even after job is finished for a while; but this probably is not the case in real hardware because my env is vms. If this also happens in real hardware then we may need to delve to see what makes this happened.

      Show
      ChiaHung Lin added a comment - After checking out the source from svn trunk, the implementation shows that the doReport() is only done after bsp peer finishes its execution by GroomServer. This looks better than original way which GroomServer periodically executes doReport() back to bsp master. The only weird thing is sometime the system is kept overloaded even after job is finished for a while; but this probably is not the case in real hardware because my env is vms. If this also happens in real hardware then we may need to delve to see what makes this happened.
      Hide
      Edward J. Yoon added a comment -

      BTW, there's a problem to provide progress report to Job Client.

      root@Cnode1:/usr/local/src/hama-trunk# core/bin/hama jar examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.
      11/09/01 15:04:49 DEBUG bsp.BSPJobClient: BSPJobClient.submitJobDir: hdfs://hnode15:9000/tmp/hadoop-root/bsp/syst
      11/09/01 15:04:50 INFO bsp.BSPJobClient: Running job: job_201109011504_0002
      11/09/01 15:04:53 INFO bsp.BSPJobClient: Current supersteps number: 0
      11/09/01 15:05:20 INFO bsp.BSPJobClient: Current supersteps number: 512
      11/09/01 15:05:20 INFO bsp.BSPJobClient: The total number of supersteps: 512
      

      During job is running, groom should update status of assigned tasks to BSPMaster.

      Show
      Edward J. Yoon added a comment - BTW, there's a problem to provide progress report to Job Client. root@Cnode1:/usr/local/src/hama-trunk# core/bin/hama jar examples/target/hama-examples-0.4.0-incubating-SNAPSHOT. 11/09/01 15:04:49 DEBUG bsp.BSPJobClient: BSPJobClient.submitJobDir: hdfs: //hnode15:9000/tmp/hadoop-root/bsp/syst 11/09/01 15:04:50 INFO bsp.BSPJobClient: Running job: job_201109011504_0002 11/09/01 15:04:53 INFO bsp.BSPJobClient: Current supersteps number: 0 11/09/01 15:05:20 INFO bsp.BSPJobClient: Current supersteps number: 512 11/09/01 15:05:20 INFO bsp.BSPJobClient: The total number of supersteps: 512 During job is running, groom should update status of assigned tasks to BSPMaster.
      Hide
      ChiaHung Lin added a comment -

      The patch attached contains code that groom server periodically reports back to bsp master.

      Show
      ChiaHung Lin added a comment - The patch attached contains code that groom server periodically reports back to bsp master.
      Hide
      Edward J. Yoon added a comment -

      I'll look at and test this patch today.

      Show
      Edward J. Yoon added a comment - I'll look at and test this patch today.
      Hide
      Edward J. Yoon added a comment -

      +1 I just committed this. Thanks Chiahung Lin.

      Show
      Edward J. Yoon added a comment - +1 I just committed this. Thanks Chiahung Lin.

        People

        • Assignee:
          ChiaHung Lin
          Reporter:
          Edward J. Yoon
        • Votes:
          0 Vote for this issue
          Watchers:
          0 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development