Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-279

Swimlanes visualization for Hadoop job progress

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.2.0
    • Fix Version/s: 0.2.0
    • Labels:
      None

      Description

      "Swimlanes" visualization (Tan09, HotCloud) for showing job progress in terms of SALSA (Tan08, WASL) states by using data from JobHistory logs. Initial version to show job progress in time across all nodes that worked on the job.

      Description of Swimlanes visualization.

        Issue Links

          Activity

          Hide
          tanjiaqi Jiaqi Tan added a comment -

          Includes: 1. updates to database schema (new table for SALSA state-machine states) and MDL schema for loading data from SALSA state-machines stored in ChukwaRecords, 2. JSP for widget that generates Swimlanes charts by reading from database.

          Show
          tanjiaqi Jiaqi Tan added a comment - Includes: 1. updates to database schema (new table for SALSA state-machine states) and MDL schema for loading data from SALSA state-machines stored in ChukwaRecords, 2. JSP for widget that generates Swimlanes charts by reading from database.
          Hide
          eyang Eric Yang added a comment -

          Applied the patch, there is no error message, however, the chart shows empty. There is a javascript error clearSelection being null. Added check for null condition, but the visualization still doesn't work. Output of swimlanes-event-data-js.jsp might not be syntactically correct.

          Show
          eyang Eric Yang added a comment - Applied the patch, there is no error message, however, the chart shows empty. There is a javascript error clearSelection being null. Added check for null condition, but the visualization still doesn't work. Output of swimlanes-event-data-js.jsp might not be syntactically correct.
          Hide
          hudson Hudson added a comment -

          Integrated in Chukwa-trunk #48 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/48/)
          . Added swim lane visualization for Hadoop job progress. (Jiaqi Tan via Eric Yang)

          Show
          hudson Hudson added a comment - Integrated in Chukwa-trunk #48 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/48/ ) . Added swim lane visualization for Hadoop job progress. (Jiaqi Tan via Eric Yang)
          Hide
          asrabkin Ari Rabkin added a comment -

          What's the status of this issue? I see that some code has been committed [r783066], but the issue isn't closed. Should it be? New visualizations sounds pretty awesome, but it's probably not ideal to commit things that don't work to the repository...

          Show
          asrabkin Ari Rabkin added a comment - What's the status of this issue? I see that some code has been committed [r783066] , but the issue isn't closed. Should it be? New visualizations sounds pretty awesome, but it's probably not ideal to commit things that don't work to the repository...
          Hide
          eyang Eric Yang added a comment -

          The visualization is done. It works fine. Initially, I had problem pin point the data because I forgot my time zone isn't UTC and the time selection widget needs to be offset by my time zone.

          There will be a follow up patch to include the backend analysis part in another JIRA.

          My fault for not closing this bug. This should be closed as fixed.
          Thanks Jiaqi.

          Show
          eyang Eric Yang added a comment - The visualization is done. It works fine. Initially, I had problem pin point the data because I forgot my time zone isn't UTC and the time selection widget needs to be offset by my time zone. There will be a follow up patch to include the backend analysis part in another JIRA. My fault for not closing this bug. This should be closed as fixed. Thanks Jiaqi.
          Hide
          asrabkin Ari Rabkin added a comment -

          giggle Glad I'm not the only one who keeps getting bitten by the timezone bug. Is there a clean way to fix it? Owen suggests that we impose UTC on all stored data.

          Show
          asrabkin Ari Rabkin added a comment - giggle Glad I'm not the only one who keeps getting bitten by the timezone bug. Is there a clean way to fix it? Owen suggests that we impose UTC on all stored data.
          Hide
          pbaclace Paul Baclace added a comment -

          Take a tip from aviation, Unix, and Java: time values should always be stored in UTC because TimeZones are viewer dependent.

          Show
          pbaclace Paul Baclace added a comment - Take a tip from aviation, Unix, and Java: time values should always be stored in UTC because TimeZones are viewer dependent.

            People

            • Assignee:
              tanjiaqi Jiaqi Tan
              Reporter:
              tanjiaqi Jiaqi Tan
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development