Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-2677

layer-spark: Improve sparkpi action output

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.2.1
    • Component/s: spark
    • Labels:

      Description

      When running the sparkpi action against the spark charm I get a lot of log output on the action result info.

      I would like to just see the output of the calculation of Pi and if that was successful.

      Example:
      http://paste.ubuntu.com/23859116/

      Code ref = https://github.com/apache/bigtop/tree/master/bigtop-packages/src/charm/spark/layer-spark

        Issue Links

          Activity

          Hide
          kwmonroe Kevin W Monroe added a comment -

          A fix for this came in with BIGTOP-2747. Note the results.meta.raw field that contains the actual result you care about, as well as the results.meta.fetchcmd field that tells you how to fetch the full action output:

          $ juju run-action spark/0 sparkpi
          results:
            meta:
              composite:
                direction: asc
                units: secs
                value: "38"
              fetchcmd: juju scp spark/0:/opt/sparkpi-results/1493156618.log .
              raw: |
                Pi is roughly 3.1405471405471403
              start: 2017-04-25T21:43:38Z
              stop: 2017-04-25T21:44:16Z
            outcome: success
          status: completed
          timing:
            completed: 2017-04-25 21:44:17 +0000 UTC
            enqueued: 2017-04-25 21:43:34 +0000 UTC
            started: 2017-04-25 21:43:38 +0000 UTC
          
          $ juju scp spark/0:/opt/sparkpi-results/1493156618.log .
          
          $ cat 1493156618.log | pastebinit
          

          http://paste.ubuntu.com/24456551/

          Show
          kwmonroe Kevin W Monroe added a comment - A fix for this came in with BIGTOP-2747 . Note the results.meta.raw field that contains the actual result you care about, as well as the results.meta.fetchcmd field that tells you how to fetch the full action output: $ juju run-action spark/0 sparkpi results: meta: composite: direction: asc units: secs value: "38" fetchcmd: juju scp spark/0:/opt/sparkpi-results/1493156618.log . raw: | Pi is roughly 3.1405471405471403 start: 2017-04-25T21:43:38Z stop: 2017-04-25T21:44:16Z outcome: success status: completed timing: completed: 2017-04-25 21:44:17 +0000 UTC enqueued: 2017-04-25 21:43:34 +0000 UTC started: 2017-04-25 21:43:38 +0000 UTC $ juju scp spark/0:/opt/sparkpi-results/1493156618.log . $ cat 1493156618.log | pastebinit http://paste.ubuntu.com/24456551/
          Hide
          kwmonroe Kevin W Monroe added a comment -

          The real result that the user wants to see is in your pastebin:

          "Pi is roughly 3.142128"

          This is currently buried alongside yarn stuffs in the "raw" action output. When the action succeeds, this extra yarn stuff is unnecessary. If the action failed, however, you'd probably want to see it.

          I think the best option is to override the "output" key in the action results dict. We create a "results: output: status: completed" entry which is totally superfluous given that "status: completed" is already present in the default action output. Perhaps we can alter the "results: output" key to house the "Pi is roughly xyz" so you would have some way to see the wanted output without having to scan the "raw" bits.

          Show
          kwmonroe Kevin W Monroe added a comment - The real result that the user wants to see is in your pastebin: "Pi is roughly 3.142128" This is currently buried alongside yarn stuffs in the "raw" action output. When the action succeeds, this extra yarn stuff is unnecessary. If the action failed, however, you'd probably want to see it. I think the best option is to override the "output" key in the action results dict. We create a "results: output: status: completed" entry which is totally superfluous given that "status: completed" is already present in the default action output. Perhaps we can alter the "results: output" key to house the "Pi is roughly xyz" so you would have some way to see the wanted output without having to scan the "raw" bits.

            People

            • Assignee:
              kwmonroe Kevin W Monroe
              Reporter:
              arosales Antonio Rosales
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 24h
                24h
                Remaining:
                Remaining Estimate - 24h
                24h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development