Uploaded image for project: 'Falcon'
  1. Falcon
  2. FALCON-796

Enable users to triage data processing issues through falcon

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7
    • Component/s: None
    • Labels:
      None

      Description

      Falcon has a wealth of information that can help users to triage issues relating to their pipelines, but this information is not available in a consumable form to the users. Here is a suggestion to solve for this issue to some extent.

      In a typical usage scenario, users may find that their data has not been generated and that would require them to walk up the dependency tree to identify the root cause. Luckily all this information is available within Falcon and can be surfaced.

      A mock of how this would look:

      falcon instance -type feed -name FEED-abc -start 2014-10-01T00:00Z -end 2014-10-01T00:00Z -triage

      FEED-abc/2014-10-01T00:00Z MISSING
        + PROCESS-def/2014-10-02T04:00Z WAITING
          + FEED-ghi/2014-10-01T00:00Z MISSING
            + PROCESS-mno/2014-10-01T01:30Z  RUNNING
          + FEED-ghi/2014-10-01T01:00Z MISSING
            + PROCESS-mno/2014-10-01T02:30Z  FAILED
          + FEED-ghi/*                 AVAIALBLE
          + FEED-jkl/*                 AVAILABLE
      

      falcon instance -type feed -name FEED-abc -start 2014-09-01T00:00Z -end 2014-09-01T00:00Z -triage

      FEED-abc/2014-09-01T00:00Z AVAILABLE
      

      falcon instance -type process -name PROCESS-def -start 2014-10-01T00:00Z -end 2014-10-01T00:00Z -triage

      PROCESS-def/2014-10-02T04:00Z WAITING
        + FEED-ghi/2014-10-01T00:00Z MISSING
          + PROCESS-mno/2014-10-01T01:30Z  RUNNING
        + FEED-ghi/2014-10-01T01:00Z MISSING
          + PROCESS-mno/2014-10-01T02:30Z  FAILED
        + FEED-ghi/*                 AVAIALBLE
        + FEED-jkl/*                 AVAILABLE
      
      1. FALCON-796.patch
        28 kB
        Ajay Yadava
      2. FALCON-796.patch
        31 kB
        Ajay Yadava
      3. FALCON-796-v2.patch
        28 kB
        Ajay Yadava

        Issue Links

          Activity

          Hide
          sanjeevtripurari Sanjeev T added a comment -

          A PROCESS/FEED may have many level of dependencies, can we have depth level of process.

          Show
          sanjeevtripurari Sanjeev T added a comment - A PROCESS/FEED may have many level of dependencies, can we have depth level of process.
          Hide
          sriksun Srikanth Sundarrajan added a comment -

          Makes sense. I had considered and dropped it, as the triage options exclusively reports only failed/unavailable sections of the dependency tree, if operationally it would help, we can include it. Besides, it will also help the falcon server by not overwhelming it with too much to do in a single call.

          Show
          sriksun Srikanth Sundarrajan added a comment - Makes sense. I had considered and dropped it, as the triage options exclusively reports only failed/unavailable sections of the dependency tree, if operationally it would help, we can include it. Besides, it will also help the falcon server by not overwhelming it with too much to do in a single call.
          Hide
          ajayyadava Ajay Yadava added a comment -

          This patch must be applied after FALCON-1039.

          Show
          ajayyadava Ajay Yadava added a comment - This patch must be applied after FALCON-1039 .
          Hide
          ajayyadava Ajay Yadava added a comment -

          Will put a review board request after 1039 is merged.

          Show
          ajayyadava Ajay Yadava added a comment - Will put a review board request after 1039 is merged.
          Hide
          sriksun Srikanth Sundarrajan added a comment -

          Can you merge FALCON-1039 if there are no further changes and upload patch for FALCON-796

          Show
          sriksun Srikanth Sundarrajan added a comment - Can you merge FALCON-1039 if there are no further changes and upload patch for FALCON-796
          Hide
          ajayyadava Ajay Yadava added a comment -

          Rebased the patch.

          Show
          ajayyadava Ajay Yadava added a comment - Rebased the patch.
          Hide
          ajayyadava Ajay Yadava added a comment -

          Attaching the latest patch which is available in review board also.

          Show
          ajayyadava Ajay Yadava added a comment - Attaching the latest patch which is available in review board also.
          Hide
          sriksun Srikanth Sundarrajan added a comment -

          +1

          Show
          sriksun Srikanth Sundarrajan added a comment - +1
          Hide
          ajayyadava Ajay Yadava added a comment -

          Committed to master. Thanks Srikanth Sundarrajan for reviewing!

          Show
          ajayyadava Ajay Yadava added a comment - Committed to master. Thanks Srikanth Sundarrajan for reviewing!

            People

            • Assignee:
              ajayyadava Ajay Yadava
              Reporter:
              sriksun Srikanth Sundarrajan
            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development