Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-34025

Show data skew score on Flink Dashboard

    XMLWordPrintableJSON

Details

    Description

      Problem: Currently users have to click on every operator and check how much data each subtask is processing to see if there is data skew. This is particularly cumbersome and error-prone for jobs with big job graphs. Data skew is an important metric that should be more visible.

       

      Proposed solution:

      • Show a data skew score on each operator (see screenshot below). This would be an improvement, but would not be sufficient. As it would still not be easy to see the data skew score for jobs with very large job graphs (it'd require a lot of zooming in/out).
      • Show data skew score for each operator under a new "Data Skew" tab next to the Exceptions tab. See screenshot below .

       

      Attachments

        1. skew_proposal.png
          214 kB
          Emre Kartoglu
        2. skew_tab.png
          273 kB
          Emre Kartoglu

        Issue Links

          Activity

            People

              iemre Emre Kartoglu
              iemre Emre Kartoglu
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: