Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-27797

Shuffle service metric "registeredConnections" not tracked correctly

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.0.0
    • Fix Version/s: None
    • Component/s: Shuffle
    • Labels:
      None

      Description

      In ExternalShuffleBlockHandler:

          // Number of registered connections to the shuffle service
          private Counter registeredConnections = new Counter();
      
          public ShuffleMetrics() {
            ...
            allMetrics.put("numRegisteredConnections", registeredConnections);
          }
      

      But the counter that's actually updated is in TransportContext. The call site is in TransportChannelHandler:

        @Override
        public void channelRegistered(ChannelHandlerContext ctx) throws Exception {
          transportContext.getRegisteredConnections().inc();
          super.channelRegistered(ctx);
        }
      
        @Override
        public void channelUnregistered(ChannelHandlerContext ctx) throws Exception {
          transportContext.getRegisteredConnections().dec();
          super.channelUnregistered(ctx);
        }
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              vanzin Marcelo Vanzin
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: