Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14524

NNTop total counts does not add up as expected

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      opType='*' is sometimes smaller than the sum of the individual operation types.

      {
        "windows": [
          {
            "windowLenMs": 300000,
            "ops": [
              {
                "totalCount": 24158,
                "opType": "rpc.complete",
                "topUsers": [{ "count": 2944, "user": "user1" }]
              },
              {
                "totalCount": 15921,
                "opType": "rpc.rename",
                "topUsers": [{ "count": 2891, "user": "user1" }]
              },
              {
                "totalCount": 3015834,
                "opType": "*",
                "topUsers": [{ "count": 66652, "user": "user1" }]
              },
              {
                "totalCount": 2086,
                "opType": "rpc.abandonBlock",
                "topUsers": [{ "count": 603, "user": "user1" }]
              },
              {
                "totalCount": 30258,
                "opType": "rpc.addBlock",
                "topUsers": [{ "count": 3182, "user": "user1" }]
              },
              {
                "totalCount": 101440,
                "opType": "rpc.getServerDefaults",
                "topUsers": [{ "count": 3521, "user": "user1" }]
              },
              {
                "totalCount": 25258,
                "opType": "rpc.create",
                "topUsers": [{ "count": 1864, "user": "user1" }]
              },
              {
                "totalCount": 1377563,
                "opType": "rpc.getFileInfo",
                "topUsers": [{ "count": 56541, "user": "user1" }]
              },
              {
                "totalCount": 60836,
                "opType": "rpc.renewLease",
                "topUsers": [{ "count": 3783, "user": "user1" }]
              },
              {
                "totalCount": 182212,
                "opType": "rpc.getListing",
                "topUsers": [{ "count": 1848, "user": "user1" }]
              },
              {
                "totalCount": 380,
                "opType": "rpc.updateBlockForPipeline",
                "topUsers": [{ "count": 58, "user": "user1" }]
              },
              {
                "totalCount": 215,
                "opType": "rpc.updatePipeline",
                "topUsers": [{ "count": 18, "user": "user1" }]
              }
            ]
          }
        ],
        "timestamp": "2019-01-12"
      }
      

       
      opType='*' from user user1 is 66652, but the sum of counts for other optype values by user1 is actually larger: 77253

        Attachments

        1. HDFS-14524.001.patch
          30 kB
          Ahmed Hussein

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                ahussein Ahmed Hussein
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: