Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently, the NNThroughputBenchmark test BlockReportStats relies on sorted datanodes array in the lexicographical order of datanode's xferAddr.

      • There is an assertion of datanode's xferAddr lexicographical order when filling the datanodes, see the code.
      • When searching the datanode by DatanodeInfo, it uses binary search against the datanodes array, see the code

      In DatanodeID, the xferAddr is defined as host:port. In NNThroughputBenchmark, the port is simply the index of the tiny datanode plus one.

      The problem here is that, when there are more than 9 tiny datanodes (numThreads), the lexicographical order of datanode's xferAddr will be invalid as the string value of datanode index is not in lexicographical order any more. For example,

      ...
      192.168.54.40:8
      192.168.54.40:9
      192.168.54.40:10
      192.168.54.40:11
      ...
      

      192.168.54.40:9 is greater than 192.168.54.40:10. The assertion will fail and the binary search won't work.

      The simple fix is to calculate the datanode index by port directly, instead of using binary search.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                liuml07 Mingliang Liu
                Reporter:
                liuml07 Mingliang Liu
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: