Hadoop Common
  1. Hadoop Common
  2. HADOOP-7492

Add automated test for the RPC IP addr change detection and reconnect feature

    Details

    • Type: Test Test
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.23.0
    • Fix Version/s: None
    • Component/s: ipc
    • Labels:
      None

      Description

      Make sure HADOOP-7472 is forward ported to 0.23 by adding an automated test.

      1. TestRPCAddressChange-1.patch
        5 kB
        Kihwal Lee
      2. TestRPCAddressChange.patch
        4 kB
        Kihwal Lee

        Issue Links

          Activity

          Hide
          Kihwal Lee added a comment -

          A while back one of the reviewers asked to file this and mark it as a blocker. I guess the situation has changed.

          Show
          Kihwal Lee added a comment - A while back one of the reviewers asked to file this and mark it as a blocker. I guess the situation has changed.
          Hide
          Suresh Srinivas added a comment -

          Kihwals, why is this marked a blocker?

          Show
          Suresh Srinivas added a comment - Kihwals, why is this marked a blocker?
          Hide
          Kihwal Lee added a comment -

          Given the state of HADOOP-7537, this test might not make it. I will wait until Daryn's changes and then think about other ways of testing it.

          Show
          Kihwal Lee added a comment - Given the state of HADOOP-7537 , this test might not make it. I will wait until Daryn's changes and then think about other ways of testing it.
          Hide
          Kihwal Lee added a comment -

          The new patch incorporates Suresh's review comments.

          Show
          Kihwal Lee added a comment - The new patch incorporates Suresh's review comments.
          Hide
          Suresh Srinivas added a comment -

          Comments:

          1. Can you please add in testAddrChange() method comments what the behavior at the client that is being tested, instead of terse, "address change detection by the RPC client". Description that covers, RPC client on seeing connection failure, re-resolves the address and connects to the new address would be good. This will make the test easy for others to understand.
          2. Can you please set in conf, "ipc.client.connect.max.retries" to 10 (even though default is 10, which might change) to ensure more than two attempts are made for the test to pass. done in two retries? Also along with whenNew could you add a comment that return old address for two retry attempts and resolve to a new address in the third attempt?
          3. Nit: LocaTestServer#call() has indentation issues
          4. Nit: Please add to class javadoc "Unit tests for RPC addr change detection in {@link Client}
          5. Nit: typo attemp
          Show
          Suresh Srinivas added a comment - Comments: Can you please add in testAddrChange() method comments what the behavior at the client that is being tested, instead of terse, "address change detection by the RPC client". Description that covers, RPC client on seeing connection failure, re-resolves the address and connects to the new address would be good. This will make the test easy for others to understand. Can you please set in conf, "ipc.client.connect.max.retries" to 10 (even though default is 10, which might change) to ensure more than two attempts are made for the test to pass. done in two retries? Also along with whenNew could you add a comment that return old address for two retry attempts and resolve to a new address in the third attempt? Nit: LocaTestServer#call() has indentation issues Nit: Please add to class javadoc "Unit tests for RPC addr change detection in {@link Client} Nit: typo attemp
          Hide
          Kihwal Lee added a comment -

          In the attached patch, the constructor of InetSocketAddress is mocked to return wrong or correct address. It requires PowerMock to run. I will make it "patch available" once HADOOP-7537 is resolved.

          Show
          Kihwal Lee added a comment - In the attached patch, the constructor of InetSocketAddress is mocked to return wrong or correct address. It requires PowerMock to run. I will make it "patch available" once HADOOP-7537 is resolved.
          Hide
          Kihwal Lee added a comment -

          Test output - requires PowerMockito

          2011-08-11 14:03:03,281 INFO  ipc.Server (Server.java:run(349)) - Starting Socket Reader #1 for port 49429
          2011-08-11 14:03:03,311 INFO  ipc.Server (Server.java:run(642)) - IPC Server Responder: starting
          2011-08-11 14:03:03,314 INFO  ipc.Server (Server.java:run(473)) - IPC Server listener on 49429: starting
          2011-08-11 14:03:03,315 INFO  ipc.Server (Server.java:run(1466)) - IPC Server handler 0 on 49429: starting
          2011-08-11 14:03:03,318 INFO  ipc.Server (Server.java:run(349)) - Starting Socket Reader #1 for port 34748
          2011-08-11 14:03:03,319 INFO  ipc.Server (Server.java:run(642)) - IPC Server Responder: starting
          2011-08-11 14:03:03,319 INFO  ipc.Server (Server.java:run(473)) - IPC Server listener on 34748: starting
          2011-08-11 14:03:03,331 INFO  ipc.Server (Server.java:run(1466)) - IPC Server handler 0 on 34748: starting
          2011-08-11 14:03:05,301 INFO  ipc.Server (Server.java:stop(1715)) - Stopping server on 49429
          2011-08-11 14:03:05,302 INFO  ipc.Server (Server.java:run(1539)) - IPC Server handler 0 on 49429: exiting
          2011-08-11 14:03:05,302 INFO  ipc.Server (Server.java:run(505)) - Stopping IPC Server listener on 49429
          2011-08-11 14:03:05,303 INFO  ipc.Server (Server.java:run(647)) - Stopping IPC Server Responder
          2011-08-11 14:03:06,382 INFO  ipc.Client (Client.java:handleConnectionFailure(670)) - Retrying connect to server: localhost.localdomain/127.0.0.1:49429. Already tried 0 time(s).
          2011-08-11 14:03:07,400 INFO  ipc.Client (Client.java:handleConnectionFailure(670)) - Retrying connect to server: localhost.localdomain/127.0.0.1:49429. Already tried 1 time(s).
          2011-08-11 14:03:07,413 WARN  ipc.Client (Client.java:updateAddress(422)) - Address change detected. Old: localhost.localdomain/127.0.0.1:49429 New: /127.0.0.1:34748
          2011-08-11 14:03:08,426 INFO  ipc.Client (Client.java:handleConnectionFailure(670)) - Retrying connect to server: /127.0.0.1:34748. Already tried 0 time(s).
          2011-08-11 14:03:08,490 INFO  ipc.Server (Server.java:stop(1715)) - Stopping server on 34748
          2011-08-11 14:03:08,490 INFO  ipc.Server (Server.java:run(1539)) - IPC Server handler 0 on 34748: exiting
          2011-08-11 14:03:08,490 INFO  ipc.Server (Server.java:run(505)) - Stopping IPC Server listener on 34748
          2011-08-11 14:03:08,491 INFO  ipc.Server (Server.java:run(647)) - Stopping IPC Server Responder
          
          Show
          Kihwal Lee added a comment - Test output - requires PowerMockito 2011-08-11 14:03:03,281 INFO ipc.Server (Server.java:run(349)) - Starting Socket Reader #1 for port 49429 2011-08-11 14:03:03,311 INFO ipc.Server (Server.java:run(642)) - IPC Server Responder: starting 2011-08-11 14:03:03,314 INFO ipc.Server (Server.java:run(473)) - IPC Server listener on 49429: starting 2011-08-11 14:03:03,315 INFO ipc.Server (Server.java:run(1466)) - IPC Server handler 0 on 49429: starting 2011-08-11 14:03:03,318 INFO ipc.Server (Server.java:run(349)) - Starting Socket Reader #1 for port 34748 2011-08-11 14:03:03,319 INFO ipc.Server (Server.java:run(642)) - IPC Server Responder: starting 2011-08-11 14:03:03,319 INFO ipc.Server (Server.java:run(473)) - IPC Server listener on 34748: starting 2011-08-11 14:03:03,331 INFO ipc.Server (Server.java:run(1466)) - IPC Server handler 0 on 34748: starting 2011-08-11 14:03:05,301 INFO ipc.Server (Server.java:stop(1715)) - Stopping server on 49429 2011-08-11 14:03:05,302 INFO ipc.Server (Server.java:run(1539)) - IPC Server handler 0 on 49429: exiting 2011-08-11 14:03:05,302 INFO ipc.Server (Server.java:run(505)) - Stopping IPC Server listener on 49429 2011-08-11 14:03:05,303 INFO ipc.Server (Server.java:run(647)) - Stopping IPC Server Responder 2011-08-11 14:03:06,382 INFO ipc.Client (Client.java:handleConnectionFailure(670)) - Retrying connect to server: localhost.localdomain/127.0.0.1:49429. Already tried 0 time(s). 2011-08-11 14:03:07,400 INFO ipc.Client (Client.java:handleConnectionFailure(670)) - Retrying connect to server: localhost.localdomain/127.0.0.1:49429. Already tried 1 time(s). 2011-08-11 14:03:07,413 WARN ipc.Client (Client.java:updateAddress(422)) - Address change detected. Old: localhost.localdomain/127.0.0.1:49429 New: /127.0.0.1:34748 2011-08-11 14:03:08,426 INFO ipc.Client (Client.java:handleConnectionFailure(670)) - Retrying connect to server: /127.0.0.1:34748. Already tried 0 time(s). 2011-08-11 14:03:08,490 INFO ipc.Server (Server.java:stop(1715)) - Stopping server on 34748 2011-08-11 14:03:08,490 INFO ipc.Server (Server.java:run(1539)) - IPC Server handler 0 on 34748: exiting 2011-08-11 14:03:08,490 INFO ipc.Server (Server.java:run(505)) - Stopping IPC Server listener on 34748 2011-08-11 14:03:08,491 INFO ipc.Server (Server.java:run(647)) - Stopping IPC Server Responder
          Hide
          Kihwal Lee added a comment -

          HADOOP-7472 has been committed to TRUNK. The testing of IP address change detection was done manually, but we will have a way to automate the test with HADOOP-7537.

          Show
          Kihwal Lee added a comment - HADOOP-7472 has been committed to TRUNK. The testing of IP address change detection was done manually, but we will have a way to automate the test with HADOOP-7537 .

            People

            • Assignee:
              Kihwal Lee
              Reporter:
              Kihwal Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:

                Development