Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-7200

[C++][Flight] Running Arrow Flight benchmark on two hosts doesn't work

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskConvert to sub-taskLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.15.0, 0.15.1
    • 0.16.0
    • Benchmarking, C++, FlightRPC
    • AWS EC2
      Instance type: t3a.xlarge
      AMI: ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20191002
      Number of instances: 2
      They are capable of pinging each other.

    Description

      I was trying to evaluate the performance of Apache Arrow Flight on two hosts (one as the client and the other one as the server), using [the official benchmark|https://github.com/apache/arrow/blob/master/cpp/src/arrow/flight/flight_benchmark.cc].

      Flags I used to build the project were:

       

      -DARROW_FLIGHT=ON
      -DCMAKE_BUILD_TYPE=Debug
      -DARROW_BUILD_BENCHMARKS=ON
      

       

      The branch I used was maint-0.15.x since there was a build error on the master branch. (The build error on master only existed in the environment where I set up two hosts: AWS. On my local environment (macOS) the build was successful on the master branch. I don't think this build error is relevant to the issue since there is no difference in the cpp source code.)

      On the host acting as the server, I ran 

      ./arrow-flight-perf-server

      On the host acting as the client, I ran 

      ./arrow-flight-benchmark --server_host ip-172-31-11-18

      It gives the following error:

      Failed with error: << IOError: gRPC returned unavailable error, with message: Connect Failed. Detail: Unavailable

       

       If I ran 

      ./arrow-flight-benchmark --server_host ip-172-31-11-17

      the error will be different:

      IOError: Server was not available after 10 attempts

      This is understandable since this host doesn't exist at all.

      This indicates that Flight is able to find the existing host (ip-172-31-11-18), but the communication somehow didn't succeed.

      The benchmark works fine if I run it with the localhost, either by not specifying the server_host flag or running the server in another process on the same host.

      I am not sure if the problem is in the environment or in the code itself. Could someone please give me some hint on how to resolve the problem?

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            cxma Chengxin Ma Assign to me
            cxma Chengxin Ma
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - Not Specified
              Not Specified
              Remaining:
              Remaining Estimate - 0h
              0h
              Logged:
              Time Spent - 40m
              40m

              Issue deployment