[ARROW-16043] [C++] S3-related crash on macOS

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 8.0.0
• Component/s: C++, Continuous Integration, Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/31462
• Assignee: Kouhei Sutou
• Reporter: Antoine Pitrou
• Created: 28/Mar/22 12:55
• Updated: 11/Jan/23 11:41
• Resolved: 15/Apr/22 20:24

Description

I'm not sure that's due to a AWS SDK version bump, but the C++ (and Python) S3 tests have started crashing on macOS CI.

There's a lldb traceback here:
https://github.com/apache/arrow/runs/5714619423?check_suite_focus=true#step:8:222
Excerpt:

* thread #1, stop reason = signal SIGSTOP
  * frame #0: 0x00007fff202d2bca libc++.1.dylib`std::__1::basic_istream<char, std::__1::char_traits<char> >::sentry::sentry(std::__1::basic_istream<char, std::__1::char_traits<char> >&, bool) + 26
    frame #1: 0x00007fff202d4c32 libc++.1.dylib`std::__1::basic_istream<char, std::__1::char_traits<char> >::tellg() + 132
    frame #2: 0x000000011000f445 libaws-cpp-sdk-core.dylib`Aws::Utils::Crypto::MD5CommonCryptoImpl::Calculate(std::__1::basic_istream<char, std::__1::char_traits<char> >&) + 71
    frame #3: 0x000000010ffd562f libaws-cpp-sdk-core.dylib`Aws::Utils::Crypto::MD5::Calculate(std::__1::basic_istream<char, std::__1::char_traits<char> >&) + 19
    frame #4: 0x000000010ffcf5d5 libaws-cpp-sdk-core.dylib`Aws::Utils::HashingUtils::CalculateMD5(std::__1::basic_iostream<char, std::__1::char_traits<char> >&) + 51
    frame #5: 0x000000010fff88d0 libaws-cpp-sdk-core.dylib`Aws::Client::AWSClient::AddChecksumToRequest(std::__1::shared_ptr<Aws::Http::HttpRequest> const&, Aws::AmazonWebServiceRequest const&) const + 1250
    frame #6: 0x000000010fff9dfb libaws-cpp-sdk-core.dylib`Aws::Client::AWSClient::BuildHttpRequest(Aws::AmazonWebServiceRequest const&, std::__1::shared_ptr<Aws::Http::HttpRequest> const&) const + 77
    frame #7: 0x000000010fff541d libaws-cpp-sdk-core.dylib`Aws::Client::AWSClient::AttemptOneRequest(std::__1::shared_ptr<Aws::Http::HttpRequest> const&, Aws::AmazonWebServiceRequest const&, char const*, char const*, char const*) const + 51
    frame #8: 0x000000010fff4137 libaws-cpp-sdk-core.dylib`Aws::Client::AWSClient::AttemptExhaustively(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const*, char const*, char const*) const + 875
    frame #9: 0x000000010fffc3ee libaws-cpp-sdk-core.dylib`Aws::Client::AWSXMLClient::MakeRequest(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const*, char const*, char const*) const + 60
    frame #10: 0x000000010fc65913 libaws-cpp-sdk-s3.dylib`Aws::S3::S3Client::PutObject(Aws::S3::Model::PutObjectRequest const&) const + 1811
    frame #11: 0x00000001030dfa1c arrow-s3fs-test`arrow::fs::TestS3FS::SetUp(this=0x00007fbcb3cf3340) at s3fs_test.cc:421:7
    frame #12: 0x0000000110100a0b libgtestd.1.11.0.dylib`void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(object=0x00007fbcb3cf3340, method=11 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00, location="SetUp()")(), char const*) at gtest.cc:2607:10
    frame #13: 0x00000001100c324a libgtestd.1.11.0.dylib`void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(object=0x00007fbcb3cf3340, method=11 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00, location="SetUp()")(), char const*) at gtest.cc:2643:14
    frame #14: 0x00000001100c313c libgtestd.1.11.0.dylib`testing::Test::Run(this=0x00007fbcb3cf3340) at gtest.cc:2677:3
    frame #15: 0x00000001100c4371 libgtestd.1.11.0.dylib`testing::TestInfo::Run(this=0x00007fbcb3c1f790) at gtest.cc:2861:11
    frame #16: 0x00000001100c5414 libgtestd.1.11.0.dylib`testing::TestSuite::Run(this=0x00007fbcb3c1f8f0) at gtest.cc:3015:28
    frame #17: 0x00000001100d4b2e libgtestd.1.11.0.dylib`testing::internal::UnitTestImpl::RunAllTests(this=0x00007fbcb3c207d0) at gtest.cc:5855:44
    frame #18: 0x0000000110105abb libgtestd.1.11.0.dylib`bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(object=0x00007fbcb3c207d0, method=50 47 0d 10 01 00 00 00 00 00 00 00 00 00 00 00, location="auxiliary test code (environments or event listeners)")(), char const*) at gtest.cc:2607:10
    frame #19: 0x00000001100d44ea libgtestd.1.11.0.dylib`bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(object=0x00007fbcb3c207d0, method=50 47 0d 10 01 00 00 00 00 00 00 00 00 00 00 00, location="auxiliary test code (environments or event listeners)")(), char const*) at gtest.cc:2643:14
    frame #20: 0x00000001100d43cb libgtestd.1.11.0.dylib`testing::UnitTest::Run(this=0x000000011011f440) at gtest.cc:5438:10
    frame #21: 0x000000010f782ed1 libgtest_maind.1.11.0.dylib`RUN_ALL_TESTS() at gtest.h:2490:46
    frame #22: 0x000000010f782ead libgtest_maind.1.11.0.dylib`main(argc=1, argv=0x00007ffeecb61390) at gtest_main.cc:52:10
    frame #23: 0x00007fff20381f3d libdyld.dylib`start + 1

Issue Links

• duplicates: ARROW-16070 [C++] [R] brew + aws sdk version segault
• is duplicated by: ARROW-16139 [Python] Crash in tests/test_dataset.py::test_write_dataset_s3
• links to: GitHub Pull Request #12895