[KUDU-1889] Support OpenSSL 1.1.0

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.8.0
• Assignee: Adar Dembo
• Reporter: Dan Burkert
• Created: 17/Feb/17 22:17
• Updated: 12/Feb/19 23:36
• Resolved: 21/May/18 17:07

Description

We currently can't compile against OpenSSL 1.1.0. Probably low priority right now, but eventually distros are going to start shipping with 1.1.0 by default.

[121/323] Building CXX object src/kudu/security/CMakeFiles/security.dir/cert.cc.o
FAILED: src/kudu/security/CMakeFiles/security.dir/cert.cc.o
/usr/local/opt/ccache/libexec/c++   -DKUDU_HEADERS_NO_STUBS=1 -DKUDU_HEADERS_USE_RICH_SLICE=1 -DKUDU_HEADERS_USE_SHORT_STATUS_MACROS=1 -DKUDU_STATIC_DEFINE -DTCMALLOC_ENABLED -D__STDC_FORMAT_MACROS -Dsecurity_EXPORTS -Isrc -I../../src -isystem ../../thirdparty/installed/common/include -isystem ../../thirdparty/installed/uninstrumented/include -I/usr/local/opt/openssl@1.1/include -I/System/Library/Frameworks/Kerberos.framework/Headers -msse4.2 -Wall -Wno-sign-compare -Wno-deprecated -pthread -fno-strict-aliasing -DBOOST_DATE_TIME_POSIX_TIME_STD_CONFIG -ggdb -Qunused-arguments -Wno-ambiguous-member-template -Wdocumentation-deprecated-sync -std=c++11 -g -fPIC   -fPIC -MD -MT src/kudu/security/CMakeFiles/security.dir/cert.cc.o -MF src/kudu/security/CMakeFiles/security.dir/cert.cc.o.d -o src/kudu/security/CMakeFiles/security.dir/cert.cc.o -c /Users/dan/src/cloudera/kudu/src/kudu/security/cert.cc
/Users/dan/src/cloudera/kudu/src/kudu/security/cert.cc:158:29: error: member access into incomplete type 'X509_req_st'
  CHECK_GT(CRYPTO_add(&data_->references, 1, CRYPTO_LOCK_X509_REQ), 1)
                            ^
../../thirdparty/installed/uninstrumented/include/glog/logging.h:770:48: note: expanded from macro 'CHECK_GT'
#define CHECK_GT(val1, val2) CHECK_OP(_GT, > , val1, val2)
                                               ^
../../thirdparty/installed/uninstrumented/include/glog/logging.h:741:26: note: expanded from macro 'CHECK_OP'
  CHECK_OP_LOG(name, op, val1, val2, google::LogMessageFatal)
                         ^
../../thirdparty/installed/uninstrumented/include/glog/logging.h:722:44: note: expanded from macro 'CHECK_OP_LOG'
             google::GetReferenceableValue(val1),        \
                                           ^
/usr/local/opt/openssl@1.1/include/openssl/x509.h:91:16: note: forward declaration of 'X509_req_st'
typedef struct X509_req_st X509_REQ;
               ^
/Users/dan/src/cloudera/kudu/src/kudu/security/cert.cc:158:46: error: use of undeclared identifier 'CRYPTO_LOCK_X509_REQ'
  CHECK_GT(CRYPTO_add(&data_->references, 1, CRYPTO_LOCK_X509_REQ), 1)
                                             ^
../../thirdparty/installed/uninstrumented/include/glog/logging.h:770:48: note: expanded from macro 'CHECK_GT'
#define CHECK_GT(val1, val2) CHECK_OP(_GT, > , val1, val2)
                                               ^
../../thirdparty/installed/uninstrumented/include/glog/logging.h:741:26: note: expanded from macro 'CHECK_OP'
  CHECK_OP_LOG(name, op, val1, val2, google::LogMessageFatal)
                         ^
../../thirdparty/installed/uninstrumented/include/glog/logging.h:722:44: note: expanded from macro 'CHECK_OP_LOG'
             google::GetReferenceableValue(val1),        \
                                           ^
2 errors generated.
[122/323] Building CXX object src/kudu/security/CMakeFiles/security.dir/crypto.cc.o
FAILED: src/kudu/security/CMakeFiles/security.dir/crypto.cc.o
/usr/local/opt/ccache/libexec/c++   -DKUDU_HEADERS_NO_STUBS=1 -DKUDU_HEADERS_USE_RICH_SLICE=1 -DKUDU_HEADERS_USE_SHORT_STATUS_MACROS=1 -DKUDU_STATIC_DEFINE -DTCMALLOC_ENABLED -D__STDC_FORMAT_MACROS -Dsecurity_EXPORTS -Isrc -I../../src -isystem ../../thirdparty/installed/common/include -isystem ../../thirdparty/installed/uninstrumented/include -I/usr/local/opt/openssl@1.1/include -I/System/Library/Frameworks/Kerberos.framework/Headers -msse4.2 -Wall -Wno-sign-compare -Wno-deprecated -pthread -fno-strict-aliasing -DBOOST_DATE_TIME_POSIX_TIME_STD_CONFIG -ggdb -Qunused-arguments -Wno-ambiguous-member-template -Wdocumentation-deprecated-sync -std=c++11 -g -fPIC   -fPIC -MD -MT src/kudu/security/CMakeFiles/security.dir/crypto.cc.o -MF src/kudu/security/CMakeFiles/security.dir/crypto.cc.o.d-o src/kudu/security/CMakeFiles/security.dir/crypto.cc.o -c /Users/dan/src/cloudera/kudu/src/kudu/security/crypto.cc
/Users/dan/src/cloudera/kudu/src/kudu/security/crypto.cc:82:33: error: use of undeclared identifier 'EVP_MD_CTX_destroy'
  static constexpr auto free = &EVP_MD_CTX_destroy;
                                ^
1 error generated.
[124/323] Building CXX object src/kudu/security/CMakeFiles/security.dir/openssl_util.cc.o
/Users/dan/src/cloudera/kudu/src/kudu/security/openssl_util.cc:64:6: warning: unused function 'LockingCB' [-Wunused-function]
void LockingCB(int mode, int type, const char* /*file*/, int /*line*/) {
     ^
1 warning generated.
[125/323] Building CXX object src/kudu/security/CMakeFiles/security.dir/ca/cert_management.cc.o
FAILED: src/kudu/security/CMakeFiles/security.dir/ca/cert_management.cc.o
/usr/local/opt/ccache/libexec/c++   -DKUDU_HEADERS_NO_STUBS=1 -DKUDU_HEADERS_USE_RICH_SLICE=1 -DKUDU_HEADERS_USE_SHORT_STATUS_MACROS=1 -DKUDU_STATIC_DEFINE -DTCMALLOC_ENABLED -D__STDC_FORMAT_MACROS -Dsecurity_EXPORTS -Isrc -I../../src -isystem ../../thirdparty/installed/common/include -isystem ../../thirdparty/installed/uninstrumented/include -I/usr/local/opt/openssl@1.1/include -I/System/Library/Frameworks/Kerberos.framework/Headers -msse4.2 -Wall -Wno-sign-compare -Wno-deprecated -pthread -fno-strict-aliasing -DBOOST_DATE_TIME_POSIX_TIME_STD_CONFIG -ggdb -Qunused-arguments -Wno-ambiguous-member-template -Wdocumentation-deprecated-sync -std=c++11 -g -fPIC   -fPIC -MD -MT src/kudu/security/CMakeFiles/security.dir/ca/cert_management.cc.o -MF src/kudu/security/CMakeFiles/security.dir/ca/cert_management.cc.o.d -o src/kudu/security/CMakeFiles/security.dir/ca/cert_management.cc.o -c /Users/dan/src/cloudera/kudu/src/kudu/security/ca/cert_management.cc
In file included from /Users/dan/src/cloudera/kudu/src/kudu/security/ca/cert_management.cc:18:
../../src/kudu/security/ca/cert_management.h:38:26: error: typedef redefinition with different types ('struct env_md_st' vs 'struct evp_md_st')
typedef struct env_md_st EVP_MD;
                         ^
/usr/local/opt/openssl@1.1/include/openssl/ossl_typ.h:91:26: note: previous definition is here
typedef struct evp_md_st EVP_MD;
                         ^
/Users/dan/src/cloudera/kudu/src/kudu/security/ca/cert_management.cc:366:11: error: member access into incomplete type 'X509_REQ' (aka 'X509_req_st')
  if (!req->req_info ||
          ^
/usr/local/opt/openssl@1.1/include/openssl/x509.h:91:16: note: forward declaration of 'X509_req_st'
typedef struct X509_req_st X509_REQ;
               ^
2 errors generated.
[127/323] Building CXX object src/kudu/security/CMakeFiles/security.dir/tls_context.cc.o
FAILED: src/kudu/security/CMakeFiles/security.dir/tls_context.cc.o
/usr/local/opt/ccache/libexec/c++   -DKUDU_HEADERS_NO_STUBS=1 -DKUDU_HEADERS_USE_RICH_SLICE=1 -DKUDU_HEADERS_USE_SHORT_STATUS_MACROS=1 -DKUDU_STATIC_DEFINE -DTCMALLOC_ENABLED -D__STDC_FORMAT_MACROS -Dsecurity_EXPORTS -Isrc -I../../src -isystem ../../thirdparty/installed/common/include -isystem ../../thirdparty/installed/uninstrumented/include -I/usr/local/opt/openssl@1.1/include -I/System/Library/Frameworks/Kerberos.framework/Headers -msse4.2 -Wall -Wno-sign-compare -Wno-deprecated -pthread -fno-strict-aliasing -DBOOST_DATE_TIME_POSIX_TIME_STD_CONFIG -ggdb -Qunused-arguments -Wno-ambiguous-member-template -Wdocumentation-deprecated-sync -std=c++11 -g -fPIC   -fPIC -MD -MT src/kudu/security/CMakeFiles/security.dir/tls_context.cc.o -MF src/kudu/security/CMakeFiles/security.dir/tls_context.cc.o.d -o src/kudu/security/CMakeFiles/security.dir/tls_context.cc.o -c /Users/dan/src/cloudera/kudu/src/kudu/security/tls_context.cc
In file included from /Users/dan/src/cloudera/kudu/src/kudu/security/tls_context.cc:30:
../../src/kudu/security/ca/cert_management.h:38:26: error: typedef redefinition with different types ('struct env_md_st' vs 'struct evp_md_st')
typedef struct env_md_st EVP_MD;
                         ^
/usr/local/opt/openssl@1.1/include/openssl/ossl_typ.h:91:26: note: previous definition is here
typedef struct evp_md_st EVP_MD;
                         ^
1 error generated.
[130/323] Building CXX object src/kudu/security/CMakeFiles/security.dir/token_signer.cc.o
ninja: build stopped: subcommand failed.

Issue Links

• is depended upon by: KUDU-2427 Add support for Ubuntu 18.04
• is related to: KUDU-2439 There's no way to safely clean up a KuduClient or Messenger
• is related to: KUDU-2700 Memory leak in kudu-tool-test when using OpenSSL 1.1