[IMPALA-2827] Frontend does not compile with Thrift 0.9.3

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: Impala 2.5.0
• Component/s: Infrastructure
• Labels: osx
• Target Version: Product Backlog
• Assignee: Unassigned
• Reporter: Martin Grund
• Created: 08/Jan/16 19:49
• Updated: 11/Jun/18 19:48
• Resolved: 11/Jun/18 19:48

Description

Impala on Mac cannot use Thrift 0.9.0 and we have to move to a higher version. The latest release is 0.9.3 and for unkown reasons this breaks compilation of our Java frontend.

The compile error can be very easily fixed by removing some of the generated code that tries to Implement an asynchronous interface, however the root cause seems to be something else. It probably is related to the interface that is provided by Hive for the HS2 interface.

[INFO] -------------------------------------------------------------
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /Users/mgrund/Impala/fe/generated-sources/gen-java/com/cloudera/impala/thrift/ImpalaHiveServer2Service.java:[264,114] cannot find symbol
  symbol:   class AsyncProcessor
  location: class org.apache.hive.service.cli.thrift.TCLIService
[INFO] 1 error

Digging deeper the reason is pretty simple, that the Hive code that we use to Import the TCLIService does simply not contain the async part since it was not generated with Thrift 0.9.3.

Here is the necessary code in the Impala build:

IF (FIL STREQUAL "TCLIService.thrift" OR FIL STREQUAL "parquet.thrift" OR
        FIL STREQUAL "cli_service.thrift")
      # HiveServer2 and Parquet thrift generation
      # Do not generate Java source because we should use the jar from Hive or Parquet
      add_custom_command(
        OUTPUT ${OUTPUT_BE_FILE}
        COMMAND ${THRIFT_COMPILER} ${CPP_ARGS} ${FIL}
        COMMAND ${THRIFT_COMPILER} ${PYTHON_ARGS} ${FIL}
        DEPENDS ${ABS_FIL}
        COMMENT "Running thrift compiler on ${FIL}"
        VERBATIM
      )

Issue Links

• blocks: IMPALA-2761 Build and Run Impala on OS X
• duplicates: IMPALA-5690 Upgrade Thrift version to 0.9.3