[MAPREDUCE-7373] Building MapReduce NativeTask fails on Fedora 34+

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.4.0, 3.2.4, 3.3.3
• Component/s: build, nativetask
• Labels: pull-request-available
• Target Version/s: 3.3.3
• Hadoop Flags: Reviewed
• Assignee: Kengo Seki
• Reporter: Kengo Seki
• Created: 30/Mar/22 01:52
• Updated: 19/Apr/22 06:54
• Resolved: 18/Apr/22 14:15

Description

Fedora 34 adopted GCC 11, in which C++17 features are enabled by default.
https://gcc.gnu.org/projects/cxx-status.html#cxx17

Building MapReduce NativeTask with it leads to the following error.
(I found it on branch-3.2, but it's supposed to be the same as trunk)

$ mvn package -Pdist,native -DskipTests -Dtar -Dmaven.javadoc.skip=true

...

[WARNING] In file included from /home/vagrant/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/native/src/lib/MapOutputCollector.h:30,
[WARNING]                  from /home/vagrant/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/native/src/handler/MCollectorOutputHandler.cc:24:
[WARNING] /home/vagrant/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/native/src/lib/PartitionBucket.h:127:36: error: ISO C++17 does not allow dynamic exception specifications
[WARNING]   127 |   void spill(IFileWriter * writer) throw (IOException, UnsupportException);
[WARNING]       |                                    ^~~~~
[WARNING] make[2]: *** [CMakeFiles/nativetask_static.dir/build.make:160: CMakeFiles/nativetask_static.dir/main/native/src/handler/MCollectorOutputHandler.cc.o] Error 1
[WARNING] make[1]: *** [CMakeFiles/Makefile2:115: CMakeFiles/nativetask_static.dir/all] Error 2
[WARNING] make: *** [Makefile:91: all] Error 2

...

[INFO] Apache Hadoop MapReduce HistoryServer Plugins ...... SUCCESS [  0.570 s]
[INFO] Apache Hadoop MapReduce NativeTask ................. FAILURE [ 11.016 s]
[INFO] Apache Hadoop MapReduce Uploader ................... SKIPPED
[INFO] Apache Hadoop MapReduce Examples ................... SKIPPED
[INFO] Apache Hadoop MapReduce ............................ SKIPPED
[INFO] Apache Hadoop MapReduce Streaming .................. SKIPPED
[INFO] Apache Hadoop Distributed Copy ..................... SKIPPED
[INFO] Apache Hadoop Archives ............................. SKIPPED
[INFO] Apache Hadoop Archive Logs ......................... SKIPPED
[INFO] Apache Hadoop Rumen ................................ SKIPPED
[INFO] Apache Hadoop Gridmix .............................. SKIPPED
[INFO] Apache Hadoop Data Join ............................ SKIPPED
[INFO] Apache Hadoop Extras ............................... SKIPPED
[INFO] Apache Hadoop Pipes ................................ SKIPPED
[INFO] Apache Hadoop OpenStack support .................... SKIPPED
[INFO] Apache Hadoop Amazon Web Services support .......... SKIPPED
[INFO] Apache Hadoop Kafka Library support ................ SKIPPED
[INFO] Apache Hadoop Azure support ........................ SKIPPED
[INFO] Apache Hadoop Aliyun OSS support ................... SKIPPED
[INFO] Apache Hadoop Client Aggregator .................... SKIPPED
[INFO] Apache Hadoop Scheduler Load Simulator ............. SKIPPED
[INFO] Apache Hadoop Resource Estimator Service ........... SKIPPED
[INFO] Apache Hadoop Azure Data Lake support .............. SKIPPED
[INFO] Apache Hadoop Tools Dist ........................... SKIPPED
[INFO] Apache Hadoop Tools ................................ SKIPPED
[INFO] Apache Hadoop Client API ........................... SKIPPED
[INFO] Apache Hadoop Client Runtime ....................... SKIPPED
[INFO] Apache Hadoop Client Packaging Invariants .......... SKIPPED
[INFO] Apache Hadoop Client Test Minicluster .............. SKIPPED
[INFO] Apache Hadoop Client Packaging Invariants for Test . SKIPPED
[INFO] Apache Hadoop Client Packaging Integration Tests ... SKIPPED
[INFO] Apache Hadoop Distribution ......................... SKIPPED
[INFO] Apache Hadoop Client Modules ....................... SKIPPED
[INFO] Apache Hadoop Cloud Storage ........................ SKIPPED
[INFO] Apache Hadoop Cloud Storage Project ................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  04:10 min
[INFO] Finished at: 2022-03-30T01:49:59Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.2.4-SNAPSHOT:cmake-compile (cmake-compile) on project hadoop-mapreduce-client-nativetask: make failed with error code 2 -> [Help 1]

Issue Links

• is depended upon by: HADOOP-18198 Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes
• links to: GitHub Pull Request #4120