Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
1.2-incubating
-
None
-
Installing local hadoop when the pregenerated tools repository is not available.
Description
Much of Trafodion is written in C++ and use C++ to make calls to HDFS API's. Therefore, anyone building Trafodion must download a Hadoop version that contain native library binaries.
Trafodion provides a convenience script, called install_local_hadoop, that sets up the Hadoop eco-system that can be used to build and test the product. This script downloads Cloudera based versions of Hadoop and HBase. In addition, install_local_hadoop allows you to specify a environment variable called MY_LOCAL_SW_DIST where you can store tar files for these products. So when you run install_local_hadoop, it gets the tar files from the location specified in MY_LOCAL_SW_DIST thereby not requiring a download from the web.
If MY_LOCAL_SW_DIST is not specified, install_local_hadoop downloads the files from hard coded locations. The hard coded location for Hadoop is: http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.5.0-cdh5.3.0.tar.gz. Unfortunately, this version no longer contains the native Hadoop libraries. When Trafodion is built, an error that the hdfs library is not found is returned.
Many of the Trafodion contributors use the precreated files stored in MY_LOCAL_SW_DIST, therefore, we just cannot change the location in install_local_hadoop. It needs to be coordinated with Trafodion contributors who use a precreated directory called MY_LOCAL_SW_DIST.
This JIRA has been written to resolve this issue.
