Uploaded image for project: 'Apache Trafodion'
  1. Apache Trafodion
  2. TRAFODION-716

LP Bug: 1384897 - Bulkloader with compression not working with current sqenvcom.sh

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      Tried to use BulkLoader with compression.
      Would not work at it could not find the lz4 compression library.
      This library is supplied with cloudera.

      So we had to modify sqenvcom.sh as follows:
      ls /usr/lib/hadoop/hadoop-cdh5.jar >/dev/null 2>&1
      if [ $? -eq 0 ]; then

      1. we are on a cluster with Cloudera installed
      2. -------------------------------------------
      1. native library directories and include directories
      2. export HADOOP_LIB_DIR=/usr/lib64 <=== commented out this line.
        export HADOOP_LIB_DIR=/usr/lib/hadoop/lib/native <=== added this line.

      And later on:

      1. Library Path may include local over-rides
        export LD_LIBRARY_PATH=$CC_LIB:$MPI_ROOT/lib/$MPILIB:$MY_SQROOT/export/lib"$SQ_MBTYPE":$HADOOP_LIB_DIR:$LOC_JVMLIBS:$HWMSLIBS:${SP_EXPORT_LIB}:.

      Added $HADOOP_LIB_DIR: in the export above.

      Please fix sqenvcom so that it works without needed these changes.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              guyman Guy Groulx
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: