Uploaded image for project: 'Apache HAWQ'
  1. Apache HAWQ
  2. HAWQ-1540

PXF should not accept parameters for init command

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3.0.0-incubating
    • Component/s: PXF
    • Labels:
      None

      Description

      Currently PXF accepts --hadoop-home parameter for init command, which is used in greenplum-embedded deployment to specify where Hadoop client artifacts are installed. The desired behavior is:

      • do not accept any parameters during init command
      • rely on configuration provided by conf/pxf-env.sh file
      • introduce new HADOOP_DISTRO variable that can take the following values: [<empty>, HDP, CDH, CUSTOM]
      • when the value is empty – the script will check for existence of RPM-installed HDP hadoop-client in /usr/hdp/current/hadoop-client/client and if found, will use HDP-specific private classpath template to generate new pxf-private.classpath file. If not found, the script will check for existence of RPM-installed CDH hadoop-client in /usr/lib/hadoop/client and if found, will use CDH-specific private classpath template to generate new pxf-private.classpath file. If none were found, the script will check for a TAR-installed hadoop client in the directory specified by the new variable HADOOP_ROOT under ${HADOOP_ROOT}/hadoop/share/hadoop/common/lib. If found, the script will use TAR-specific private classpath template to generate new pxf-private.classpath file. That template assumes Hadoop client (and optionally Hive and HBase clients) are installed under a common directorythat is pointed to by HADOOP_ROOT variable. A user will have to specify this variable or edit the pxf-env.sh config file before attempting pxf init command. If HADOOP_ROOT is not set of ${HADOOP_ROOT}/hadoop/share/hadoop/common/lib directory does not exist, the script will error out.
      • when the value is HDP – the script will check for existence of RPM-installed HDP hadoop-client in /usr/hdp/current/hadoop-client/client and if found, will use HDP-specific private classpath template to generate new pxf-private.classpath file. If not found, the script will check for TAR-installed distribution as described above.
      • when the value is CDH – the script will check for existence of RPM-installed CDH hadoop-client in /usr/lib/hadoop/client and if found, will use CDH-specific private classpath template to generate new pxf-private.classpath file. If not found, the script will check for TAR-installed distribution as described above.
      • when the value is CUSTOM – the script will check for TAR-installed distribution as described above, except it will not error out if HADOOP_ROOT is not set or points to a non-existing directory. If HADOOP_ROOT is set, the value will be substituted in tar-based template, even if the directory does not yet exist. If HADOOP_ROOT is not set, the tar-based template is copied to the conf/pxf-private.classpath file and the user will have to perform the editing of this file.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                adenissov Alexander Denissov
                Reporter:
                adenissov Alexander Denissov
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: