Details

      Description

      As I'm using flink with the HBASE_CONF_DIR env variable and don't install hbase, then I get the error message below.

      bin/config.sh: line 303: hbase: command not found

      1. FLINK-6286-3.patch
        1 kB
        Jinjiang Ling
      2. FLINK-6286-2.patch
        1 kB
        Jinjiang Ling
      3. FLINK-6286-1.patch
        0.9 kB
        Jinjiang Ling
      4. FLINK-6286-0.patch
        0.9 kB
        Jinjiang Ling

        Issue Links

          Activity

          Hide
          greghogan Greg Hogan added a comment -

          Fixed in 3042b182c818a68f93b163fcf02db397580e1782

          Show
          greghogan Greg Hogan added a comment - Fixed in 3042b182c818a68f93b163fcf02db397580e1782
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/3711

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3711
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user lingjinjiang commented on the issue:

          https://github.com/apache/flink/pull/3711

          @greghogan @zentol thanks. Can this pull request be merged or closed now ?

          Show
          githubbot ASF GitHub Bot added a comment - Github user lingjinjiang commented on the issue: https://github.com/apache/flink/pull/3711 @greghogan @zentol thanks. Can this pull request be merged or closed now ?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zentol commented on the issue:

          https://github.com/apache/flink/pull/3711

          @lingjinjiang You are now the proud owner of contributor permissions and can assign issues to yourself.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zentol commented on the issue: https://github.com/apache/flink/pull/3711 @lingjinjiang You are now the proud owner of contributor permissions and can assign issues to yourself.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user greghogan commented on the issue:

          https://github.com/apache/flink/pull/3711

          @fhueske can grant you Jira permissions.

          Show
          githubbot ASF GitHub Bot added a comment - Github user greghogan commented on the issue: https://github.com/apache/flink/pull/3711 @fhueske can grant you Jira permissions.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user lingjinjiang commented on the issue:

          https://github.com/apache/flink/pull/3711

          @greghogan thanks for your review. I have another question, how can I assign the issue to myself ?

          Show
          githubbot ASF GitHub Bot added a comment - Github user lingjinjiang commented on the issue: https://github.com/apache/flink/pull/3711 @greghogan thanks for your review. I have another question, how can I assign the issue to myself ?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user greghogan commented on the issue:

          https://github.com/apache/flink/pull/3711

          @lingjinjiang thanks for your contribution! Merging ...

          Show
          githubbot ASF GitHub Bot added a comment - Github user greghogan commented on the issue: https://github.com/apache/flink/pull/3711 @lingjinjiang thanks for your contribution! Merging ...
          Hide
          lingjinjiang Jinjiang Ling added a comment -

          modify the log message

          Show
          lingjinjiang Jinjiang Ling added a comment - modify the log message
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user lingjinjiang commented on the issue:

          https://github.com/apache/flink/pull/3711

          @greghogan, thanks for your suggestion. I have changed the log message.

          Show
          githubbot ASF GitHub Bot added a comment - Github user lingjinjiang commented on the issue: https://github.com/apache/flink/pull/3711 @greghogan, thanks for your suggestion. I have changed the log message.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user greghogan commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3711#discussion_r111397645

          — Diff: flink-dist/src/main/flink-bin/bin/config.sh —
          @@ -310,8 +310,16 @@ fi
          INTERNAL_HADOOP_CLASSPATHS="$

          {HADOOP_CLASSPATH}

          :$

          {HADOOP_CONF_DIR}

          :$

          {YARN_CONF_DIR}

          "

          if [ -n "$

          {HBASE_CONF_DIR}" ]; then
          - # Setup the HBase classpath.
          - INTERNAL_HADOOP_CLASSPATHS="${INTERNAL_HADOOP_CLASSPATHS}:`hbase classpath`"
          + # Search hbase command in PATH.
          + HBASE_IN_PATH=$(PATH="${HBASE_HOME}/bin:$PATH" \
          + which hbase 2>/dev/null)
          + # Whether the hbase command is found.
          + if [ -f "${HBASE_IN_PATH}" ]; then
          + # The hbase command is found, then setup the HBase classpath.
          + INTERNAL_HADOOP_CLASSPATHS="${INTERNAL_HADOOP_CLASSPATHS}:`${HBASE_IN_PATH} classpath`"
          + else
          + echo "The HBASE_IN_PATH is setted but can't setup HBase classpath. Please verify hbase command exists."
          — End diff –

          Perhaps more like "HBASE_CONF_DIR=\"${HBASE_CONF_DIR}

          \" is set but 'hbase' command was not found in \"$

          {HBASE_PATH}

          \" so classpath could not be updated."? I assumed HBASE_PATH="$

          {HBASE_HOME}

          /bin:$PATH" as a local variable.

          Show
          githubbot ASF GitHub Bot added a comment - Github user greghogan commented on a diff in the pull request: https://github.com/apache/flink/pull/3711#discussion_r111397645 — Diff: flink-dist/src/main/flink-bin/bin/config.sh — @@ -310,8 +310,16 @@ fi INTERNAL_HADOOP_CLASSPATHS="$ {HADOOP_CLASSPATH} :$ {HADOOP_CONF_DIR} :$ {YARN_CONF_DIR} " if [ -n "$ {HBASE_CONF_DIR}" ]; then - # Setup the HBase classpath. - INTERNAL_HADOOP_CLASSPATHS="${INTERNAL_HADOOP_CLASSPATHS}:`hbase classpath`" + # Search hbase command in PATH. + HBASE_IN_PATH=$(PATH="${HBASE_HOME}/bin:$PATH" \ + which hbase 2>/dev/null) + # Whether the hbase command is found. + if [ -f "${HBASE_IN_PATH}" ]; then + # The hbase command is found, then setup the HBase classpath. + INTERNAL_HADOOP_CLASSPATHS="${INTERNAL_HADOOP_CLASSPATHS}:`${HBASE_IN_PATH} classpath`" + else + echo "The HBASE_IN_PATH is setted but can't setup HBase classpath. Please verify hbase command exists." — End diff – Perhaps more like "HBASE_CONF_DIR=\"${HBASE_CONF_DIR} \" is set but 'hbase' command was not found in \"$ {HBASE_PATH} \" so classpath could not be updated."? I assumed HBASE_PATH="$ {HBASE_HOME} /bin:$PATH" as a local variable.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user lingjinjiang commented on the issue:

          https://github.com/apache/flink/pull/3711

          @greghogan , I add some comments and log the message about hbase command not found.

          Show
          githubbot ASF GitHub Bot added a comment - Github user lingjinjiang commented on the issue: https://github.com/apache/flink/pull/3711 @greghogan , I add some comments and log the message about hbase command not found.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user greghogan commented on the issue:

          https://github.com/apache/flink/pull/3711

          Would it be better to replace the outer if-statement (line 312) with the test for the presence of `hbase`? And to log a message that `HBASE_CONF_DIR` is defined but `hbase` was not found so no HBase jars are included on the classpath?

          Show
          githubbot ASF GitHub Bot added a comment - Github user greghogan commented on the issue: https://github.com/apache/flink/pull/3711 Would it be better to replace the outer if-statement (line 312) with the test for the presence of `hbase`? And to log a message that `HBASE_CONF_DIR` is defined but `hbase` was not found so no HBase jars are included on the classpath?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user lingjinjiang commented on the issue:

          https://github.com/apache/flink/pull/3711

          Hi @greghogan , In the config.sh, there is a if statement identifies whether HBASE_CONF_DIR is setted.
          If the HBASE_CONF_DIR is setted, then will use the "hbase classpath" command to add the hbase's classpath to flink's classpath.
          But it does not check whether the "hbase" command is existed before using it.
          Then in the host where I'm using flink, another application need this env variable and there is no "hbase" command, then the error happen.

          Show
          githubbot ASF GitHub Bot added a comment - Github user lingjinjiang commented on the issue: https://github.com/apache/flink/pull/3711 Hi @greghogan , In the config.sh, there is a if statement identifies whether HBASE_CONF_DIR is setted. If the HBASE_CONF_DIR is setted, then will use the "hbase classpath" command to add the hbase's classpath to flink's classpath. But it does not check whether the "hbase" command is existed before using it. Then in the host where I'm using flink, another application need this env variable and there is no "hbase" command, then the error happen.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user greghogan commented on the issue:

          https://github.com/apache/flink/pull/3711

          Hi @lingjinjiang, I am not familiar with configuring HBase for Flink and we don't seem to have documentation on this. What does it mean to have `HBASE_CONF_DIR` set but not have `hbase` available?

          Show
          githubbot ASF GitHub Bot added a comment - Github user greghogan commented on the issue: https://github.com/apache/flink/pull/3711 Hi @lingjinjiang, I am not familiar with configuring HBase for Flink and we don't seem to have documentation on this. What does it mean to have `HBASE_CONF_DIR` set but not have `hbase` available?
          Hide
          lingjinjiang Jinjiang Ling added a comment -

          Hi,Greg Hogan, I have attach a patch and opened a pull request. Can you take a review?

          Show
          lingjinjiang Jinjiang Ling added a comment - Hi, Greg Hogan , I have attach a patch and opened a pull request. Can you take a review?
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user lingjinjiang opened a pull request:

          https://github.com/apache/flink/pull/3711

          FLINK-6286 [script] Fix the hbase command not found error

          When using flink with the HBASE_CONF_DIR env variable and don't install hbase, then will get the "hbase command not found" error.

          https://issues.apache.org/jira/browse/FLINK-6286

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/lingjinjiang/flink master

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3711.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3711


          commit f613e4b221bf4583574be11fe4c3fd5558290545
          Author: lingjinjiang <lingjinjiang@gmail.com>
          Date: 2017-04-12T02:53:52Z

          FLINK-6286 [script] Fix the hbase command not found error


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user lingjinjiang opened a pull request: https://github.com/apache/flink/pull/3711 FLINK-6286 [script] Fix the hbase command not found error When using flink with the HBASE_CONF_DIR env variable and don't install hbase, then will get the "hbase command not found" error. https://issues.apache.org/jira/browse/FLINK-6286 You can merge this pull request into a Git repository by running: $ git pull https://github.com/lingjinjiang/flink master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3711.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3711 commit f613e4b221bf4583574be11fe4c3fd5558290545 Author: lingjinjiang <lingjinjiang@gmail.com> Date: 2017-04-12T02:53:52Z FLINK-6286 [script] Fix the hbase command not found error
          Hide
          lingjinjiang Jinjiang Ling added a comment -

          Remove a unnecessary echo command.

          Show
          lingjinjiang Jinjiang Ling added a comment - Remove a unnecessary echo command.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user lingjinjiang closed the pull request at:

          https://github.com/apache/flink/pull/3705

          Show
          githubbot ASF GitHub Bot added a comment - Github user lingjinjiang closed the pull request at: https://github.com/apache/flink/pull/3705
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user lingjinjiang opened a pull request:

          https://github.com/apache/flink/pull/3705

          FLINK-6286 Fix the hbase command not found error

          Fix the hbase command not found error.
          https://issues.apache.org/jira/browse/FLINK-6286

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/lingjinjiang/flink master

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3705.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3705


          commit 7cd62bedac81ce078ba0933722026f260ed2eedd
          Author: lingjinjiang <lingjinjiang@gmail.com>
          Date: 2017-04-10T10:50:50Z

          FLINK-6286 Fix the hbase command not found error


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user lingjinjiang opened a pull request: https://github.com/apache/flink/pull/3705 FLINK-6286 Fix the hbase command not found error Fix the hbase command not found error. https://issues.apache.org/jira/browse/FLINK-6286 You can merge this pull request into a Git repository by running: $ git pull https://github.com/lingjinjiang/flink master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3705.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3705 commit 7cd62bedac81ce078ba0933722026f260ed2eedd Author: lingjinjiang <lingjinjiang@gmail.com> Date: 2017-04-10T10:50:50Z FLINK-6286 Fix the hbase command not found error

            People

            • Assignee:
              lingjinjiang Jinjiang Ling
              Reporter:
              lingjinjiang Jinjiang Ling
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development