Bigtop
  1. Bigtop
  2. BIGTOP-1034

Custom user's home directory's creation fails in init-hdfs.sh

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.7.0
    • Component/s: debian, rpm

      Description

      The init-hdfs.sh script has set -ex at the top for a good reason. However, the script itself when creating a custom user's home directory on HDFS, does a ls to see if the directory already exists: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/init-hdfs.sh#L91

      This fails if the directory doesn't exist which makes the entire script fail. This check could be enclosed so it doesn't make the script fail.

      1. BIGTOP-1034.1.patch
        1 kB
        Mark Grover
      2. BIGTOP-1034.2.patch
        1 kB
        Henry Wang
      3. init-hdfs.patch
        0.8 kB
        Henry Wang

        Activity

        Hide
        Konstantin Boudnik added a comment -

        I we'd better off by moving forward with the way of shaping up the HDFS with the groovy + tarball approach, that is faster and - more importantly - idempotent.

        The current script is a dead end - it is too slow and a way overcomplicated.

        Show
        Konstantin Boudnik added a comment - I we'd better off by moving forward with the way of shaping up the HDFS with the groovy + tarball approach, that is faster and - more importantly - idempotent. The current script is a dead end - it is too slow and a way overcomplicated.
        Hide
        Mark Grover added a comment -

        Cos, I completely agree with you that the groovy + tarball approach is the right long term approach. However, given that the groovy + tarball script is not complete yet and that this is a really minor bug fix, I am attaching a patch for this JIRA. Would you please be able to review it?

        Show
        Mark Grover added a comment - Cos, I completely agree with you that the groovy + tarball approach is the right long term approach. However, given that the groovy + tarball script is not complete yet and that this is a really minor bug fix, I am attaching a patch for this JIRA. Would you please be able to review it?
        Hide
        Mark Grover added a comment -

        Henry, your patch looks good.

        May I request you to provide a patch via git format-patch, please?
        This the workflow I personally follow:
        1. Ensure that you have all of your change as 1 commit which has the correct commit message - something like BIGTOP-1034: Custom user's home directory's creation fails in init-hdfs.sh
        2. Then run a command like: git format-patch HEAD^..HEAD --stdout > BIGTOP-1034.2.patch
        3. Upload the BIGTOP-1034.2.patch file to this JIRA
        git format-patch works better for all of us because it's easy to attribute the commit to you in the git history since it contains your name and email address.

        Show
        Mark Grover added a comment - Henry, your patch looks good. May I request you to provide a patch via git format-patch, please? This the workflow I personally follow: 1. Ensure that you have all of your change as 1 commit which has the correct commit message - something like BIGTOP-1034 : Custom user's home directory's creation fails in init-hdfs.sh 2. Then run a command like: git format-patch HEAD^..HEAD --stdout > BIGTOP-1034 .2.patch 3. Upload the BIGTOP-1034 .2.patch file to this JIRA git format-patch works better for all of us because it's easy to attribute the commit to you in the git history since it contains your name and email address.
        Hide
        Mark Grover added a comment -

        +1 and committed!

        Show
        Mark Grover added a comment - +1 and committed!

          People

          • Assignee:
            Mark Grover
            Reporter:
            Mark Grover
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development