Bigtop
  1. Bigtop
  2. BIGTOP-1034

Custom user's home directory's creation fails in init-hdfs.sh

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.7.0
    • Component/s: Debian, RPM

      Description

      The init-hdfs.sh script has set -ex at the top for a good reason. However, the script itself when creating a custom user's home directory on HDFS, does a ls to see if the directory already exists: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/init-hdfs.sh#L91

      This fails if the directory doesn't exist which makes the entire script fail. This check could be enclosed so it doesn't make the script fail.

      1. init-hdfs.patch
        0.8 kB
        Henry Wang
      2. BIGTOP-1034.2.patch
        1 kB
        Henry Wang
      3. BIGTOP-1034.1.patch
        1 kB
        Mark Grover

        Activity

        Mark Grover created issue -
        Hide
        Konstantin Boudnik added a comment -

        I we'd better off by moving forward with the way of shaping up the HDFS with the groovy + tarball approach, that is faster and - more importantly - idempotent.

        The current script is a dead end - it is too slow and a way overcomplicated.

        Show
        Konstantin Boudnik added a comment - I we'd better off by moving forward with the way of shaping up the HDFS with the groovy + tarball approach, that is faster and - more importantly - idempotent. The current script is a dead end - it is too slow and a way overcomplicated.
        Konstantin Boudnik made changes -
        Field Original Value New Value
        Labels hackathon-7-2013
        Mark Grover made changes -
        Assignee Mark Grover [ mgrover ]
        Mark Grover made changes -
        Attachment BIGTOP-1034.1.patch [ 12594562 ]
        Mark Grover made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Mark Grover added a comment -

        Cos, I completely agree with you that the groovy + tarball approach is the right long term approach. However, given that the groovy + tarball script is not complete yet and that this is a really minor bug fix, I am attaching a patch for this JIRA. Would you please be able to review it?

        Show
        Mark Grover added a comment - Cos, I completely agree with you that the groovy + tarball approach is the right long term approach. However, given that the groovy + tarball script is not complete yet and that this is a really minor bug fix, I am attaching a patch for this JIRA. Would you please be able to review it?
        Henry Wang made changes -
        Attachment init-hdfs.patch [ 12594563 ]
        Hide
        Mark Grover added a comment -

        Henry, your patch looks good.

        May I request you to provide a patch via git format-patch, please?
        This the workflow I personally follow:
        1. Ensure that you have all of your change as 1 commit which has the correct commit message - something like BIGTOP-1034: Custom user's home directory's creation fails in init-hdfs.sh
        2. Then run a command like: git format-patch HEAD^..HEAD --stdout > BIGTOP-1034.2.patch
        3. Upload the BIGTOP-1034.2.patch file to this JIRA
        git format-patch works better for all of us because it's easy to attribute the commit to you in the git history since it contains your name and email address.

        Show
        Mark Grover added a comment - Henry, your patch looks good. May I request you to provide a patch via git format-patch, please? This the workflow I personally follow: 1. Ensure that you have all of your change as 1 commit which has the correct commit message - something like BIGTOP-1034 : Custom user's home directory's creation fails in init-hdfs.sh 2. Then run a command like: git format-patch HEAD^..HEAD --stdout > BIGTOP-1034 .2.patch 3. Upload the BIGTOP-1034 .2.patch file to this JIRA git format-patch works better for all of us because it's easy to attribute the commit to you in the git history since it contains your name and email address.
        Henry Wang made changes -
        Attachment BIGTOP-1034.2.patch [ 12594566 ]
        Hide
        Mark Grover added a comment -

        +1 and committed!

        Show
        Mark Grover added a comment - +1 and committed!
        Mark Grover made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Roman Shaposhnik made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Mark Grover
            Reporter:
            Mark Grover
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development