Bigtop
  1. Bigtop
  2. BIGTOP-236

The user installed by hbase gets an incorrect home directory

    Details

    • Type: Task Task
    • Status: Resolved
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 0.1.0, 0.7.0
    • Fix Version/s: 0.8.0
    • Component/s: general
    • Labels:

      Description

      HBase installs its user with a home directory of /var/run/hbase which is inconsistent with other components and should really be /var/lib/hbase

      1. BIGTOP-236_3.patch
        3 kB
        Yi Zhou
      2. BIGTOP-236.patch
        1 kB
        Yi Zhou

        Activity

        Hide
        Yi Zhou added a comment -

        Attached patch for BIGTOP-236

        Show
        Yi Zhou added a comment - Attached patch for BIGTOP-236
        Hide
        Yi Zhou added a comment -

        Details for fixed:
        The home directory of 'hbase' user will be pointed to '/var/lib/hbase' which is created via adding '-m' option.

        Show
        Yi Zhou added a comment - Details for fixed: The home directory of 'hbase' user will be pointed to '/var/lib/hbase' which is created via adding '-m' option.
        Hide
        Yi Zhou added a comment -

        Submit Patch for BIGTOP-236

        Show
        Yi Zhou added a comment - Submit Patch for BIGTOP-236
        Hide
        Roman Shaposhnik added a comment -

        Yi Zhou a couple of points:

        1. I believe that we make /var/run/<component> a home dir for any component that doesn't store state in the local filesystem
        2. as a consequence of #1 /var/lib home directories must be packaged inside of RPMs and not created on the fly
        Show
        Roman Shaposhnik added a comment - Yi Zhou a couple of points: I believe that we make /var/run/<component> a home dir for any component that doesn't store state in the local filesystem as a consequence of #1 /var/lib home directories must be packaged inside of RPMs and not created on the fly
        Hide
        Yi Zhou added a comment -

        Hi Roman,
        For HBase compenont,we observed that its hbase*.pid file will be stored in /var/run/hbase directory, but have no found any data in /var/lib/hbase. Now we only will be changing /var/run/hbase to /var/lib/hbase as home directory without creating on the fly in hbase.spec ?

        Show
        Yi Zhou added a comment - Hi Roman, For HBase compenont,we observed that its hbase*.pid file will be stored in /var/run/hbase directory, but have no found any data in /var/lib/hbase. Now we only will be changing /var/run/hbase to /var/lib/hbase as home directory without creating on the fly in hbase.spec ?
        Hide
        Roman Shaposhnik added a comment -

        Yi Zhou sorry forgot to ask: what problem are we trying to solve here? How's the current layout inconvenient?

        Show
        Roman Shaposhnik added a comment - Yi Zhou sorry forgot to ask: what problem are we trying to solve here? How's the current layout inconvenient?
        Hide
        Yi Zhou added a comment - - edited

        As per Peter's description, the user of 'hbase' gets '/var/run/hbase' as home directory shown in /etc/passwd after HBase component be installed via bigtop. Now we change the home directory of 'hbase' user to '/var/lib/hbase' via hbase.mk in bigtop packaging.

        Please see below details for example in /etc/passwd:
        Original:
        hbase:x:493:493:HBase:/var/run/hbase:/sbin/nologin
        Changed:
        hbase:x:493:493:HBase:/var/lib/hbase:/sbin/nologin

        Show
        Yi Zhou added a comment - - edited As per Peter's description, the user of 'hbase' gets '/var/run/hbase' as home directory shown in /etc/passwd after HBase component be installed via bigtop. Now we change the home directory of 'hbase' user to '/var/lib/hbase' via hbase.mk in bigtop packaging. Please see below details for example in /etc/passwd: Original: hbase:x:493:493:HBase:/var/run/hbase:/sbin/nologin Changed: hbase:x:493:493:HBase:/var/lib/hbase:/sbin/nologin
        Hide
        Roman Shaposhnik added a comment -

        You still didn't answer my question as to what is the exact problem you're trying to solve.

        Show
        Roman Shaposhnik added a comment - You still didn't answer my question as to what is the exact problem you're trying to solve.
        Hide
        Yi Zhou added a comment -

        Hi Roman,
        The patch is to fix the home directory of 'hbase' user changed from '/var/run/hbase' to '/var/lib/hbase' when the 'hbase' user is created.
        In hbase.mk file, we change the home directory to '/var/lib/hbase' like below:
        getent passwd hbase 2>&1 > /dev/null || /usr/sbin/useradd -c "HBase" -s /sbin/nologin -g hbase -r -d /var/lib/hbase hbase 2> /dev/null || :

        Show
        Yi Zhou added a comment - Hi Roman, The patch is to fix the home directory of 'hbase' user changed from '/var/run/hbase' to '/var/lib/hbase' when the 'hbase' user is created. In hbase.mk file, we change the home directory to '/var/lib/hbase' like below: getent passwd hbase 2>&1 > /dev/null || /usr/sbin/useradd -c "HBase" -s /sbin/nologin -g hbase -r -d /var/lib/hbase hbase 2> /dev/null || :
        Hide
        Yi Zhou added a comment -

        From Perter description, he recognized that the home directory of 'hbase' user should really be /var/lib/hbase.

        Show
        Yi Zhou added a comment - From Perter description, he recognized that the home directory of 'hbase' user should really be /var/lib/hbase.
        Hide
        Roman Shaposhnik added a comment -

        May be it is a good question to ask Peter Linnell then: was it just an inconsistency issue or functionality?

        Show
        Roman Shaposhnik added a comment - May be it is a good question to ask Peter Linnell then: was it just an inconsistency issue or functionality?
        Hide
        Mark Grover added a comment -

        I agree with the intent of this JIRA and the issue. On newer OS's /var/run is tempfs and it makes sense for the user's home directory to persist across reboots, and be in /var/lib. We make /var/lib as home for most of our components anyways, so it's good to be consistent anyways.

        However, I do think there are a few other places we will have to update in addition to what the patch proposes

        For the service init script:
        bigtop-packages/src/common/hbase/hbase.svc
        For debian side of things:
        bigtop-packages/src/deb/hbase/hbase.dirs
        bigtop-packages/src/deb/hbase/hbase.postinst
        Adding an entry to the %files macro:
        bigtop-packages/src/rpm/hbase/SPECS/hbase.spec

        I have a patch ready but I want to give the opportunity to Yi first in case he wants to update his patch to make these additional changes. Please feel free to go ahead and make these changes.

        Show
        Mark Grover added a comment - I agree with the intent of this JIRA and the issue. On newer OS's /var/run is tempfs and it makes sense for the user's home directory to persist across reboots, and be in /var/lib. We make /var/lib as home for most of our components anyways, so it's good to be consistent anyways. However, I do think there are a few other places we will have to update in addition to what the patch proposes For the service init script: bigtop-packages/src/common/hbase/hbase.svc For debian side of things: bigtop-packages/src/deb/hbase/hbase.dirs bigtop-packages/src/deb/hbase/hbase.postinst Adding an entry to the %files macro: bigtop-packages/src/rpm/hbase/SPECS/hbase.spec I have a patch ready but I want to give the opportunity to Yi first in case he wants to update his patch to make these additional changes. Please feel free to go ahead and make these changes.
        Hide
        Yi Zhou added a comment - - edited

        Thanks Mark and Roman for given suggestions and comments.

        Hi Mark,
        Now I am trying to make changes as per your instructions, below is my changed for details.
        I am not sure about these changed in hbase.spec. Could you kindly pls. help to review the changed ?

        #1 bigtop-packages/src/common/hbase/hbase.svc( change to /var/lib/hbase)
        WORKING_DIR="/var/lib/hbase"

        #2 bigtop-packages/src/deb/hbase/hbase.dirs ( add /var/lib/hbase entry)
        /var/lib/hbase

        #3 bigtop-packages/src/deb/hbase/hbase.postinst ( add /var/lib/hbase path)
        chown hbase:hbase -R /var/run/hbase /var/log/hbase /var/lib/hbase

        #4 bigtop-packages/src/rpm/hbase/SPECS/hbase.spec( Changed from /var/run/hbase to /var/lib/hbase and add an entry in %file):
        1) getent passwd hbase 2>&1 > /dev/null || /usr/sbin/useradd -c "HBase" -s /sbin/nologin -g hbase -r -d /var/lib/hbase hbase 2> /dev/null || :
        2) Add a entry in %file marco section:
        %dir /var/lib/hbase

        Show
        Yi Zhou added a comment - - edited Thanks Mark and Roman for given suggestions and comments. Hi Mark, Now I am trying to make changes as per your instructions, below is my changed for details. I am not sure about these changed in hbase.spec. Could you kindly pls. help to review the changed ? #1 bigtop-packages/src/common/hbase/hbase.svc( change to /var/lib/hbase) WORKING_DIR="/var/lib/hbase" #2 bigtop-packages/src/deb/hbase/hbase.dirs ( add /var/lib/hbase entry) /var/lib/hbase #3 bigtop-packages/src/deb/hbase/hbase.postinst ( add /var/lib/hbase path) chown hbase:hbase -R /var/run/hbase /var/log/hbase /var/lib/hbase #4 bigtop-packages/src/rpm/hbase/SPECS/hbase.spec( Changed from /var/run/hbase to /var/lib/hbase and add an entry in %file): 1) getent passwd hbase 2>&1 > /dev/null || /usr/sbin/useradd -c "HBase" -s /sbin/nologin -g hbase -r -d /var/lib/hbase hbase 2> /dev/null || : 2) Add a entry in %file marco section: %dir /var/lib/hbase
        Hide
        Mark Grover added a comment -

        Apologies for the delay, Yi.

        This is looking good! The only thing I would add is you may need an extra command for "install -d" on that directory similar to here: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/rpm/hbase/SPECS/hbase.spec#L287

        Apart from that looks good, please upload a patch.

        Show
        Mark Grover added a comment - Apologies for the delay, Yi. This is looking good! The only thing I would add is you may need an extra command for "install -d" on that directory similar to here: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/rpm/hbase/SPECS/hbase.spec#L287 Apart from that looks good, please upload a patch.
        Hide
        Yi Zhou added a comment - - edited

        Submit patch(BIGTOP-236_3.patch) for BIGTOP-236

        Show
        Yi Zhou added a comment - - edited Submit patch( BIGTOP-236 _3.patch) for BIGTOP-236
        Hide
        Yi Zhou added a comment -

        Thanks Mark ! Appreciate your great hints/helps !
        I added "install -d " in hbase.spec and upload path. Please kindly review it !

        Show
        Yi Zhou added a comment - Thanks Mark ! Appreciate your great hints/helps ! I added "install -d " in hbase.spec and upload path. Please kindly review it !
        Hide
        Roman Shaposhnik added a comment -

        Mark Grover any chance you can review/commit?

        Show
        Roman Shaposhnik added a comment - Mark Grover any chance you can review/commit?
        Hide
        Mark Grover added a comment -

        +1

        Thanks for taking care of this, Yi!

        Show
        Mark Grover added a comment - +1 Thanks for taking care of this, Yi!

          People

          • Assignee:
            Yi Zhou
            Reporter:
            Peter Linnell
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development