Bigtop
  1. Bigtop
  2. BIGTOP-471

hive server setup after installation fails on creating /var/lock/subsys directory

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 0.3.0
    • Fix Version/s: 0.5.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      ubuntu 10.04 LTS

      Description

      Error message on sudo apt-get install hive-server

      Setting up hive-server (0.8.1-1) ...
      update-rc.d: warning: hive-server start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (3 4 5)
      update-rc.d: warning: hive-server stop runlevel arguments (0 1 6) do not match LSB Default-Stop values (0 1 2 6)

      • Starting (hive-server):
        touch: cannot touch `/var/lock/subsys/hive-server': No such file or directory

      if you create /var/lock/subsys, installation runs ok.

        Activity

        Hide
        Roman Shaposhnik added a comment -

        Great point! Looks like we've got a couple of those references to /var/lock going on in Debian scripts. Care to submit a patch?

        Show
        Roman Shaposhnik added a comment - Great point! Looks like we've got a couple of those references to /var/lock going on in Debian scripts. Care to submit a patch?
        Hide
        doug chang added a comment - - edited

        ok, note to self: convention is HADOOP_COMPONENT.dirs where all the subdirectories get added
        hive.dirs needs /var/lock/subsys

        hadoop.dirs needs /var/lib/hadoop/cache/hadoop/dfs and may need to add the line to hadoop.postinst for the chown like this:

        chown root:hadoop /var/lib/hadoop/ /var/lib/hadoop/cache/ /var/lib/hadoop/cache/hadoop/ /var/lib/hadoop/cache/hadoop/dfs

        flume needs to create /var/lock/subsys/flume-node; depends if you install hive first then you wont see this error message, need to look at all the scripts to make sure order doesnt matter. both flume and hive need /var/lock/subsys. verify other components and add. problem is if you install hive first then you never see the error messages. alternatively test each component separately for /var/lock/subsys requirement. tricky.

        • Starting Flume node daemon (flume-node):
          touch: cannot touch `/var/lock/subsys/flume-node': No such file or directory

        have to test – here is proof fix works:
        ubuntu@ip-10-243-135-103:~$ ls /var/lock/
        ubuntu@ip-10-243-135-103:~$ sudo apt-get install hive
        Reading package lists... Done
        Building dependency tree
        Reading state information... Done
        The following NEW packages will be installed:
        hive
        0 upgraded, 1 newly installed, 0 to remove and 25 not upgraded.
        Need to get 0B/18.3MB of archives.
        After this operation, 22.5MB of additional disk space will be used.
        WARNING: The following packages cannot be authenticated!
        hive
        Install these packages without verification [y/N]? y
        Selecting previously deselected package hive.
        (Reading database ... 80531 files and directories currently installed.)
        Unpacking hive (from .../h/hive/hive_0.8.1-1_all.deb) ...
        Processing triggers for man-db ...
        Setting up hive (0.8.1-1) ...
        update-alternatives: using /etc/hive/conf.dist to provide /etc/hive/conf (hive-conf) in auto mode.

        ubuntu@ip-10-243-135-103:~$ ls /var/lock/
        subsys
        ubuntu@ip-10-243-135-103:~$

        Show
        doug chang added a comment - - edited ok, note to self: convention is HADOOP_COMPONENT.dirs where all the subdirectories get added hive.dirs needs /var/lock/subsys hadoop.dirs needs /var/lib/hadoop/cache/hadoop/dfs and may need to add the line to hadoop.postinst for the chown like this: chown root:hadoop /var/lib/hadoop/ /var/lib/hadoop/cache/ /var/lib/hadoop/cache/hadoop/ /var/lib/hadoop/cache/hadoop/dfs flume needs to create /var/lock/subsys/flume-node; depends if you install hive first then you wont see this error message, need to look at all the scripts to make sure order doesnt matter. both flume and hive need /var/lock/subsys. verify other components and add. problem is if you install hive first then you never see the error messages. alternatively test each component separately for /var/lock/subsys requirement. tricky. Starting Flume node daemon (flume-node): touch: cannot touch `/var/lock/subsys/flume-node': No such file or directory have to test – here is proof fix works: ubuntu@ip-10-243-135-103:~$ ls /var/lock/ ubuntu@ip-10-243-135-103:~$ sudo apt-get install hive Reading package lists... Done Building dependency tree Reading state information... Done The following NEW packages will be installed: hive 0 upgraded, 1 newly installed, 0 to remove and 25 not upgraded. Need to get 0B/18.3MB of archives. After this operation, 22.5MB of additional disk space will be used. WARNING: The following packages cannot be authenticated! hive Install these packages without verification [y/N] ? y Selecting previously deselected package hive. (Reading database ... 80531 files and directories currently installed.) Unpacking hive (from .../h/hive/hive_0.8.1-1_all.deb) ... Processing triggers for man-db ... Setting up hive (0.8.1-1) ... update-alternatives: using /etc/hive/conf.dist to provide /etc/hive/conf (hive-conf) in auto mode. ubuntu@ip-10-243-135-103:~$ ls /var/lock/ subsys ubuntu@ip-10-243-135-103:~$
        Hide
        Roman Shaposhnik added a comment -

        Two comments re: patch:

        1. is there any need for /var/lib/hadoop/cache/hadoop/dfs as far as Hive is concerned?
        2. I'm not sure /var/lock/subsys is valid on Ubuntu, so perhaps we have to modify the init.d script to be different on RedHat and Ubuntu
        Show
        Roman Shaposhnik added a comment - Two comments re: patch: is there any need for /var/lib/hadoop/cache/hadoop/dfs as far as Hive is concerned? I'm not sure /var/lock/subsys is valid on Ubuntu, so perhaps we have to modify the init.d script to be different on RedHat and Ubuntu
        Hide
        doug chang added a comment - - edited

        Thanks Roman
        1) no, /var/lib/hadoop/cache/hadoop/dfs was a separate bug. Sorry, no longer needed....
        2) hive-server looks for directory /var/lock/subsys
        3) after starting hive-server, pid creation error on locating directory /var/run/hive
        ubuntu@ip-10-62-5-221:~$ /sbin/start-stop-daemon: Unable to open pidfile '/var/run/hive/hive-server.pid' for writing: No such file or directory (No such file or directory)
        3) flume is an issue also with /var/lock/subsys:
        touch: cannot touch `/var/lock/subsys/flume-node': No such file or directory

        Show
        doug chang added a comment - - edited Thanks Roman 1) no, /var/lib/hadoop/cache/hadoop/dfs was a separate bug. Sorry, no longer needed.... 2) hive-server looks for directory /var/lock/subsys 3) after starting hive-server, pid creation error on locating directory /var/run/hive ubuntu@ip-10-62-5-221:~$ /sbin/start-stop-daemon: Unable to open pidfile '/var/run/hive/hive-server.pid' for writing: No such file or directory (No such file or directory) 3) flume is an issue also with /var/lock/subsys: touch: cannot touch `/var/lock/subsys/flume-node': No such file or directory
        Hide
        Anatoli Fomenko added a comment -

        It seems like the problem stated in the description is not reproducible neither by
        `sudo apt-get install hive-server`

        Setting up hive-server (0.9.0.7-1) ...
        Installing new version of config file /etc/init.d/hive-server ...
        update-rc.d: warning: hive-server start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (3 4 5)
        update-rc.d: warning: hive-server stop runlevel arguments (0 1 6) do not match LSB Default-Stop values (0 1 2 6)

        • Starting (hive-server):
          Setting up hive-metastore (0.9.0.7-1) ...

        nor `sudo dpkg -i ...` of the hive built from the current bigtop trunk (0.5.0 in progress):

        Setting up hive-server (0.9.0-1) ...
        Installing new version of config file /etc/init.d/hive-server ...
        update-rc.d: warning: hive-server start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (3 4 5)
        update-rc.d: warning: hive-server stop runlevel arguments (0 1 6) do not match LSB Default-Stop values (0 1 2 6)

        • Starting (hive-server):
          Setting up hive-hbase (0.9.0-1) ...

        Perhaps, this issue is a candidate for a closure.

        Show
        Anatoli Fomenko added a comment - It seems like the problem stated in the description is not reproducible neither by `sudo apt-get install hive-server` Setting up hive-server (0.9.0.7-1) ... Installing new version of config file /etc/init.d/hive-server ... update-rc.d: warning: hive-server start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (3 4 5) update-rc.d: warning: hive-server stop runlevel arguments (0 1 6) do not match LSB Default-Stop values (0 1 2 6) Starting (hive-server): Setting up hive-metastore (0.9.0.7-1) ... nor `sudo dpkg -i ...` of the hive built from the current bigtop trunk (0.5.0 in progress): Setting up hive-server (0.9.0-1) ... Installing new version of config file /etc/init.d/hive-server ... update-rc.d: warning: hive-server start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (3 4 5) update-rc.d: warning: hive-server stop runlevel arguments (0 1 6) do not match LSB Default-Stop values (0 1 2 6) Starting (hive-server): Setting up hive-hbase (0.9.0-1) ... Perhaps, this issue is a candidate for a closure.
        Hide
        Roman Shaposhnik added a comment -

        Agreed with Anatoli. At least for trunk. If the problem persists in some other useful branch (like 0.3) feel free to reopen.

        Show
        Roman Shaposhnik added a comment - Agreed with Anatoli. At least for trunk. If the problem persists in some other useful branch (like 0.3) feel free to reopen.

          People

          • Assignee:
            doug chang
            Reporter:
            doug chang
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development