Bigtop
  1. Bigtop
  2. BIGTOP-471

hive server setup after installation fails on creating /var/lock/subsys directory

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 0.3.0
    • Fix Version/s: 0.5.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      ubuntu 10.04 LTS

      Description

      Error message on sudo apt-get install hive-server

      Setting up hive-server (0.8.1-1) ...
      update-rc.d: warning: hive-server start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (3 4 5)
      update-rc.d: warning: hive-server stop runlevel arguments (0 1 6) do not match LSB Default-Stop values (0 1 2 6)

      • Starting (hive-server):
        touch: cannot touch `/var/lock/subsys/hive-server': No such file or directory

      if you create /var/lock/subsys, installation runs ok.

        Activity

        doug chang created issue -
        Hide
        Roman Shaposhnik added a comment -

        Great point! Looks like we've got a couple of those references to /var/lock going on in Debian scripts. Care to submit a patch?

        Show
        Roman Shaposhnik added a comment - Great point! Looks like we've got a couple of those references to /var/lock going on in Debian scripts. Care to submit a patch?
        Roman Shaposhnik made changes -
        Field Original Value New Value
        Assignee doug chang [ dougc33333 ]
        Roman Shaposhnik made changes -
        Fix Version/s 0.4.0 [ 12318889 ]
        doug chang made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Hide
        doug chang added a comment - - edited

        ok, note to self: convention is HADOOP_COMPONENT.dirs where all the subdirectories get added
        hive.dirs needs /var/lock/subsys

        hadoop.dirs needs /var/lib/hadoop/cache/hadoop/dfs and may need to add the line to hadoop.postinst for the chown like this:

        chown root:hadoop /var/lib/hadoop/ /var/lib/hadoop/cache/ /var/lib/hadoop/cache/hadoop/ /var/lib/hadoop/cache/hadoop/dfs

        flume needs to create /var/lock/subsys/flume-node; depends if you install hive first then you wont see this error message, need to look at all the scripts to make sure order doesnt matter. both flume and hive need /var/lock/subsys. verify other components and add. problem is if you install hive first then you never see the error messages. alternatively test each component separately for /var/lock/subsys requirement. tricky.

        • Starting Flume node daemon (flume-node):
          touch: cannot touch `/var/lock/subsys/flume-node': No such file or directory

        have to test – here is proof fix works:
        ubuntu@ip-10-243-135-103:~$ ls /var/lock/
        ubuntu@ip-10-243-135-103:~$ sudo apt-get install hive
        Reading package lists... Done
        Building dependency tree
        Reading state information... Done
        The following NEW packages will be installed:
        hive
        0 upgraded, 1 newly installed, 0 to remove and 25 not upgraded.
        Need to get 0B/18.3MB of archives.
        After this operation, 22.5MB of additional disk space will be used.
        WARNING: The following packages cannot be authenticated!
        hive
        Install these packages without verification [y/N]? y
        Selecting previously deselected package hive.
        (Reading database ... 80531 files and directories currently installed.)
        Unpacking hive (from .../h/hive/hive_0.8.1-1_all.deb) ...
        Processing triggers for man-db ...
        Setting up hive (0.8.1-1) ...
        update-alternatives: using /etc/hive/conf.dist to provide /etc/hive/conf (hive-conf) in auto mode.

        ubuntu@ip-10-243-135-103:~$ ls /var/lock/
        subsys
        ubuntu@ip-10-243-135-103:~$

        Show
        doug chang added a comment - - edited ok, note to self: convention is HADOOP_COMPONENT.dirs where all the subdirectories get added hive.dirs needs /var/lock/subsys hadoop.dirs needs /var/lib/hadoop/cache/hadoop/dfs and may need to add the line to hadoop.postinst for the chown like this: chown root:hadoop /var/lib/hadoop/ /var/lib/hadoop/cache/ /var/lib/hadoop/cache/hadoop/ /var/lib/hadoop/cache/hadoop/dfs flume needs to create /var/lock/subsys/flume-node; depends if you install hive first then you wont see this error message, need to look at all the scripts to make sure order doesnt matter. both flume and hive need /var/lock/subsys. verify other components and add. problem is if you install hive first then you never see the error messages. alternatively test each component separately for /var/lock/subsys requirement. tricky. Starting Flume node daemon (flume-node): touch: cannot touch `/var/lock/subsys/flume-node': No such file or directory have to test – here is proof fix works: ubuntu@ip-10-243-135-103:~$ ls /var/lock/ ubuntu@ip-10-243-135-103:~$ sudo apt-get install hive Reading package lists... Done Building dependency tree Reading state information... Done The following NEW packages will be installed: hive 0 upgraded, 1 newly installed, 0 to remove and 25 not upgraded. Need to get 0B/18.3MB of archives. After this operation, 22.5MB of additional disk space will be used. WARNING: The following packages cannot be authenticated! hive Install these packages without verification [y/N] ? y Selecting previously deselected package hive. (Reading database ... 80531 files and directories currently installed.) Unpacking hive (from .../h/hive/hive_0.8.1-1_all.deb) ... Processing triggers for man-db ... Setting up hive (0.8.1-1) ... update-alternatives: using /etc/hive/conf.dist to provide /etc/hive/conf (hive-conf) in auto mode. ubuntu@ip-10-243-135-103:~$ ls /var/lock/ subsys ubuntu@ip-10-243-135-103:~$
        doug chang made changes -
        Attachment BIGTOP-471.patch.txt [ 12521808 ]
        Hide
        Roman Shaposhnik added a comment -

        Two comments re: patch:

        1. is there any need for /var/lib/hadoop/cache/hadoop/dfs as far as Hive is concerned?
        2. I'm not sure /var/lock/subsys is valid on Ubuntu, so perhaps we have to modify the init.d script to be different on RedHat and Ubuntu
        Show
        Roman Shaposhnik added a comment - Two comments re: patch: is there any need for /var/lib/hadoop/cache/hadoop/dfs as far as Hive is concerned? I'm not sure /var/lock/subsys is valid on Ubuntu, so perhaps we have to modify the init.d script to be different on RedHat and Ubuntu
        Hide
        doug chang added a comment - - edited

        Thanks Roman
        1) no, /var/lib/hadoop/cache/hadoop/dfs was a separate bug. Sorry, no longer needed....
        2) hive-server looks for directory /var/lock/subsys
        3) after starting hive-server, pid creation error on locating directory /var/run/hive
        ubuntu@ip-10-62-5-221:~$ /sbin/start-stop-daemon: Unable to open pidfile '/var/run/hive/hive-server.pid' for writing: No such file or directory (No such file or directory)
        3) flume is an issue also with /var/lock/subsys:
        touch: cannot touch `/var/lock/subsys/flume-node': No such file or directory

        Show
        doug chang added a comment - - edited Thanks Roman 1) no, /var/lib/hadoop/cache/hadoop/dfs was a separate bug. Sorry, no longer needed.... 2) hive-server looks for directory /var/lock/subsys 3) after starting hive-server, pid creation error on locating directory /var/run/hive ubuntu@ip-10-62-5-221:~$ /sbin/start-stop-daemon: Unable to open pidfile '/var/run/hive/hive-server.pid' for writing: No such file or directory (No such file or directory) 3) flume is an issue also with /var/lock/subsys: touch: cannot touch `/var/lock/subsys/flume-node': No such file or directory
        Gavin made changes -
        Workflow no-reopen-closed, patch-avail [ 12658704 ] patch-available, re-open possible [ 12666089 ]
        Roman Shaposhnik made changes -
        Fix Version/s 0.5.0 [ 12321865 ]
        Fix Version/s 0.4.0 [ 12318889 ]
        Hide
        Anatoli Fomenko added a comment -

        It seems like the problem stated in the description is not reproducible neither by
        `sudo apt-get install hive-server`

        Setting up hive-server (0.9.0.7-1) ...
        Installing new version of config file /etc/init.d/hive-server ...
        update-rc.d: warning: hive-server start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (3 4 5)
        update-rc.d: warning: hive-server stop runlevel arguments (0 1 6) do not match LSB Default-Stop values (0 1 2 6)

        • Starting (hive-server):
          Setting up hive-metastore (0.9.0.7-1) ...

        nor `sudo dpkg -i ...` of the hive built from the current bigtop trunk (0.5.0 in progress):

        Setting up hive-server (0.9.0-1) ...
        Installing new version of config file /etc/init.d/hive-server ...
        update-rc.d: warning: hive-server start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (3 4 5)
        update-rc.d: warning: hive-server stop runlevel arguments (0 1 6) do not match LSB Default-Stop values (0 1 2 6)

        • Starting (hive-server):
          Setting up hive-hbase (0.9.0-1) ...

        Perhaps, this issue is a candidate for a closure.

        Show
        Anatoli Fomenko added a comment - It seems like the problem stated in the description is not reproducible neither by `sudo apt-get install hive-server` Setting up hive-server (0.9.0.7-1) ... Installing new version of config file /etc/init.d/hive-server ... update-rc.d: warning: hive-server start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (3 4 5) update-rc.d: warning: hive-server stop runlevel arguments (0 1 6) do not match LSB Default-Stop values (0 1 2 6) Starting (hive-server): Setting up hive-metastore (0.9.0.7-1) ... nor `sudo dpkg -i ...` of the hive built from the current bigtop trunk (0.5.0 in progress): Setting up hive-server (0.9.0-1) ... Installing new version of config file /etc/init.d/hive-server ... update-rc.d: warning: hive-server start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (3 4 5) update-rc.d: warning: hive-server stop runlevel arguments (0 1 6) do not match LSB Default-Stop values (0 1 2 6) Starting (hive-server): Setting up hive-hbase (0.9.0-1) ... Perhaps, this issue is a candidate for a closure.
        Hide
        Roman Shaposhnik added a comment -

        Agreed with Anatoli. At least for trunk. If the problem persists in some other useful branch (like 0.3) feel free to reopen.

        Show
        Roman Shaposhnik added a comment - Agreed with Anatoli. At least for trunk. If the problem persists in some other useful branch (like 0.3) feel free to reopen.
        Roman Shaposhnik made changes -
        Status In Progress [ 3 ] Resolved [ 5 ]
        Resolution Cannot Reproduce [ 5 ]
        Roman Shaposhnik made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Anonymous made changes -
        Status Closed [ 6 ] Resolved [ 5 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open In Progress In Progress
        10d 18h 12m 1 doug chang 31/Mar/12 08:01
        In Progress In Progress Resolved Resolved
        170d 16h 15m 1 Roman Shaposhnik 18/Sep/12 00:16
        Resolved Resolved Closed Closed
        277d 38m 1 Roman Shaposhnik 22/Jun/13 00:55
        Closed Closed Resolved Resolved
        240d 2h 21m 1 17/Feb/14 02:16

          People

          • Assignee:
            doug chang
            Reporter:
            doug chang
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development