HCatalog
  1. HCatalog
  2. HCATALOG-166

Making configuration/installation automatable

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.3
    • Fix Version/s: 0.3
    • Component/s: None

      Description

      Currently, we still have installation instructions which ask people to edit the hive-site.xml,etc. This is not really feasible for automatability. Hadoop install/setup scripts have recently started taking command line arguments to set up appropriately.

      We should also update our scripts accordingly.

      1. hive-site.xml.template
        4 kB
        Sushanth Sowmyan
      2. hcat-env.sh.template
        0.9 kB
        Sushanth Sowmyan
      3. hcat-configure.sh
        10 kB
        Sushanth Sowmyan
      4. 166.patch
        15 kB
        Vikram Dixit K
      5. 166_1.patch
        16 kB
        Vikram Dixit K

        Issue Links

          Activity

          Hide
          Alan Gates added a comment -

          Issue closed with 0.4 release.

          Show
          Alan Gates added a comment - Issue closed with 0.4 release.
          Hide
          Sushanth Sowmyan added a comment -

          Committed to 0.3 and trunk. Thanks, Vikram!

          Show
          Sushanth Sowmyan added a comment - Committed to 0.3 and trunk. Thanks, Vikram!
          Hide
          Sushanth Sowmyan added a comment -

          +1, looks good, installed and tested basic operation.

          Show
          Sushanth Sowmyan added a comment - +1, looks good, installed and tested basic operation.
          Hide
          Vikram Dixit K added a comment -

          Made changes to the installation shell script provided above to work for rpm and deb packages. Minor changes in the rpm-spec file and the deb postinst file. Replaced file update-hcatalog-env.sh contents with those of the hcat-configure.sh. In the file hcat_server.sh changed HIVE_SITE_XML to point to correct hive location. In the file hcat-config.sh changed DEFAULT_CONF_DIR to point to the right location. Also changed the permissions for the hive-site.xml in the above script from 700 to 755.

          Show
          Vikram Dixit K added a comment - Made changes to the installation shell script provided above to work for rpm and deb packages. Minor changes in the rpm-spec file and the deb postinst file. Replaced file update-hcatalog-env.sh contents with those of the hcat-configure.sh. In the file hcat_server.sh changed HIVE_SITE_XML to point to correct hive location. In the file hcat-config.sh changed DEFAULT_CONF_DIR to point to the right location. Also changed the permissions for the hive-site.xml in the above script from 700 to 755.
          Hide
          Vikram Dixit K added a comment -

          Patch updated for deb and for a typo.

          Show
          Vikram Dixit K added a comment - Patch updated for deb and for a typo.
          Hide
          Vikram Dixit K added a comment -

          Patch attached.

          Show
          Vikram Dixit K added a comment - Patch attached.
          Hide
          Sushanth Sowmyan added a comment -

          Also,

          a) Has --install option to do things like setting up log dir, pid dir,etc (also ignores --output-env and --output-site when --install is specified, because it places them automatically in the conf dir) – this is so we can replace hcat_server_install.sh functionality
          b) has --kerberos_realm , defaults to hcat/$

          {METASTORE_HOST}

          @$

          {REALM}

          if realm is specified and --kerberos_principal is not specified.
          (metastore host determined elsewhere)
          c) --dbname option added

          Show
          Sushanth Sowmyan added a comment - Also, a) Has --install option to do things like setting up log dir, pid dir,etc (also ignores --output-env and --output-site when --install is specified, because it places them automatically in the conf dir) – this is so we can replace hcat_server_install.sh functionality b) has --kerberos_realm , defaults to hcat/$ {METASTORE_HOST} @$ {REALM} if realm is specified and --kerberos_principal is not specified. (metastore host determined elsewhere) c) --dbname option added
          Hide
          Sushanth Sowmyan added a comment -

          Usage:

          sudo ./hcat-configure.sh
          --proto-env=./hcat-env.sh.template
          --proto-site=./hive-site.xml.template --prefix=/usr
          --pid-dir=/var/run/hcatalog --log-dir=/var/log/hcatalog
          --conf-dir=/etc/hcatalog
          --dbjars=/opt/mysql-connector-java-3.1.14
          --dbhost=dbhost.blah.net --dbuser=hcat
          --dbpasswd=deebee --warehouse-dir=/user/hive/warehouse
          --metastore-server=metastore.blah.net

          Above mimics actual values from rpm-based installation on box using an
          unsecure cluster, and replaces hcat-env.sh and hive-site.xml in
          CONF_DIR (be careful!) Alternatively, you can use the --output-env and
          --output-site parameters and move yourself if you want to verify/test

          Show
          Sushanth Sowmyan added a comment - Usage: sudo ./hcat-configure.sh --proto-env=./hcat-env.sh.template --proto-site=./hive-site.xml.template --prefix=/usr --pid-dir=/var/run/hcatalog --log-dir=/var/log/hcatalog --conf-dir=/etc/hcatalog --dbjars=/opt/mysql-connector-java-3.1.14 --dbhost=dbhost.blah.net --dbuser=hcat --dbpasswd=deebee --warehouse-dir=/user/hive/warehouse --metastore-server=metastore.blah.net Above mimics actual values from rpm-based installation on box using an unsecure cluster, and replaces hcat-env.sh and hive-site.xml in CONF_DIR (be careful!) Alternatively, you can use the --output-env and --output-site parameters and move yourself if you want to verify/test
          Hide
          Sushanth Sowmyan added a comment -

          I took an initial stab at this, and this is tested/deployed for rpm-based installations, as a replacement for our current hcat-env-update.sh. We need to test with non-rpm-based installs, and if so, make this change.

          Show
          Sushanth Sowmyan added a comment - I took an initial stab at this, and this is tested/deployed for rpm-based installations, as a replacement for our current hcat-env-update.sh. We need to test with non-rpm-based installs, and if so, make this change.

            People

            • Assignee:
              Vikram Dixit K
              Reporter:
              Sushanth Sowmyan
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development