Index: src/docs/src/documentation/content/xdocs/site.xml =================================================================== --- src/docs/src/documentation/content/xdocs/site.xml (revision 1332394) +++ src/docs/src/documentation/content/xdocs/site.xml (working copy) @@ -41,7 +41,6 @@ - Index: src/docs/src/documentation/content/xdocs/rpminstall.xml =================================================================== --- src/docs/src/documentation/content/xdocs/rpminstall.xml (revision 1332394) +++ src/docs/src/documentation/content/xdocs/rpminstall.xml (working copy) @@ -1,219 +0,0 @@ - - - - - -
- RPM Installation -
- - -
- Server Installation from RPM - -

Prerequisites

-
    -
  • Machine on which the server can be installed - this should have - access to the Hadoop cluster and to a MySQL database
  • -
  • MySQL database
  • -
  • Hadoop cluster
  • -
  • In a secure environment, Unix user that the server will run as, and an associated Kerberos - service principal and keytabs.
  • -
  • hcatalog. & hcatalog-server .rpm packages.
  • -
- -

Throughout these instructions when you see a word in italics it - indicates a place where you should replace the word with an - appropriate value such as a hostname or password.

- -

Thrift Server Install

- -

Select a machine to install your Thrift server on. For smaller and test - installations this can be the same machine as the database, which we will - set up later. For the purposes of these instructions we will refer to this - machine as hcatsvr.acme.com.

- -

RPM installation will create a headless user named "hcat" on the server machine if it doesn't - exist. Server will run as this user.

- -

Download the MySQL Java connector libraries on hcatsvr.acme.com. - in some directory. We will refer to this directory as dbroot. You can obtain these from - MySQL's - download site.

- -

If using an rpm, install appropriate rpms:

- -

rpm -ivh hcatalog-version.rpm hcatalog-server-version.rpm

- -

Database Setup

- -

Select a machine to install the database on. This need not be the same - machine as the Thrift server. For large - clusters we recommend that they not be the same machine. For the - purposes of these instructions we will refer to this machine as - hcatdb.acme.com.

- -

Install MySQL server on hcatdb.acme.com. You can obtain - packages for MySQL from MySQL's - download site. We have developed and tested with versions 5.1.46 - and 5.1.48. We suggest you use these versions or later. - Once you have MySQL up and running, use the mysql command line - tool to add the hive user and hivemetastoredb - database. You will need to pick a password for your hive - user, and replace dbpassword in the following commands with it.

- -

mysql -u root -h hcatdb.acme.com -p

-

mysql> CREATE USER 'hive'@'hcatdb.acme.com' IDENTIFIED BY 'dbpassword';

-

mysql> CREATE DATABASE hivemetastoredb DEFAULT CHARACTER SET latin1 DEFAULT COLLATE latin1_swedish_ci;

-

mysql> GRANT ALL PRIVILEGES ON hivemetastoredb.* TO 'hive'@'hcatdb.acme.com' WITH GRANT OPTION;

-

mysql> flush privileges;

-

mysql> quit;

-

mysql -u hive -D hivemetastoredb -hhcatdb.acme.com -p < /usr/share/hcatalog/scripts/hive-schema-0.7.0.mysql.sql

- -

Thrift Server Configuration

-

Now you need to edit your /etc/hcatalog/hive-site.xml file. - Open this file in your favorite text editor. The following table shows the - values you need to configure.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ParameterValue to Set it to
javax.jdo.option.ConnectionURLIn the JDBC connection string, change DBHOSTNAME to the name - of the machine you put the MySQL server on.
javax.jdo.option.ConnectionPassworddbpassword value you used in setting up the MySQL server - above
hive.metastore.warehouse.dirThe directory can be a URI or an absolute file path. If it is an absolute file path, it will be resolved to a URI by the metastore: -

-- If default hdfs was specified in core-site.xml, path resolves to HDFS location.

-

-- Otherwise, path is resolved as local file: URI.

-

This setting becomes effective when creating new tables (takes precedence over default DBS.DB_LOCATION_URI at time of table creation).

-
hive.metastore.urisSet the hostname of your Thrift - server by replacing SVRHOST with the name of the - machine you are installing the Thrift server on.
hive.metastore.sasl.enabledSet to true if you are using kerberos security with your Hadoop - cluster, false otherwise.
hive.metastore.kerberos.keytab.fileThe path to the Kerberos keytab file containing the metastore - Thrift server's service principal. Only required if you set - hive.metastore.sasl.enabled above to true.
hive.metastore.kerberos.principalThe service principal for the metastore Thrift server. You can - reference your host as _HOST and it will be replaced with your - actual hostname. Only required if you set - hive.metastore.sasl.enabled above to true.
- -

You can now procede to starting the server.

-
- -
- Starting the Server - -

sudo service hcatalog-server start

-
- -
- Logging - -

Server activity logs are located in - /var/log/hcat_server. Logging configuration is located at - /etc/hcatalog/log4j.properties. Server logging uses - DailyRollingFileAppender by default. It will generate a new - file per day and does not expire old log files automatically.

- -
- -
- Stopping the Server -

sudo service hcatalog-server stop

- - -
- -
- Client Installation - -

rpm -ivh hcatalog-version.rpm

- -

Now you need to edit your /etc/hcatalog/hive-site.xml file. - Open this file in your favorite text editor. The following table shows the - values you need to configure. These common values should match the values set on - the HCatalog server. Do NOT copy the configuration file - from your server installation as that contains the password to your - database, which you should not distribute to your clients.

- - - - - - - - - - - - - - - - - - - - - - - -
ParameterValue to Set it to
hive.metastore.warehouse.dirThe directory can be a URI or an absolute file path. If it is an absolute file path, it will be resolved to a URI by the metastore: -

-- If default hdfs was specified in core-site.xml, path resolves to HDFS location.

-

-- Otherwise, path is resolved as local file: URI.

-

This setting becomes effective when creating new tables (takes precedence over default DBS.DB_LOCATION_URI at time of table creation).

-
hive.metastore.urisSet the hostname of your Thrift - server by replacing SVRHOST with the name of the - machine you are installing the Thrift server on.
hive.metastore.sasl.enabledSet to false by default. Set to true if it is a secure environment.
hive.metastore.kerberos.principalThe service principal for the metastore Thrift server. You can - reference your host as _HOST and it will be replaced with - actual hostname. Need to set only in secure environment.
- -

The HCatalog command line interface (CLI) can now be invoked as - /bin/hcat.

- -
- - -
Index: src/docs/src/documentation/content/xdocs/install.xml =================================================================== --- src/docs/src/documentation/content/xdocs/install.xml (revision 1332394) +++ src/docs/src/documentation/content/xdocs/install.xml (working copy) @@ -126,6 +126,16 @@ purposes of these instructions we will refer to this machine as hcatsvr.acme.com.

+

If you have not already done so, install Hive 0.9 on this machine. You + can use the + binary distributions + provided by Hive or rpms available from + Apache Bigtop. If you use + the Apache Hive binary distribution, select a directory, henceforth + referred to as hive_home, and untar the distribution there. + If you use the rpms, hive_home will be + /usr/lib/hive.

+

Install the MySQL Java connector libraries on hcatsvr.acme.com. You can obtain these from MySQL's @@ -234,15 +244,6 @@ -

A more potent script exists in the form of src/packages/hcat-configure.sh - which can configure the hive-site.xml to conform to the changes mentioned - above. It has predefined values for the most of the above mentioned fields - except for the ConnectionPassword and keytabs. It is capable of taking - those fields too as parameters in the command line. Just run it once - without parameters to see a full description of the usage of the script. - Users looking to automate the hcat installation should look to leverage - this script. -

You can now procede to starting the server.

@@ -271,7 +272,7 @@ Stopping the Server

To stop the HCatalog server, change directories to the root - directory and invoking sbin/hcat_server.sh stop

+ directory and invoking HIVE_HOME=hive_home sbin/hcat_server.sh stop

Index: RELEASE_NOTES.txt =================================================================== --- RELEASE_NOTES.txt (revision 1332394) +++ RELEASE_NOTES.txt (working copy) @@ -47,14 +47,28 @@ - Switch from StorageDrivers to SerDes. See "Backward Incompatibilities" below. - Addition of JSonSerDe to support reading and writing JSON data. + - Rather than releasing rpms from HCatalog, we rely on the rpms provided by + Apache Bigtop. + - The HCatalog binary distribution no longer includes Apache Hive. We now + require that Hive first be installed. + - The HCatalog source distribution no longer includes Apache Hive source. + It now pulls the required jars via maven. + For a full list of changes see CHANGES.txt located in the same directory as this file. Backward Incompatibilities ========================== -HCatalog no longer supports its own StorageDriver classes for data -(de)serialization. Instead it uses Hive's SerDe classes. +- HCatalog no longer supports its own StorageDriver classes for data + (de)serialization. Instead it uses Hive's SerDe classes. +- Rather than releasing rpms from HCatalog, we rely on the rpms provided by + Apache Bigtop. +- The HCatalog binary distribution no longer includes Apache Hive. We now + require that Hive first be installed. +- The HCatalog source distribution no longer includes Apache Hive source. + It now pulls the required jars via maven. + Notes ===== HBase integration with HCatalog is experimental and not yet ready for production use.