Index: hcatalog/src/test/e2e/templeton/resource/default.res IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- hcatalog/src/test/e2e/templeton/resource/default.res (revision ) +++ hcatalog/src/test/e2e/templeton/resource/default.res (revision ) @@ -0,0 +1,2 @@ +$resources = { +}; Index: hcatalog/src/test/e2e/templeton/README.txt IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- hcatalog/src/test/e2e/templeton/README.txt (revision d563db5e363ece1a382041fc08be2511644faecd) +++ hcatalog/src/test/e2e/templeton/README.txt (revision ) @@ -20,11 +20,41 @@ End to end tests in templeton runs tests against an existing templeton server. It runs hcat, mapreduce, streaming, hive and pig tests. +(at a minimum put the following into hive-site.xml into build/dist/conf + + javax.jdo.option.ConnectionURL + jdbc:derby:;databaseName=/Users/ekoifman/dev/data/tmp/metastore_db_e2e;create=true + + + + hive.metastore.uris + thrift://localhost:9083 + For Hive CLI to connect to + + +props for Namenode + Jobtracker are found in $HADOOP_HOME/conf (thus, HADOOP_HOME must be set) + +(I'm not sure this is needed) +don't forget to create directories in HDFS that hive expects ( as in +https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-RunningHive) + +launch templeton server: build/dist/hcatalog/sbin/webhcat_server.sh start +start hive metastore (build/dist/bin/hive --service metastore +hive-site.xml will make hive use this metastore when WebHCat calls hive) +) + + !!!! NOTE !!!! -------------- USE SVN TO CHECKOUT CODE FOR RUNNING TESTS AS THE TEST HARNESS IS EXTERNED FROM PIG. GIT WILL NOT IMPORT IT + (if you are using GIT, check out http://svn.apache.org/repos/asf/hive/trunk (or whichever branch) + (http://hive.apache.org/version_control.html) and symlink + hcatalog/src/test/e2e/harness/ dir) +Also, make sure you have ./e2e/templeton/resource/ which mirrors ./e2e/hcatalog/resource/ +and modify ./templeton/build.xml to include ./resource/* in "tar" target. + Test cases ---------- The tests are defined in src/test/e2e/templeton/tests/*.conf @@ -57,12 +87,22 @@ 3. Copy contents of src/test/e2e/templeton/inpdir to hdfs +(e.g. ./bin/hadoop fs -put ~/dev/hive/hcatalog/src/test/e2e/templeton/inpdir/ webhcate2e) 4. You will need to two jars in the same HDFS directory as the contents of inpdir. piggybank.jar, which can be obtained from Pig. The second is the hadoop-examples.jar, which can be obtained from your Hadoop distribution. This should be called hexamples.jar when it is uploaded to HDFS. +also copy hadoop-streaming.jar (but it seems to look for it in /user/templeton/) +Also make sure you have the following in HDFS +a. hdfs:///user/templeton/hadoop-streaming.jar +b. hdfs:///apps/templeton/pig-0.10.1.tar.gz +c. hdfs:///apps/templeton/hive-0.10.0.tar.gz +5. Make sure TEMPLETON_HOME evnironment variable is set + +6. hadoop/conf/core-site.xml should have items described in http://hive.apache.org/docs/hcat_r0.5.0/rest_server_install.html#Permissions + Running the tests ----------------- Use the following command to run tests - @@ -73,6 +113,7 @@ If you want to run specific test group you can specify the group, for example: -Dtests.to.run='-t TestHive' If you want to run specific test in a group group you can specify the test, for example: -Dtests.to.run='-t TestHive_1' +For example, tests/ddl.conf has several groups such as 'name' => 'REST_DDL_TABLE_BASIC'; use REST_DDL_TABLE_BASIC as the name Running the hcat authorization tests @@ -110,3 +151,6 @@ This assumes you've got webhdfs at the address above, the inpdir info in /user/templeton, and templeton running on the default port. You can change any of those properties in the build file. It's best to set HADOOP_HOME_WARN_SUPPRESS=true everywhere you can. +Also usefu to add to conf/hadoop-env.sh +export HADOOP_OPTS="-Djava.security.krb5.realm=OX.AC.UK -Djava.security.krb5.kdc=kdc0.ox.ac.uk:kdc1.ox.ac.uk" +to prevent warning about SCDynamicStore which may throw some tests off Index: hcatalog/src/test/e2e/templeton/build.xml IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- hcatalog/src/test/e2e/templeton/build.xml (revision d563db5e363ece1a382041fc08be2511644faecd) +++ hcatalog/src/test/e2e/templeton/build.xml (revision ) @@ -24,6 +24,7 @@ + @@ -51,6 +52,7 @@ + @@ -60,6 +62,9 @@ + + + Index: hcatalog/src/test/e2e/templeton/resource/windows.res IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- hcatalog/src/test/e2e/templeton/resource/windows.res (revision ) +++ hcatalog/src/test/e2e/templeton/resource/windows.res (revision ) @@ -0,0 +1,2 @@ +$resources = { +};