Index: src/docs/src/documentation/content/xdocs/loadstore.xml =================================================================== --- src/docs/src/documentation/content/xdocs/loadstore.xml (revision 1378860) +++ src/docs/src/documentation/content/xdocs/loadstore.xml (working copy) @@ -68,51 +68,131 @@
HCatalog Data Types -

Restrictions apply to the types of columns HCatLoader can read.

-

HCatLoader can read only the data types listed in the table. +

Restrictions apply to the types of columns HCatLoader can read from HCatalog-managed tables.

+

HCatLoader can read only the data types listed in the table below. The table shows how Pig will interpret the HCatalog data type.

- - - - +
-

HCatalog Data Type

-
-

Pig Data Type

-
+ + + - - + + - - + + - - + + - - + + + + + + + + + + + + + + + + + + +
+

Primitives

+
+

+
-

primitives (int, long, float, double, string)

-
-

int, long, float, double, string to chararray

-
+

HCatalog Data Type

+
+

Pig Data Type

+
-

map (key type should be string, valuetype must be string)

-
-

map

-
+

int

+
+

int

+
-

List<any type>

-
-

bag

-
+

long

+
+

long

+
-

struct<any type fields>

-
-

tuple

-
+

float

+
+

float

+
+

double

+
+

double

+
+

string

+
+

chararray

+
+

boolean

+
+

boolean

+
+

binary

+
+

bytearray

+
+ + + + + + + + + + + + + + + + + + + + +
+

Complex Types

+
+

+
+

HCatalog Data Type

+
+

Pig Data Type

+
+

map
(key type should be string)

+
+

map

+
+

List<any type>

+
+

bag

+
+

struct<any type fields>

+
+

tuple

+
+ +


+

Currently HCatLoader cannot map the smallint and tinyint datatypes to Pig data types. +This issue exists in HCatalog version 0.4.0; +the fix for Jira issue HCATALOG-425 should be available in HCatalog version 0.5.0.

+
@@ -355,51 +435,132 @@
HCatalog Data Types -

Restrictions apply to the types of columns HCatStorer can write.

-

HCatStorer can write only the data types listed in the table. +

Restrictions apply to the types of columns HCatStorer can write to HCatalog-managed tables.

+

HCatStorer can write only the data types listed in the table. The table shows how Pig will interpret the HCatalog data type.

- - - - + +
-

HCatalog Data Type

-
-

Pig Data Type

-
+ + + - - + + - - + + - - + + - - + + + + + + + + + + + + + + + + + + +
+

Primitives

+
+

+
-

primitives (int, long, float, double, string)

-
-

int, long, float, double, string to chararray

-
+

Pig Data Type

+
+

HCatalog Data Type

+
-

map (key type should be string, valuetype must be string)

-
-

map

-
+

int

+
+

int

+
-

List<any type>

-
-

bag

-
+

long

+
+

long

+
-

struct<any type fields>

-
-

tuple

-
+

float

+
+

float

+
+

double

+
+

double

+
+

chararray

+
+

string

+
+

boolean

+
+

boolean

+
+

bytearray

+
+

binary

+
+ + + + + + + + + + + + + + + + + + + + +
+

Complex Types

+
+

+
+

Pig Data Type

+
+

HCatalog Data Type

+
+

map

+
+

map
(key type should be string)

+
+

bag

+
+

List<any type>

+
+

tuple

+
+

struct<any type fields>

+
+ +


+

Currently HCatLoader cannot map the smallint and tinyint datatypes to Pig data types. +This issue exists in HCatalog version 0.4.0; +the fix for Jira issue HCATALOG-425 should be available in HCatalog version 0.5.0.

+
Index: src/docs/src/documentation/content/xdocs/inputoutput.xml =================================================================== --- src/docs/src/documentation/content/xdocs/inputoutput.xml (revision 1378860) +++ src/docs/src/documentation/content/xdocs/inputoutput.xml (working copy) @@ -149,7 +149,85 @@ +
+ HCatRecord + +

HCatRecord is the type supported for storing values in HCatalog tables.

+

The types in an HCatalog table schema determine the types of objects returned for different fields in HCatRecord. This table shows the mappings between Java classes for MapReduce programs and HCatalog data types:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

HCatalog Data Type

Java Class in MapReduce

Values

TINYINT

java.lang.Byte

-128 to 127

SMALLINT

java.lang.Short

-215 to 215-1 (-32,768 to 32,767)

INT

java.lang.Integer

-231 to 231-1 (-2,147,483,648 to 2,147,483,647)

BIGINT

java.lang.Long

-263 to 263-1 (-9,223,372,036,854,775,808 to 9,223,372,036,854,775,807)

BOOLEAN

java.lang.Boolean

true or false

FLOAT

java.lang.Float

single-precision floating-point value

DOUBLE

java.lang.Double

double-precision floating-point value

BINARY

byte[]

binary data

STRING

java.lang.String

character string

STRUCT

java.util.List

structured data

ARRAY

java.util.List

values of one data type

MAP

java.util.Map

key-value pairs

+ +
+ + +
Running MapReduce with HCatalog

Your MapReduce program needs to be told where the Thrift server is.