Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.2.1
-
None
-
None
Description
When inserting to a table on s3a://, Hive attempts to `chgrp` the files but files in s3a:// do not have group ownership.
hive> insert into INVENTORY select * from INVENTORY_Q1_2006; -chgrp: '' does not match expected pattern for group Usage: hadoop fs [generic options] -chgrp [-R] GROUP PATH...
Full output of the query here:
hive> insert into INVENTORY select * from INVENTORY_Q1_2006; -chgrp: '' does not match expected pattern for group Usage: hadoop fs [generic options] -chgrp [-R] GROUP PATH... Query ID = admin_20160617201151_5f953fbe-acde-4774-9ad7-06cffc76dd72 Total jobs = 1 Launching Job 1 out of 1 Status: Running (Executing on YARN cluster with App id application_1466165341299_0011) -------------------------------------------------------------------------------- VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED -------------------------------------------------------------------------------- Map 1 .......... SUCCEEDED 1 1 0 0 0 0 -------------------------------------------------------------------------------- VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 8.71 s -------------------------------------------------------------------------------- Loading data to table mydb.inventory -chgrp: '' does not match expected pattern for group Usage: hadoop fs [generic options] -chgrp [-R] GROUP PATH... Table mydb.inventory stats: [numFiles=12, numRows=6020352, totalSize=25250706, rawDataSize=96325632] OK Time taken: 19.123 seconds
The table:
CREATE TABLE IF NOT EXISTS inventory ( MONTH_ID int, ITEM_ID int, BOH_QTY float, EOH_QTY float ) row format delimited fields terminated by '|' escaped by '\\' stored as ORC LOCATION 's3a://mybucket/hive/warehouse/mydb.db/inventory' tblproperties ("orc.compress"="SNAPPY");
Attachments
Issue Links
- relates to
-
HADOOP-13309 Document S3A known limitations in file ownership and permission model.
- Resolved
-
HADOOP-13310 S3A reporting of file group as empty is harmful to compatibility for the shell.
- Resolved