[HIVE-14145] Too small length of column 'PARAM_VALUE' in table 'SERDE_PARAMS' - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 2.3.0, 3.0.0
Component/s: Metastore
Labels:
None

Description

Customer has following table

create external table hive_hbase_test(
HBASE_KEY string,
ENTITY_NAME string,
ENTITY_ID string,
CLAIM_HEADER_ID string,
CLAIM_LINE_ID string,
MEDICAL_CLAIM_SOURCE_SYSTEM string,
UNIQUE_MEMBER_ID string,
MEMBER_SOURCE_SYSTEM string,
SUBSCRIBER_ID string,
COVERAGE_CLASS_CODE string,
SERVICING_PROVIDER_ID string,
PROVIDER_SOURCE_SYSTEM string,
SERVICING_PROVIDER_SPECIALTY string,
SERVICING_STANDARD_PROVIDER_SPECIALTY string,
SERVICING_PROVIDER_TYPE_CODE string,
REFERRING_PROVIDER_ID string,
ADMITTING_PROVIDER_ID string,
ATTENDING_PROVIDER_ID string,
OPERATING_PROVIDER_ID string,
BILLING_PROVIDER_ID string,
ORDERING_PROVIDER_ID string,
HEALTH_PLAN_SOURCE_ID string,
HEALTH_PLAN_PAYER_NAME string,
BUSINESS_UNIT string,
OPERATING_UNIT string,
PRODUCT string,
MARKET string,
DEPARTMENT string,
IPA string,
SUPPLEMENTAL_DATA_TYPE string,
PSEUDO_CLAIM_FLAG string,
CLAIM_STATUS string,
CLAIM_LINE_STATUS string,
CLAIM_DENIED_FLAG string,
SERVICE_LINE_DENIED_FLAG string,
DENIED_REASON_CODE string,
SERVICE_LINE_DENIED_REASON_CODE string,
DAYS_DENIED int,
DIAGNOSIS_DATE timestamp,
SERVICE_DATE TIMESTAMP,
SERVICE_FROM_DATE TIMESTAMP,
SERVICE_TO_DATE TIMESTAMP,
ADMIT_DATE TIMESTAMP,
ADMIT_TYPE string,
ADMIT_SOURCE_TYPE string,
DISCHARGE_DATE TIMESTAMP,
DISCHARGE_STATUS_CODE string,
SERVICE_LINE_TYPE_OF_SERVICE string,
TYPE_OF_BILL_CODE string,
INPATIENT_FLAG string,
PLACE_OF_SERVICE_CODE string,
FACILITY_CODE string,
AUTHORIZATION_NUMBER string,
CLAIM_REFERRAL_NUMBER string,
CLAIM_TYPE string,
CLAIM_ADJUSTMENT_TYPE string,
ICD_DIAGNOSIS_CODE_1 string,
PRESENT_ON_ADMISSION_FLAG_1 string,
ICD_DIAGNOSIS_CODE_2 string,
PRESENT_ON_ADMISSION_FLAG_2 string,
ICD_DIAGNOSIS_CODE_3 string,
PRESENT_ON_ADMISSION_FLAG_3 string,
ICD_DIAGNOSIS_CODE_4 string,
PRESENT_ON_ADMISSION_FLAG_4 string,
ICD_DIAGNOSIS_CODE_5 string,
PRESENT_ON_ADMISSION_FLAG_5 string,
ICD_DIAGNOSIS_CODE_6 string,
PRESENT_ON_ADMISSION_FLAG_6 string,
ICD_DIAGNOSIS_CODE_7 string,
PRESENT_ON_ADMISSION_FLAG_7 string,
ICD_DIAGNOSIS_CODE_8 string,
PRESENT_ON_ADMISSION_FLAG_8 string,
ICD_DIAGNOSIS_CODE_9 string,
PRESENT_ON_ADMISSION_FLAG_9 string,
ICD_DIAGNOSIS_CODE_10 string,
PRESENT_ON_ADMISSION_FLAG_10 string,
ICD_DIAGNOSIS_CODE_11 string,
PRESENT_ON_ADMISSION_FLAG_11 string,
ICD_DIAGNOSIS_CODE_12 string,
PRESENT_ON_ADMISSION_FLAG_12 string,
ICD_DIAGNOSIS_CODE_13 string,
PRESENT_ON_ADMISSION_FLAG_13 string,
ICD_DIAGNOSIS_CODE_14 string,
PRESENT_ON_ADMISSION_FLAG_14 string,
ICD_DIAGNOSIS_CODE_15 string,
PRESENT_ON_ADMISSION_FLAG_15 string,
ICD_DIAGNOSIS_CODE_16 string,
PRESENT_ON_ADMISSION_FLAG_16 string,
ICD_DIAGNOSIS_CODE_17 string,
PRESENT_ON_ADMISSION_FLAG_17 string,
ICD_DIAGNOSIS_CODE_18 string,
PRESENT_ON_ADMISSION_FLAG_18 string,
ICD_DIAGNOSIS_CODE_19 string,
PRESENT_ON_ADMISSION_FLAG_19 string,
ICD_DIAGNOSIS_CODE_20 string,
PRESENT_ON_ADMISSION_FLAG_20 string,
ICD_DIAGNOSIS_CODE_21 string,
PRESENT_ON_ADMISSION_FLAG_21 string,
ICD_DIAGNOSIS_CODE_22 string,
PRESENT_ON_ADMISSION_FLAG_22 string,
ICD_DIAGNOSIS_CODE_23 string,
PRESENT_ON_ADMISSION_FLAG_23 string,
ICD_DIAGNOSIS_CODE_24 string,
PRESENT_ON_ADMISSION_FLAG_24 string,
ICD_DIAGNOSIS_CODE_25 string,
PRESENT_ON_ADMISSION_FLAG_25 string,
QUANTITY_OF_SERVICES decimal(10,2),
REVENUE_CODE string,
PROCEDURE_CODE string,
PROCEDURE_CODE_MODIFIER_1 string,
PROCEDURE_CODE_MODIFIER_2 string,
PROCEDURE_CODE_MODIFIER_3 string,
PROCEDURE_CODE_MODIFIER_4 string,
ICD_VERSION_CODE_TYPE string,
ICD_PROCEDURE_CODE_1 string,
ICD_PROCEDURE_CODE_2 string,
ICD_PROCEDURE_CODE_3 string,
ICD_PROCEDURE_CODE_4 string,
ICD_PROCEDURE_CODE_5 string,
ICD_PROCEDURE_CODE_6 string,
ICD_PROCEDURE_CODE_7 string,
ICD_PROCEDURE_CODE_8 string,
ICD_PROCEDURE_CODE_9 string,
ICD_PROCEDURE_CODE_10 string,
ICD_PROCEDURE_CODE_11 string,
ICD_PROCEDURE_CODE_12 string,
ICD_PROCEDURE_CODE_13 string,
ICD_PROCEDURE_CODE_14 string,
ICD_PROCEDURE_CODE_15 string,
ICD_PROCEDURE_CODE_16 string,
ICD_PROCEDURE_CODE_17 string,
ICD_PROCEDURE_CODE_18 string,
ICD_PROCEDURE_CODE_19 string,
ICD_PROCEDURE_CODE_20 string,
ICD_PROCEDURE_CODE_21 string,
ICD_PROCEDURE_CODE_22 string,
ICD_PROCEDURE_CODE_23 string,
ICD_PROCEDURE_CODE_24 string,
ICD_PROCEDURE_CODE_25 string,
E_CODE_1 string,
E_CODE_TYPE_1 string,
E_CODE_2 string,
E_CODE_TYPE_2 string,
E_CODE_3 string,
E_CODE_TYPE_3 string,
EMERGENCY_FLAG string,
HOSPITAL_RELATED_FLAG string,
OUTSIDE_LABS_FLAG string,
PPS_CODE string,
NATIONAL_DRUG_CODE string,
VALUE_AMOUNT decimal(10,2),
CAPITATED_SERVICE_FLAG string,
NETWORK_STATUS_FLAG string,
ADJUDICATED_DATE TIMESTAMP,
LEDGER_PAID_DATE TIMESTAMP,
ENTERED_DATE TIMESTAMP,
SUBMITTED_DATE TIMESTAMP,
PAYMENT_PROCESS_DATE TIMESTAMP,
PAYMENT_AMOUNT decimal(10,2),
REQUESTED_AMOUNT decimal(10,2),
ALLOWED_AMOUNT decimal(10,2),
COPAY_AMOUNT decimal(10,2),
COINSURANCE_AMOUNT decimal(10,2),
DEDUCTIBLE_AMOUNT decimal(10,2),
PATIENT_LIABILITY_AMOUNT DECIMAL(10,2),
COORDINATION_OF_BENEFITS_AMOUNT string,
FEE_FOR_SERVICE_AMOUNT decimal(10,2),
WITHHOLD_AMOUNT decimal(10,2),
CAPITATION_AMOUNT decimal(10,2),
DISCOUNTED_AMOUNT decimal(10,2),
DRG_GROUPER_PAID string,
DRG_CODE_PAID string,
DRG_GROUPER_VERSION_PAID string,
DRG_SEVERITY_LEVEL_PAID string,
DRG_OUTLIER_PAID string,
DRG_GROUPER_BILLED string,
DRG_CODE_BILLED string,
DRG_GROUPER_VERSION_BILLED string,
DRG_SEVERITY_LEVEL_BILLED string,
DRG_OUTLIER_BILLED string,
MED_CLAIM_USER_DEFINED_1 string,
MED_CLAIM_USER_DEFINED_2 string,
MED_CLAIM_USER_DEFINED_3 string,
RECORD_STATUS string,
RECORD_EFFECTIVE_START_DATE TIMESTAMP,
RECORD_EFFECTIVE_END_DATE TIMESTAMP,
ADMITTING_DIAGNOSIS_CODE string,
CPT_II_CODE string,
ORIGINAL_CLAIM_NUMBER string,
SERVICING_SOURCE_PROVIDER_ID string,
HEDIS_FLAG string,
ORIGINAL_SOURCE_ID string,
ORIGINAL_SOURCE string,
CLAIM_OR_ENCOUNTER_IDENTIFIER string,
CLAIM_DRG_AMOUNT decimal(10,2),
PRODUCT_OR_SERVICE_ID_QUALIFIER string,
UNIT_OR_BASIS_FOR_MEASUREMENT_CODE string,
DIAGNOSIS_CODE_POINTER string,
CO_PAY_STATUS_CODE string,
PATIENT_ACCOUNT_NUMBER string,
DELAY_REASON_CODE string,
ADJUSTMENT_REASON_CODE string,
ADJUSTMENT_AMOUNT decimal(18,2),
ADJUSTMENT_QUANTITY decimal(15,2),
PRICING_METHODOLOGY string,
REPRICED_SAVING_AMOUNT decimal(15,2),
REPRICING_ORGANIZATION_IDENTIFIER string,
REPRICING_PER_DIEM_OR_FLAT_RATE_AMOUNT decimal(9,2),
REPRICED_APPROVED_AMBULATORY_PATIENT_GROUP_CODE string,
REPRICED_APPROVED_AMBULATORY_PATIENT_GROUP_AMOUNT decimal(18,2),
REJECT_REASON_CODE string,
AMOUNT_QUALIFER_CODE string,
DATE_TIME_QUALIFIER string,
BILLING_PROVIDER_TAX_IDENTIFICATION_NUMBER string,
CONTRACT_TYPE_CODE string,
CONTRACT_AMOUNT decimal(18,2),
CONTRACT_PERCENTAGE decimal(6,2),
CONTRACT_CODE string,
DIAGNOSIS_TYPE_CODE_PRINCIPAL string,
DIAGNOSIS_TYPE_CODE string,
PATIENT_REASON_FOR_VISIT string,
PRINCIPAL_PROCEDURE_QUALIFIER_CODE string,
PRINCIPAL_PROCEDURE_CODE string,
DATE_TIME_PERIOD string,
FACILITY_TAX_AMOUNT decimal(18,2)
)
ROW FORMAT SERDE
  'org.apache.hadoop.hive.hbase.HBaseSerDe'
STORED BY
  'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
  'serialization.format'='1',
  'hbase.columns.mapping'='
:key,
df:ENTITY_NAME,
df:ENTITY_ID,
df:CLAIM_HEADER_ID,
df:CLAIM_LINE_ID,
df:MEDICAL_CLAIM_SOURCE_SYSTEM,
df:UNIQUE_MEMBER_ID,
df:MEMBER_SOURCE_SYSTEM,
df:SUBSCRIBER_ID,
df:COVERAGE_CLASS_CODE,
df:SERVICING_PROVIDER_ID,
df:PROVIDER_SOURCE_SYSTEM,
df:SERVICING_PROVIDER_SPECIALTY,
df:SERVICING_STANDARD_PROVIDER_SPECIALTY,
df:SERVICING_PROVIDER_TYPE_CODE,
df:REFERRING_PROVIDER_ID,
df:ADMITTING_PROVIDER_ID,
df:ATTENDING_PROVIDER_ID,
df:OPERATING_PROVIDER_ID,
df:BILLING_PROVIDER_ID,
df:ORDERING_PROVIDER_ID,
df:HEALTH_PLAN_SOURCE_ID,
df:HEALTH_PLAN_PAYER_NAME,
df:BUSINESS_UNIT,
df:OPERATING_UNIT,
df:PRODUCT,
df:MARKET,
df:DEPARTMENT,
df:IPA,
df:SUPPLEMENTAL_DATA_TYPE,
df:PSEUDO_CLAIM_FLAG,
df:CLAIM_STATUS,
df:CLAIM_LINE_STATUS,
df:CLAIM_DENIED_FLAG,
df:SERVICE_LINE_DENIED_FLAG,
df:DENIED_REASON_CODE,
df:SERVICE_LINE_DENIED_REASON_CODE,
df:DAYS_DENIED,
df:DIAGNOSIS_DATE,
df:SERVICE_DATE,
df:SERVICE_FROM_DATE,
df:SERVICE_TO_DATE,
df:ADMIT_DATE,
df:ADMIT_TYPE,
df:ADMIT_SOURCE_TYPE,
df:DISCHARGE_DATE,
df:DISCHARGE_STATUS_CODE,
df:SERVICE_LINE_TYPE_OF_SERVICE,
df:TYPE_OF_BILL_CODE,
df:INPATIENT_FLAG,
df:PLACE_OF_SERVICE_CODE,
df:FACILITY_CODE,
df:AUTHORIZATION_NUMBER,
df:CLAIM_REFERRAL_NUMBER,
df:CLAIM_TYPE,
df:CLAIM_ADJUSTMENT_TYPE,
df:ICD_DIAGNOSIS_CODE_1,
df:PRESENT_ON_ADMISSION_FLAG_1,
df:ICD_DIAGNOSIS_CODE_2,
df:PRESENT_ON_ADMISSION_FLAG_2,
df:ICD_DIAGNOSIS_CODE_3,
df:PRESENT_ON_ADMISSION_FLAG_3,
df:ICD_DIAGNOSIS_CODE_4,
df:PRESENT_ON_ADMISSION_FLAG_4,
df:ICD_DIAGNOSIS_CODE_5,
df:PRESENT_ON_ADMISSION_FLAG_5,
df:ICD_DIAGNOSIS_CODE_6,
df:PRESENT_ON_ADMISSION_FLAG_6,
df:ICD_DIAGNOSIS_CODE_7,
df:PRESENT_ON_ADMISSION_FLAG_7,
df:ICD_DIAGNOSIS_CODE_8,
df:PRESENT_ON_ADMISSION_FLAG_8,
df:ICD_DIAGNOSIS_CODE_9,
df:PRESENT_ON_ADMISSION_FLAG_9,
df:ICD_DIAGNOSIS_CODE_10,
df:PRESENT_ON_ADMISSION_FLAG_10,
df:ICD_DIAGNOSIS_CODE_11,
df:PRESENT_ON_ADMISSION_FLAG_11,
df:ICD_DIAGNOSIS_CODE_12,
df:PRESENT_ON_ADMISSION_FLAG_12,
df:ICD_DIAGNOSIS_CODE_13,
df:PRESENT_ON_ADMISSION_FLAG_13,
df:ICD_DIAGNOSIS_CODE_14,
df:PRESENT_ON_ADMISSION_FLAG_14,
df:ICD_DIAGNOSIS_CODE_15,
df:PRESENT_ON_ADMISSION_FLAG_15,
df:ICD_DIAGNOSIS_CODE_16,
df:PRESENT_ON_ADMISSION_FLAG_16,
df:ICD_DIAGNOSIS_CODE_17,
df:PRESENT_ON_ADMISSION_FLAG_17,
df:ICD_DIAGNOSIS_CODE_18,
df:PRESENT_ON_ADMISSION_FLAG_18,
df:ICD_DIAGNOSIS_CODE_19,
df:PRESENT_ON_ADMISSION_FLAG_19,
df:ICD_DIAGNOSIS_CODE_20,
df:PRESENT_ON_ADMISSION_FLAG_20,
df:ICD_DIAGNOSIS_CODE_21,
df:PRESENT_ON_ADMISSION_FLAG_21,
df:ICD_DIAGNOSIS_CODE_22,
df:PRESENT_ON_ADMISSION_FLAG_22,
df:ICD_DIAGNOSIS_CODE_23,
df:PRESENT_ON_ADMISSION_FLAG_23,
df:ICD_DIAGNOSIS_CODE_24,
df:PRESENT_ON_ADMISSION_FLAG_24,
df:ICD_DIAGNOSIS_CODE_25,
df:PRESENT_ON_ADMISSION_FLAG_25,
df:QUANTITY_OF_SERVICES,
df:REVENUE_CODE,
df:PROCEDURE_CODE,
df:PROCEDURE_CODE_MODIFIER_1,
df:PROCEDURE_CODE_MODIFIER_2,
df:PROCEDURE_CODE_MODIFIER_3,
df:PROCEDURE_CODE_MODIFIER_4,
df:ICD_VERSION_CODE_TYPE,
df:ICD_PROCEDURE_CODE_1,
df:ICD_PROCEDURE_CODE_2,
df:ICD_PROCEDURE_CODE_3,
df:ICD_PROCEDURE_CODE_4,
df:ICD_PROCEDURE_CODE_5,
df:ICD_PROCEDURE_CODE_6,
df:ICD_PROCEDURE_CODE_7,
df:ICD_PROCEDURE_CODE_8,
df:ICD_PROCEDURE_CODE_9,
df:ICD_PROCEDURE_CODE_10,
df:ICD_PROCEDURE_CODE_11,
df:ICD_PROCEDURE_CODE_12,
df:ICD_PROCEDURE_CODE_13,
df:ICD_PROCEDURE_CODE_14,
df:ICD_PROCEDURE_CODE_15,
df:ICD_PROCEDURE_CODE_16,
df:ICD_PROCEDURE_CODE_17,
df:ICD_PROCEDURE_CODE_18,
df:ICD_PROCEDURE_CODE_19,
df:ICD_PROCEDURE_CODE_20,
df:ICD_PROCEDURE_CODE_21,
df:ICD_PROCEDURE_CODE_22,
df:ICD_PROCEDURE_CODE_23,
df:ICD_PROCEDURE_CODE_24,
df:ICD_PROCEDURE_CODE_25,
df:E_CODE_1,
df:E_CODE_TYPE_1,
df:E_CODE_2,
df:E_CODE_TYPE_2,
df:E_CODE_3,
df:E_CODE_TYPE_3,
df:EMERGENCY_FLAG,
df:HOSPITAL_RELATED_FLAG,
df:OUTSIDE_LABS_FLAG,
df:PPS_CODE,
df:NATIONAL_DRUG_CODE,
df:VALUE_AMOUNT,
df:CAPITATED_SERVICE_FLAG,
df:NETWORK_STATUS_FLAG,
df:ADJUDICATED_DATE,
df:LEDGER_PAID_DATE,
df:ENTERED_DATE,
df:SUBMITTED_DATE,
df:PAYMENT_PROCESS_DATE,
df:PAYMENT_AMOUNT,
df:REQUESTED_AMOUNT,
df:ALLOWED_AMOUNT,
df:COPAY_AMOUNT,
df:COINSURANCE_AMOUNT,
df:DEDUCTIBLE_AMOUNT,
df:PATIENT_LIABILITY_AMOUNT,
df:COORDINATION_OF_BENEFITS_AMOUNT,
df:FEE_FOR_SERVICE_AMOUNT,
df:WITHHOLD_AMOUNT,
df:CAPITATION_AMOUNT,
df:DISCOUNTED_AMOUNT,
df:DRG_GROUPER_PAID,
df:DRG_CODE_PAID,
df:DRG_GROUPER_VERSION_PAID,
df:DRG_SEVERITY_LEVEL_PAID,
df:DRG_OUTLIER_PAID,
df:DRG_GROUPER_BILLED,
df:DRG_CODE_BILLED,
df:DRG_GROUPER_VERSION_BILLED,
df:DRG_SEVERITY_LEVEL_BILLED,
df:DRG_OUTLIER_BILLED,
df:MED_CLAIM_USER_DEFINED_1,
df:MED_CLAIM_USER_DEFINED_2,
df:MED_CLAIM_USER_DEFINED_3,
df:RECORD_STATUS,
df:RECORD_EFFECTIVE_START_DATE,
df:RECORD_EFFECTIVE_END_DATE,
df:ADMITTING_DIAGNOSIS_CODE,
df:CPT_II_CODE,
df:ORIGINAL_CLAIM_NUMBER,
df:SERVICING_SOURCE_PROVIDER_ID,
df:HEDIS_FLAG,
df:ORIGINAL_SOURCE_ID,
df:ORIGINAL_SOURCE,
df:CLAIM_OR_ENCOUNTER_IDENTIFIER,
df:CLAIM_DRG_AMOUNT,
df:PRODUCT_OR_SERVICE_ID_QUALIFIER,
df:UNIT_OR_BASIS_FOR_MEASUREMENT_CODE,
df:DIAGNOSIS_CODE_POINTER,
df:CO_PAY_STATUS_CODE,
df:PATIENT_ACCOUNT_NUMBER,
df:DELAY_REASON_CODE,
df:ADJUSTMENT_REASON_CODE,
df:ADJUSTMENT_AMOUNT,
df:ADJUSTMENT_QUANTITY,
df:PRICING_METHODOLOGY,
df:REPRICED_SAVING_AMOUNT,
df:REPRICING_ORGANIZATION_IDENTIFIER,
df:REPRICING_PER_DIEM_OR_FLAT_RATE_AMOUNT,
df:REPRICED_APPROVED_AMBULATORY_PATIENT_GROUP_CODE,
df:REPRICED_APPROVED_AMBULATORY_PATIENT_GROUP_AMOUNT,
df:REJECT_REASON_CODE,
df:AMOUNT_QUALIFER_CODE,
df:DATE_TIME_QUALIFIER,
df:BILLING_PROVIDER_TAX_IDENTIFICATION_NUMBER,
df:CONTRACT_TYPE_CODE,
df:CONTRACT_AMOUNT,
df:CONTRACT_PERCENTAGE,
df:CONTRACT_CODE,
df:DIAGNOSIS_TYPE_CODE_PRINCIPAL,
df:DIAGNOSIS_TYPE_CODE,
df:PATIENT_REASON_FOR_VISIT,
df:PRINCIPAL_PROCEDURE_QUALIFIER_CODE,
df:PRINCIPAL_PROCEDURE_CODE,
df:DATE_TIME_PERIOD,
df:FACILITY_TAX_AMOUNT
')
TBLPROPERTIES (
  'hbase.table.name'='emp',
  'COLUMN_STATS_ACCURATE'='false');

After executing

hive > desc hive_hbase_test;

we got exception

2016-06-30 14:41:20,803 ERROR [main]: ql.Driver (SessionState.java:printError(963)) - FAILED: RuntimeException MetaException(message:org.apache.hadoop.hive.serde2.SerDeException org.apache.hadoop.hive.hbase.HBaseSerDe: columns has 225 elements while hbase.columns.mapping has 163 elements (counting the key if implicit))
java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException org.apache.hadoop.hive.hbase.HBaseSerDe: columns has 225 elements while hbase.columns.mapping has 163 elements (counting the key if implicit))
        at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:278)
        at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:258)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genTablePlan(SemanticAnalyzer.java:9370)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9687)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9668)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:10188)
        at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:325)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10199)
        at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:209)
        at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:425)
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:309)
        at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1139)
        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1187)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1076)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1066)
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException org.apache.hadoop.hive.hbase.HBaseSerDe: columns has 225 elements while hbase.columns.mapping has 163 elements (counting the key if implicit))
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:399)
        at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:276)
        ... 27 more

The root-cause is that length of column 'PARAM_VALUE' in table 'SERDE_PARAMS' is too small.

mysql> desc SERDE_PARAMS;
+-------------+---------------+------+-----+---------+-------+
| Field       | Type          | Null | Key | Default | Extra |
+-------------+---------------+------+-----+---------+-------+
| SERDE_ID    | bigint(20)    | NO   | PRI | NULL    |       |
| PARAM_KEY   | varchar(256)  | NO   | PRI | NULL    |       |
| PARAM_VALUE | varchar(4000) | YES  |     | NULL    |       |
+-------------+---------------+------+-----+---------+-------+
3 rows in set (0,00 sec)

varchar(4000) is not enough. Solution is to change type varchar(4000) to text.

ALTER TABLE SERDE_PARAMS MODIFY PARAM_VALUE TEXT;

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

HIVE-14145.2.patch
01/Jul/16 10:09
3 kB
Oleksiy Sayankin
HIVE-14145.1.patch
01/Jul/16 08:58
2 kB
Oleksiy Sayankin

Issue Links

Is contained by

HIVE-12274 Increase width of columns used for general configuration in the metastore.

Closed

Activity

People

Assignee:: Oleksiy Sayankin

Reporter:: Oleksiy Sayankin

Votes:: 0 Vote for this issue

Watchers:: 7 Start watching this issue

Dates

Created:: 01/Jul/16 08:22

Updated:: 23/May/18 00:00

Resolved:: 26/Mar/17 21:26