Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
2.4.1
-
None
-
None
-
centos 7, HDP2.5
Description
Using hdp2.5 (ambari 2.4.1) with a Hive view, a table has been created by an external tool (hdfs connector from confluent.io). I can see the table metadata (column names and type) but trying to get data gives me the error:
S020 Data storage error
[blahblah]Caused by: javax.persistence.RollbackException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.6.2.v20151217-774c696): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'ambari.DS_JOBIMPL_4' doesn't exist
Error Code: 1146[a lot more lines]
Now I saw another answer giving a cut&paste solution. This does not work for me, as I then end up with a different error:
S020 Data storage error
[blahblah]Caused by: javax.persistence.RollbackException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.6.2.v20151217-774c696): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column 'DS_guid' in 'field list'
Error Code: 1054[a lot more lines]
It looks to me like the answer is out of date.
I suppose the original reason still holds:
ROOT CAUSE: This is a limitation with the Mysql DB itself with varchar accounting to more than 75000 maximum row size where as the mysql DB can accept only 65535 as its max value.
But I would love to understand a few things:
- Why does this still happen (I have seen a few commits already in ambari trying to fix this)
- How can I regenerate the relevant CREATE statement
- Why sometimes is it DS_JOBIMPL_4, sometimes DS_JOBIMPL_8 and how can I be sure that I am creating the right tables
- In which log file can I find the relevant failded CREATE statement
- Is there a way to fully avoid this
That's a lot of questions but it always happened to me, no matter which HDP version so I would really love to go to the bottom of it.
Thanks,