Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
Description
This might need more details to flush out since Hbase writing and HDFS writing are different
Ryan: The problem was mainly durability, if I remember correctly
There are different durability models between Kite HBase and Kite HDFS
HDFS writes entire files and commits them so retries work like standard OutputFormat commiters
but, HBase needs to do something different
me: sure thats still an implementation details undernath
Ryan: IIRC, the connector API basically wants to signal when everything should be committed, all at once
that seems difficult to implement with HBase
If you know more about Sqoop and think it can be done, I'm all for making the changes to enable it from the Kite side