Duplicate of PHOENIX-946

thanks james I had started working on a phoenix hive storage handler but the other threads seems like a more complicte discussion

Hi james
With our last discussion I have started upo on this work again.
I have a first working copy of the hivephoenix storage handler.
I need to iron out predicate pushdowns and splits but the work is nearing the end

Hello james I took ages to get a first working copy, so sorry for the delay.
I will propose a first version shortly could you assign me to this task.

No problem, nicolas maillard. Take a look at ravi's latest pull for stuff you may be able to leverage: https://github.com/apache/phoenix/pull/22 or the Pig loader support we now have.

Hello James
I already did essentially to make sure we have a code consistency and reuse and also to make my life easier.
I have the hive loader working to some extent but the predicate pushdown is limited and the split is suboptimal right now. So I will push a first version allowing to create and write in phoenix while I polish up the loader part, it would work but the performance would be terrible.

This issue had been closed duplicate, but inital work can be done here

Initial patch for this feature

Here ta first patch for this feature a more in depth documentation can be found here
https://github.com/nmaillard/Phoenix-Hive

Thanks nicolas maillard for the patch. As the patch includes a lot files, do you mind doing a Pull Request on git so we can have an effective discussion there. Sorry for the trouble.

Agree, ravi, a pull request would make it quite a bit easier to review. Would you mind doing that, nicolas maillard?

Yes, makes sense and I wanted to get a couple more UT in there. Sorry for the trouble, I'll do a pull request asap

PR sent out, with a couple more UTs
https://github.com/apache/phoenix/pull/74
Hope this makes it easier to review

Updated the pull request with initial feedback.
Should I do second pull request or this update the way to go.
Have not seen if this update has been seen

nicolas maillard it'd be good to attach the patch here so that the precommit build can run on the patch and catch issues if any.

Ok i'll attach a the patch with the updated PR as well.
For the updated PR, let me know if I have addressed the more important issues and any feedback

Thanks for the revisions, nicolas maillard. I've left some feedback for you on the pull request.

I'm so looking forward to the PhoenixStorageHandler – just awesome, no matter how bad the predicate push down may be initially. Just a question nicolas maillard: Is it really necessary to specify the primary-key fields, especially if it's an existing table

Cleaned a bit the patch. Let's discuss what else we need to add to get it committed.

GitHub user ss77892 opened a pull request:

https://github.com/apache/phoenix/pull/150

PHOENIX-331 impementation

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ss77892/phoenix PHOENIX-331

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/phoenix/pull/150.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #150

commit 5545a6fdddbb12a621a1db1a4b5f2cebc37b9530
Author: Sergey Soldatov <sergey.soldatov@gmail.com>
Date: 2016-02-19T08:32:52Z

Initial Patch

Sergey Soldatov I'm interested in this. please point me out the quick start for installation and configuration. Thanks!

Thanks for picking up this important work, Sergey Soldatov. There still appears to be a fair amount of copy/paste code from our MR integration. For example:

• HivePhoenixInputFormat extends PhoenixInputFormat but seems to copy/paste much of the code. Is that necessary? Can some refactoring be done instead?
• Same thing with HivePhoenixRecordReader which extends PhoenixRecordReader.

ravi, [~ndimiduk, and Gabriel Reid - would be great if you could review.