Uploaded image for project: 'Sqoop (Retired)'
  1. Sqoop (Retired)
  2. SQOOP-611

supoort multi-version records while importing the data into HBase

Details

    • New Feature
    • Status: Reopened
    • Minor
    • Resolution: Unresolved
    • 1.4.1-incubating, 1.4.2
    • None
    • hbase-integration

    Description

      While importing the data into HBase, want use a column (which is long/integer type) to be as timestamp (or version) by adding the --hbase-timestamp-column option. so, I can insert multiple records of the same row key according to the long timestamp/version. of course, the hbase table must be set the version property greater than 1.

      Attachments

        1. hbase_multiple_versions.patch
          13 kB
          William Watson
        2. SQOOP-611.patch
          10 kB
          Zhancheng Deng

        Activity

          zhan8610189 Zhancheng Deng added a comment -

          patch for new feature (add the --hbase-timestamp-column option)

          zhan8610189 Zhancheng Deng added a comment - patch for new feature (add the --hbase-timestamp-column option)
          zhan8610189 Zhancheng Deng added a comment -

          patch for new feature(add the --hbase-timestamp-column option)

          zhan8610189 Zhancheng Deng added a comment - patch for new feature(add the --hbase-timestamp-column option)
          cheolsoo Cheolsoo Park added a comment -

          Hello Zhancheng,

          Thank you very much for your patch! I have few comments:

          1) Can you please post your patch to review board? https://reviews.apache.org.
          2) Can you please update the user doc? The hbase options are documented in src/docs/user/hbase-args.txt.
          3) Can you please add some unit test if possible? HBase tests can be found in HBaseTestCase.java and TestHBaseImport.java.

          cheolsoo Cheolsoo Park added a comment - Hello Zhancheng, Thank you very much for your patch! I have few comments: 1) Can you please post your patch to review board? https://reviews.apache.org . 2) Can you please update the user doc? The hbase options are documented in src/docs/user/hbase-args.txt. 3) Can you please add some unit test if possible? HBase tests can be found in HBaseTestCase.java and TestHBaseImport.java.
          cheolsoo Cheolsoo Park added a comment - - edited

          The jira shouldn't be marked as a resolved until its patch is committed. So I am reopening it.

          Thanks!

          cheolsoo Cheolsoo Park added a comment - - edited The jira shouldn't be marked as a resolved until its patch is committed. So I am reopening it. Thanks!
          wattsinabox William Watson added a comment -

          We could really use this patch at Korrelate. We are wanting to use an existing timestamp as the version in HBase. Thanks for considering pulling it in!

          wattsinabox William Watson added a comment - We could really use this patch at Korrelate. We are wanting to use an existing timestamp as the version in HBase. Thanks for considering pulling it in!
          wattsinabox William Watson added a comment -

          I assume zhan8610189 hasn't replied yet, cheolsoo. However, since I need this patch, I have rebased it off of 1.4.4 and redone the patch. I can also add those tests and such and post to the review board if zhan8610189 never did so. Just let me know.

          wattsinabox William Watson added a comment - I assume zhan8610189 hasn't replied yet, cheolsoo . However, since I need this patch, I have rebased it off of 1.4.4 and redone the patch. I can also add those tests and such and post to the review board if zhan8610189 never did so. Just let me know.

          wattsinabox, please go ahead and upload the new patch to the review board. We will be more than happy to review it!

          jarcec Jarek Jarcec Cecho added a comment - wattsinabox , please go ahead and upload the new patch to the review board. We will be more than happy to review it!
          wattsinabox William Watson added a comment -

          jarcec, I have no idea how to use the review board, but I have submitted something... probably. Check this out and let me know if I did it right, please.

          https://reviews.apache.org/r/14377/

          wattsinabox William Watson added a comment - jarcec , I have no idea how to use the review board, but I have submitted something... probably. Check this out and let me know if I did it right, please. https://reviews.apache.org/r/14377/
          wattsinabox William Watson added a comment -

          In fact, I know I didn't do it right b/c I accidentally reviewed my own thing, but still... check it out.

          wattsinabox William Watson added a comment - In fact, I know I didn't do it right b/c I accidentally reviewed my own thing, but still... check it out.

          Please accept my apologies for the delay to getting back to you wattsinabox. It seems that you either did not "published" the review (there is a green box somewhere with button "publish") or did not specified "sqoop" group in the field "reviewers". As a result I can't get to the review. Would you mind taking look on it and ensuring that the review has been published and that Sqoop is in the reviewer group?

          jarcec Jarek Jarcec Cecho added a comment - Please accept my apologies for the delay to getting back to you wattsinabox . It seems that you either did not "published" the review (there is a green box somewhere with button "publish") or did not specified "sqoop" group in the field "reviewers". As a result I can't get to the review. Would you mind taking look on it and ensuring that the review has been published and that Sqoop is in the reviewer group?
          wattsinabox William Watson added a comment -

          jarcec, I thought it wasn't giving me a way to add reviewers, but it turns out the interface is just terrible and/or I'm an idiot.

          Published!

          https://reviews.apache.org/r/14377/

          wattsinabox William Watson added a comment - jarcec , I thought it wasn't giving me a way to add reviewers, but it turns out the interface is just terrible and/or I'm an idiot. Published! https://reviews.apache.org/r/14377/

          Hi wattsinabox, I can confirm that I can access the review now, thank you!

          jarcec Jarek Jarcec Cecho added a comment - Hi wattsinabox , I can confirm that I can access the review now, thank you!

          People

            Unassigned Unassigned
            zhan8610189 Zhancheng Deng
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: