Sqoop
  1. Sqoop
  2. SQOOP-611

supoort multi-version records while importing the data into HBase

    Details

    • Type: New Feature New Feature
    • Status: Reopened
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.4.1-incubating, 1.4.2
    • Fix Version/s: None
    • Component/s: hbase-integration
    • Labels:

      Description

      While importing the data into HBase, want use a column (which is long/integer type) to be as timestamp (or version) by adding the --hbase-timestamp-column option. so, I can insert multiple records of the same row key according to the long timestamp/version. of course, the hbase table must be set the version property greater than 1.

      1. SQOOP-611.patch
        10 kB
        Zhancheng Deng
      2. hbase_multiple_versions.patch
        13 kB
        William Watson

        Activity

        Hide
        Zhancheng Deng added a comment -

        patch for new feature (add the --hbase-timestamp-column option)

        Show
        Zhancheng Deng added a comment - patch for new feature (add the --hbase-timestamp-column option)
        Hide
        Zhancheng Deng added a comment -

        patch for new feature(add the --hbase-timestamp-column option)

        Show
        Zhancheng Deng added a comment - patch for new feature(add the --hbase-timestamp-column option)
        Hide
        Cheolsoo Park added a comment -

        Hello Zhancheng,

        Thank you very much for your patch! I have few comments:

        1) Can you please post your patch to review board? https://reviews.apache.org.
        2) Can you please update the user doc? The hbase options are documented in src/docs/user/hbase-args.txt.
        3) Can you please add some unit test if possible? HBase tests can be found in HBaseTestCase.java and TestHBaseImport.java.

        Show
        Cheolsoo Park added a comment - Hello Zhancheng, Thank you very much for your patch! I have few comments: 1) Can you please post your patch to review board? https://reviews.apache.org . 2) Can you please update the user doc? The hbase options are documented in src/docs/user/hbase-args.txt. 3) Can you please add some unit test if possible? HBase tests can be found in HBaseTestCase.java and TestHBaseImport.java.
        Hide
        Cheolsoo Park added a comment - - edited

        The jira shouldn't be marked as a resolved until its patch is committed. So I am reopening it.

        Thanks!

        Show
        Cheolsoo Park added a comment - - edited The jira shouldn't be marked as a resolved until its patch is committed. So I am reopening it. Thanks!
        Hide
        William Watson added a comment -

        We could really use this patch at Korrelate. We are wanting to use an existing timestamp as the version in HBase. Thanks for considering pulling it in!

        Show
        William Watson added a comment - We could really use this patch at Korrelate. We are wanting to use an existing timestamp as the version in HBase. Thanks for considering pulling it in!
        Hide
        William Watson added a comment -

        I assume Zhancheng Deng hasn't replied yet, Cheolsoo Park. However, since I need this patch, I have rebased it off of 1.4.4 and redone the patch. I can also add those tests and such and post to the review board if Zhancheng Deng never did so. Just let me know.

        Show
        William Watson added a comment - I assume Zhancheng Deng hasn't replied yet, Cheolsoo Park . However, since I need this patch, I have rebased it off of 1.4.4 and redone the patch. I can also add those tests and such and post to the review board if Zhancheng Deng never did so. Just let me know.
        Hide
        Jarek Jarcec Cecho added a comment -

        William Watson, please go ahead and upload the new patch to the review board. We will be more than happy to review it!

        Show
        Jarek Jarcec Cecho added a comment - William Watson , please go ahead and upload the new patch to the review board. We will be more than happy to review it!
        Hide
        William Watson added a comment -

        Jarek Jarcec Cecho, I have no idea how to use the review board, but I have submitted something... probably. Check this out and let me know if I did it right, please.

        https://reviews.apache.org/r/14377/

        Show
        William Watson added a comment - Jarek Jarcec Cecho , I have no idea how to use the review board, but I have submitted something... probably. Check this out and let me know if I did it right, please. https://reviews.apache.org/r/14377/
        Hide
        William Watson added a comment -

        In fact, I know I didn't do it right b/c I accidentally reviewed my own thing, but still... check it out.

        Show
        William Watson added a comment - In fact, I know I didn't do it right b/c I accidentally reviewed my own thing, but still... check it out.
        Hide
        Jarek Jarcec Cecho added a comment -

        Please accept my apologies for the delay to getting back to you William Watson. It seems that you either did not "published" the review (there is a green box somewhere with button "publish") or did not specified "sqoop" group in the field "reviewers". As a result I can't get to the review. Would you mind taking look on it and ensuring that the review has been published and that Sqoop is in the reviewer group?

        Show
        Jarek Jarcec Cecho added a comment - Please accept my apologies for the delay to getting back to you William Watson . It seems that you either did not "published" the review (there is a green box somewhere with button "publish") or did not specified "sqoop" group in the field "reviewers". As a result I can't get to the review. Would you mind taking look on it and ensuring that the review has been published and that Sqoop is in the reviewer group?
        Hide
        William Watson added a comment -

        Jarek Jarcec Cecho, I thought it wasn't giving me a way to add reviewers, but it turns out the interface is just terrible and/or I'm an idiot.

        Published!

        https://reviews.apache.org/r/14377/

        Show
        William Watson added a comment - Jarek Jarcec Cecho , I thought it wasn't giving me a way to add reviewers, but it turns out the interface is just terrible and/or I'm an idiot. Published! https://reviews.apache.org/r/14377/
        Hide
        Jarek Jarcec Cecho added a comment -

        Hi William Watson, I can confirm that I can access the review now, thank you!

        Show
        Jarek Jarcec Cecho added a comment - Hi William Watson , I can confirm that I can access the review now, thank you!

          People

          • Assignee:
            Unassigned
            Reporter:
            Zhancheng Deng
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development