Pig
  1. Pig
  2. PIG-2821

HBaseStorage should work with secure hbase

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 0.11, 0.10.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Tags:
      hbase

      Description

      HBaseStorage needs to add HBase delegation token to the Job object if hbase security is enabled.

      1. PIG-2821-branch10.patch
        16 kB
        Rohini Palaniswamy
      2. PIG-2821-trunk.patch
        16 kB
        Rohini Palaniswamy
      3. PIG-2821-1.patch
        10 kB
        Rohini Palaniswamy

        Issue Links

          Activity

          Hide
          Daniel Dai added a comment -

          Patch committed to 0.10/trunk. Thanks Rohini!

          Show
          Daniel Dai added a comment - Patch committed to 0.10/trunk. Thanks Rohini!
          Hide
          Daniel Dai added a comment -

          +1, will checkin soon.

          Show
          Daniel Dai added a comment - +1, will checkin soon.
          Hide
          Rohini Palaniswamy added a comment -

          Removed reverting of PIG-2578. Will deal with
          issues from PIG-2578 in PIG-2870 or another jira.

          Changed HBaseStorage to store all hbase properties in UDFContext properties. Credentials was getting added to the Job inadvertently when PigOutputFormat.checkOutputSpecs called setStoreLocation. So there is a way adding credentials works even with PIG-2578 though unexpected.

          Show
          Rohini Palaniswamy added a comment - Removed reverting of PIG-2578 . Will deal with issues from PIG-2578 in PIG-2870 or another jira. Changed HBaseStorage to store all hbase properties in UDFContext properties. Credentials was getting added to the Job inadvertently when PigOutputFormat.checkOutputSpecs called setStoreLocation. So there is a way adding credentials works even with PIG-2578 though unexpected.
          Hide
          Rohini Palaniswamy added a comment -

          Will rework patch to add the hbase-site.xml to UDFContext instead of job conf.

          Show
          Rohini Palaniswamy added a comment - Will rework patch to add the hbase-site.xml to UDFContext instead of job conf.
          Hide
          Rohini Palaniswamy added a comment -

          Added code to fetch delegation tokens and put hbase-site.xml as part of jobconf(PIG-2822) if it is in classapth instead of having to specify --conf option.

          Reverted PIG-2578 as because of that setting config or credentials in job was not actually being passed to the actual job launched. The testcase in MultiQueryLocal was also not valid as the condition if (job.getConfiguration().get(key)==null) was always true. It was always using the suffix initialized in the constructor and never from the job. If a StoreFunc adds config to Job then it should be its responsibility to handle multiple instances of it. HCATALOG-314 handles it for hcatalog. We should not remove the ability for a StoreFunc to set a config on the job.

          Show
          Rohini Palaniswamy added a comment - Added code to fetch delegation tokens and put hbase-site.xml as part of jobconf( PIG-2822 ) if it is in classapth instead of having to specify --conf option. Reverted PIG-2578 as because of that setting config or credentials in job was not actually being passed to the actual job launched. The testcase in MultiQueryLocal was also not valid as the condition if (job.getConfiguration().get(key)==null) was always true. It was always using the suffix initialized in the constructor and never from the job. If a StoreFunc adds config to Job then it should be its responsibility to handle multiple instances of it. HCATALOG-314 handles it for hcatalog. We should not remove the ability for a StoreFunc to set a config on the job.

            People

            • Assignee:
              Rohini Palaniswamy
              Reporter:
              Francis Liu
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development