I had added a comment to clarify in the beginning of the first if block. If there is no security defined in hbase-site.xml, the whole block will not get executed. So we don't expect the block to be executed with hadoop 0.20.2 or hbase 0.90.x which do not have security. We are using reflection so that when HBaseStorage class compiled against hadoop 1.x but run against 0.20.2 or 0.90.x does not throw NoSuchMethodError for UserGroupInformation methods when executing the addHBaseDelegationToken method.
Looks like now that the Class.forName line moved into an if block, the corresponding ClassNotFoundException should move in there as well.
ClassNotFoundException happens in the outer if block as well. So not using a separate try catch block for the inner if block.
The description in this JIRA says " it should try and reuse the hbase delegation token in JobConf " if the kerberos token is not available. Where does this reuse happen?
Pig in the front end (when run via command line) has access to the user TGT (either keytab or kinit from command line). It fetches the HBase delegation token by authenticating to HBase using KERBEROS and adds it to the JobConf. In the backend, HTable initiation picks up the delegation token from the JobConf and does further operations on HBase using DIGEST authentication. The getDelegationToken call is one call which cannot be done using DIGEST authentication(delegation token). The client always needs to authenticate with Kerberos to get the delegation token.
The case with Oozie is that the pig frontend run happens on a mapper and there is no access to user TGT. The JobConf passed to pig (using mapreduce.job.credentials.binary) already has the required delegation tokens to talk to all services - HDFS, JT and HBASE. In HBaseStorage, we were always trying to get the hbase delegation token. Since there was no TGT, it was failing in the pig launcher mapper. Now just added an extra check, so that we don't try to fetch the token if we don't have TGT.
Yes. It would be good to apply this to 0.11.1 as well. I will do it.
I will mark it back as patch available if it clarifies your questions. If you need me to add any comments to make it clear, I can do it.