[HBASE-17997] In dev environment, add jruby-complete jar to classpath only when jruby is needed

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Labels: s3
• Hadoop Flags: Reviewed

Release Note

When JRUBY_HOME is specified, if the command is "hbase shell" or "hbase org.jruby.Main", CLASSPATH and HBASE_OPTS will be updated according to JRUBY_HOME specified
* Jar under JRUBY_HOME is added to CLASSPATH
* The following will be added into HBASE_OPTS

-Djruby.home=$JRUBY_HOME -Djruby.lib=$JRUBY_HOME/lib


That is, as long as JRUBY_HOME is specified, JRUBY_HOME specified will take precedence.
* In dev env, the jar recorded in cached_classpath_jruby.txt will be ignored
* In non dev env, jruby-complete jar packaged with HBase will be ignored

• Assignee: Xiang Li
• Reporter: Ted Yu
• Created: 04/May/17 18:49
• Updated: 17/Jan/19 05:58
• Resolved: 25/May/17 15:08

Description

HBASE-15199 moves jruby-complete-1.6.8.jar to lib/ruby directory.

However, jruby-complete-1.6.8.jar still appears in cached_classpath.txt

This means that user would see exception similar to the following when starting hbase in standalone mode with s3a as rootdir :

2017-05-04 16:41:32,854 WARN  [RpcServer.FifoWFPBQ.priority.handler=18,queue=0,port=38659] internal.S3MetadataResponseHandler: Unable to parse last modified date: Thu, 04 May 2017 16:27:09 GMT
java.lang.IllegalStateException: Joda-time 2.2 or later version is required, but found version: null
  at com.amazonaws.util.DateUtils.handleException(DateUtils.java:149)
  at com.amazonaws.util.DateUtils.parseRFC822Date(DateUtils.java:195)
  at com.amazonaws.services.s3.internal.ServiceUtils.parseRfc822Date(ServiceUtils.java:78)
  at com.amazonaws.services.s3.internal.AbstractS3ResponseHandler.populateObjectMetadata(AbstractS3ResponseHandler.java:115)
  at com.amazonaws.services.s3.internal.S3ObjectResponseHandler.handle(S3ObjectResponseHandler.java:52)
  at com.amazonaws.services.s3.internal.S3ObjectResponseHandler.handle(S3ObjectResponseHandler.java:30)
  at com.amazonaws.http.AmazonHttpClient.handleResponse(AmazonHttpClient.java:1072)
  at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:746)
  at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489)
  at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
  at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785)
  at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1191)
  at org.apache.hadoop.fs.s3a.S3AInputStream.reopen(S3AInputStream.java:148)
  at org.apache.hadoop.fs.s3a.S3AInputStream.lazySeek(S3AInputStream.java:281)
  at org.apache.hadoop.fs.s3a.S3AInputStream.read(S3AInputStream.java:364)
  at org.apache.hadoop.fs.FSInputStream.read(FSInputStream.java:75)
  at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:92)
  at org.apache.hadoop.hbase.io.hfile.HFileBlock.positionalReadWithExtra(HFileBlock.java:722)
  at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1420)
  at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1677)
  at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1504)
  at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:439)
  at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:904)
  at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:267)
  at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:169)
  at org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:363)
  at org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:217)
  at org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2132)
  at org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2122)
  at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5687)
  at org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2679)
  at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2665)
  at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2647)
  at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6906)
  at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6885)
  at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2007)