Source changes - FishEye

Shows the 20 most recent commits for HBase.

Suresh Subbiah <sureshs@apache.org> committed 3e5dd571cbfb7a486a0e4e86f6e52845ef2a0579 (0 files)
Reviews: none

Merge [TRAFODION-2663] PR-1140 Simplify HBase config settings in installers

Steve Varnau <steve.varnau@esgyn.com> committed 35f929954c11ab92df230d39f84507369e799c0c (1 file)
Reviews: none

[TRAFODION-2663] move hbase client setting to trafodion-site.xml

Steve Varnau <steve.varnau@esgyn.com> committed c0e395117a66a81904ba1650f7996ad35d740f94 (2 files)
Reviews: none

[TRAFODION-2663] Use hbase bulkload staging dir default value
This change-set exposed a race condition in installation.

Hbase regionserver was dying because pyinstaller is starting
regionserver before the new staging dir exists in hdfs. Prior to
simplifying the settings, installer won the race of creating hdfs
dir before regionserver looked for it.

To address this, we should just use HBase default value for.

Also remove hbase setting for hbase.client.keyvalue.maxsize from
installer. That can be added to trafodion-site.xml if needed.

Steve Varnau <steve.varnau@esgyn.com> committed 98620b76e7625f8ec9c73b64a0d64e6d9f5eb66b (1 file)
Reviews: none

[TRAFODION-2663] Add re-check in hbcheck
Simplifying the HBase config seems to have changed the timing of
HBase start-up on HDP nodes. We are trying to talk to HBase while
master is still initializing. I have added re-tries in hbcheck
to compensate.

Hans Zeller <hzeller@apache.org> committed ba20e3fabf1adf58fdb1287dc9b67055118aab23 (26 files)
Reviews: none

[TRAFODION-2637] Library management with Cloudera parcels
Also in this commit:

[TRAFODION-2581] Combine JVM startup code of SQL executor and UDR server

We have library management SPJs that allow a user to deploy library
files from a client machine. These SPJs make use of the pdcp command
that is installed on many clusters, but not always. The new method
uses temporary HDFS files and a TMUDF to distribute the library
file to all nodes of the cluster.

For now I left the old PUT method in place and added a new PUTFILE
method. We can remove the old method later.

A few other things fixed in the process:

- Make sure that when a UDR server calls the executor and uses a JVM,
  we share the same JVM. To do this, we need to use the same startup
  method, and I needed to change the executor startup method to accept
  the extra options used for the UDR server.

- Got rid of the "HBase compatibility mode" in JavaObjectInterface.

- A common method to execute SQL in the library management code.

- removed the obsolete init_libmgmt.sh file.

- We have special code that makes sure a UDR can request to run in
  parallel, including a request to run on all nodes. Expanded this
  special code from predefined UDFs to any UDF.

Steve Varnau <steve.varnau@esgyn.com> committed aaa5abf493baa04d11e482f752bf4d6b863f7279 (4 files)
Reviews: none

[TRAFODION-2663] Simplify HBase config settings in installers
Remove settings that are now in per-table trafodion-site.xml config.

For Ambari, 2.2 service_advisor over-rides the 2.1 version. This allows
possibility of single plug-in to manage 2.1 & 2.2 installs, though this is not
tested.

Anoop Sharma <anoop.sharma@esgyn.com> committed 5ea4af2e94ad2f86b562a516bd56470dd959f086 (0 files)
Reviews: none

PR-1126 Multiple fixes: REPLACE func maxlen, hbase unreg, DDLExpr cleanup

Anoop Sharma <anoop.sharma@esgyn.com> committed d4cbd838b18b6496b8d392d7cb0d3c21035a8612 (11 files)
Reviews: none

Multiple fixes: REPLACE func maxlen, hbase unregister, DDLExpr cleanup
-- REPLACE function max length is no longer limited to 32K
-- HBase object unregister was not working correctly. That is fixed.
-- class DDLExpr has been cleaned up to have one constructor for all cases.
   All options are now set using flags instead of constructor params.
-- "showddl <tab>, explain " command now returns an error instead of crash.

luochen01 <cluo8@uci.edu> committed 7505a796f8c480c72f2e42893188bdf4e092528c (3 files)
Reviews: none

Avoid always merging old components in prefix policy
Current, the prefix policy always looks at the components from
oldest to newest to schedule merge. One negative consequence is that
the oldest (largest) component gets merged over and over again
until it reaches the size limit. This is undesirable since it takes
O(n^2) disk IOs (n is the number of flushed components) to produce a
final component.

This patch is a temporary fix of this behavior, taken from the idea of
HBase compaction policy (https://www.ngdata.com/visualizing-hbase
-flushes-and-compactions/). The basic idea is that it introduces
some size factor (for now it's 1.2) to control the merge behavior.

When the prefix policy finds a sequence of components to merge,
we also check the oldest (largest) component in the sequence should
be smaller than 1.2*the total size of all younger components.
By doing so, we can avoid merging oldest components over and over again,
making the disk IOs O(nlog n).

Change-Id: I464da3fed38cded0aee7b319a35664eae069a2ba
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1818
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Varun Saxena <varunsaxena@apache.org> committed 0763450e479563d7c897227718f500dd71c0ab97 (1 file)
Reviews: none

YARN-6707. [ATSv2] Update HBase version to 1.2.6 (Vrushali C via Varun Saxena)
(cherry picked from commit e86eef9812d055f8689646b0f521f80cfc0c6399)

Varun Saxena <varunsaxena@apache.org> committed 71474506edd86a741ea5532de907c95281d6310a (1 file)
Reviews: none

YARN-6707. [ATSv2] Update HBase version to 1.2.6 (Vrushali C via Varun Saxena)
(cherry picked from commit e86eef9812d055f8689646b0f521f80cfc0c6399)

Conflicts:
hadoop-project/pom.xml

Varun Saxena <varunsaxena@apache.org> committed e86eef9812d055f8689646b0f521f80cfc0c6399 (1 file)
Reviews: none

YARN-6707. [ATSv2] Update HBase version to 1.2.6 (Vrushali C via Varun Saxena)

Dave Birdsall <dbirdsall@apache.org> committed ccb568462b5cdb75343d47c42a113a7c267fca24 (1 file)
Reviews: none

Merge [TRAFODION-2635] PR 1114 Fix 2 bugs in Hbase CELL access to Traf objs

Varun Saxena <varunsaxena@apache.org> committed ee6a4d0e1905d25c14c0c02e8cdad47013d0252f (7 files)
Reviews: none

YARN-6658. Remove columnFor() methods of Columns in HBaseTimeline backend (Haibo Chen via Varun Saxena)
(cherry picked from commit f7cba98504a5e968bb6ae976d20b71c84a4664f2)

Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/apptoflow/AppToFlowColumn.java

selvaganesang <selva.govindarajan@esgyn.com> committed e458290995e419361872822466bcf2acd568a9cc (24 files)
Reviews: none

[TRAFODION-2654] Change the location of trafodion-site.xml from $TRAF_HOME/etc to config [TRAFODION-2653] Sort operator loops at times
All HBase client connections from Trafodion uses Trafodion configuration
concept. Any client side property can now be set in trafodion-site.xml.

The scratch files are now open in blocking mode. Hence any scratch file
operation shouldn't return EAGAIN.

The internal table TDDL of table is now created without any table
coprocessors attached to it.

Anoop Sharma <anoop.sharma@esgyn.com> committed b109e2cf5883f8e763af853ab6fad7ce7110d9e8 (16 files)
Reviews: none

jira trafodion-2621, trafodion-2619 + others
-- trafodion-2621: add support to create like native hive or hbase tables
-- trafodion-2619: support to_date format 'yyyy/mm/dd'
-- alter table drop column on a table with user indexes would sometime
   fail. That has been fixed.
-- get schemas in a session where default catalog is set to hive,
   now correctly returns hive schemas
-- update stats on hive tables succeeds
-- update stats on hive view returns error as this operation
   is not supported

Zalo Correa <zalo.correa@esgyn.com> committed f7f69521f89ba92b7a0041d5ccd5a2f6757bf657 (1 file)
Reviews: none

Merge remote branch 'origin/master' into TRAFODION-2001
Conflicts:
core/sqf/src/seatrans/hbase-trx/src/main/java/org/apache/hadoop/hbase/regionserver/transactional/TrxTransactionState.java.tmpl

Anoop Sharma <anoop.sharma@esgyn.com> committed 113c4de556651068f9f0162029c1c1d5ec2f2ba3 (0 files)
Reviews: none

PR-1090 [TRAFODION-2610] Register native HBase tables in traf MD

Arina Ielchiieva <arina.yelchiyeva@gmail.com> committed 7f98400f949b04eb06415aeb3a3265693629371c (1 file)
Reviews: none

DRILL-5516: Limit memory usage for Hbase reader
close apache/drill#839