Uploaded image for project: 'HBase'

HBase

Source changes - FishEye

Shows the 20 most recent commits for HBase.

Anoop Sharma <anoop.sharma@esgyn.com> committed 113c4de556651068f9f0162029c1c1d5ec2f2ba3 (0 files)
Reviews: none

PR-1090 [TRAFODION-2610] Register native HBase tables in traf MD

Arina Ielchiieva <arina.yelchiyeva@gmail.com> committed 7f98400f949b04eb06415aeb3a3265693629371c (1 file)
Reviews: none

DRILL-5516: Limit memory usage for Hbase reader
close apache/drill#839

Anoop Sharma <anoop.sharma@esgyn.com> committed 6ffe0b5b6097ab503965f40734ba7f0275892f01 (49 files)
Reviews: none

TRAFODION-2610 Register native HBase tables in traf MD plus couple more
-- support for registering hbase native cell/row format tables in traf MD
   Syntax:
       register [internal] hbase table [if not exists] <table-name>
       unregister [internal] hbase table [if exists] <table-name>

-- support to get info related to registered hbase tables
   Syntax: get hbase registered table in catalog trafodion

-- enhancements to HBase mapped tables and bug fixes

-- disabling of creation of external tables on native hbase
   tables unless they are mapped tables

-- support for case-insensitive format string in TO_DATE function
   (exp/exp_datetime.h)

-- regress/seabase/TEST022 enhanced with additional tests

Sean Broeder <sbroeder@edev05.novalocal> committed 91794b534788a1fc578192b381bcaccddb1680e1 (22 files)
Reviews: none

This is a large contribution of changes from Esgyn TransactionManager and libraries that are collectively much better tested and hardened than Trafodion, but are too numerous and complex to cherry pick individually.
The changes include:
Dozens of bug fixes
Improved recovery after HBase failures
Better error reporting and tracing, which includes region keys for easier debugging
Reduced UnknownTransactionExceptions resulting from duplicate transaction registration
Better TLOG management including reducing the default number of tables

Trafodion master
Hans Zeller <hzeller@apache.org> committed 5b686a616be71fb678dbb20a181846237fad0334 (0 files)
Reviews: none

Merge [TRAFODION-2592] Better error message when HBase is down at startup. PR-1063

Hans Zeller <hzeller@apache.org> committed ebfd515abb947996fe7effe9475dc0be3b1b3db0 (3 files)
Reviews: none

[TRAFODION-2592] Better error message when HBase is down at startup.
This code was written by Narendra Goyal.

selvaganesang <selva.govindarajan@esgyn.com> committed 0c10290402c121eec54541d3720c3f407155722a (15 files)
Reviews: none

[TRAFODION-2420] RMS enhancements
Improved BMO operation accounting. Two new counters are added
a) scratchIOSize - Size in KB of the scratch IO
b) scratchIOMaxTime - Time in microseconds taken by any long pole
                 ESP instance to do scratch IO operations

The ScrBufSize, ScrBufRead and ScrBufWritten are no longer displayed
in the formatted get statistics command outputs.

Reduced the default value of CQD HBASE_NUM_CACHE_ROWS_MAX to 1024 from
10000 to reduce stress on the client JVM memory.

Disabled the memory pressure triggering in the BMO operators by increasing
the threshold GEN_MEM_PRESSURE_THRESHOLD to 10000 from 100. The memory
pressure detection code was inadvertently enabled when the cap to limit
memory pressure constant to 100 was removed.

Memory pressure triggers in BMO operators will be enabled later when
we understand the memory pressure detection better.

Varun Saxena <varunsaxena@apache.org> committed 288201346ec459b4afea184b0dee4937fe9d7cf5 (2 files)
Reviews: none

YARN-6414. ATSv2 HBase related tests fail due to guava version upgrade (Haibo Chen via Varun Saxena)

Varun Saxena <varunsaxena@apache.org> committed b8838578ce1f1ae1f93cc6b40cc98b58321ad9ee (2 files)
Reviews: none

YARN-6414. ATSv2 HBase related tests fail due to guava version upgrade (Haibo Chen via Varun Saxena)

Anoop Sharma <anoop.sharma@esgyn.com> committed 8aa532330e4dbf316a09817a033292c85cb02c4e (31 files)
Reviews: none

Various fixes, details below
-- max length limited to 16777216 bytes (16M)
   for char cols and functions (repeat, concat).
  (optimizer/SynthType.cpp, common/ComSmallDefs.h, sqlcomp/nadefaults.cpp)

-- previous max length change requires adding of a new hbase property
   called hbase.client.keyvalue.maxsize so large key/values could be
   handled in hbase cell.

   Following scripts have been updated to handle that.
   (sqf/sql/scripts/install_local_hadoop,
    install/ambari-installer/traf-mpack/common-services/TRAFODION/2.1/
         service_advisor.py
    install/installer/traf_hortonworks_mods
    install/python-installer/configs/mod_cfgs.json)

   Developers can also update hbase-site.xml with this property if
   they dont want to reinstall local hadoop.
      <property>
        <name>hbase.client.keyvalue.maxsize</name>
        <value>0</value>
      </property>

-- while accessing a hive table as an external table, the hive table and
   corresponding external table definitions are validated to be the
   same. This validates that corresponding columns have the same
   data attributes (type, length, scale, etc).
   This check causes failures if hive column is of 'string' datatype.
   That is because hive 'string' column length can be changed by
   a cqd but the corresponding external table has predefined length
   set when the table is created.
   The validation check now ignores the length attribute if the hive column is
   of 'string' datatype.
   (optimizer/BindRelExpr.cpp, common/CharType.*, NAType.*)

-- data moved into direct buffer would sometimes cause overflow and crash.
   The max direct buffer length used to send/retrieve hbase data
   is now limited to 1G (executor/ExHbaseAccess.cpp)

-- errors during vsbb upsert are now handled correctly
  (executor/ExHbaseIUD.cpp)

-- support for GET CATALOGS command
   (generator/GenRelExeUtil.cpp, executor/ExExeUtilGet.cpp)

-- An incorrect computation would sometimes cause group by rollup to
   crash compiler in NAHeap::unlinkLargeFragment()
  (generator/GenRelGrby.cpp)

-- sort of a large row that exceeded sort pre-set buffer size would crash
   if the row size exceeded the max buffer size.
   That has been fixed by allocating space for atleast one row.
  (generator/GenRelMisc.cpp)

-- purgedata on a delimited name was failing. That has been fixed.
   (optimizer/RelExeUtil.h)

-- regress/tools/runregr_privs1/privs2 fixed to handle running of
   subset of tests

-- regress/seabase/TEST031 updated with new tests

Varun Saxena <varunsaxena@apache.org> committed 1862a03030a26876f701cd842fce17f272fbb8fb (10 files)
Reviews: none

YARN-6027. Support fromid(offset) filter for /flows API (Rohith Sharma K S via Varun Saxena)
(cherry picked from commit 63c06ec44e633567c378e28898e319143593ff30)

Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TestTimelineReaderWebServicesHBaseStorage.java

hadoop YARN-5355-branch-2
Varun Saxena <varunsaxena@apache.org> committed ff55ff2747283d5d1745bb528aeaa65b773e8dba (10 files)
Reviews: none

YARN-6027. Support fromid(offset) filter for /flows API (Rohith Sharma K S via Varun Saxena)
(cherry picked from commit 63c06ec44e633567c378e28898e319143593ff30)

Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TestTimelineReaderWebServicesHBaseStorage.java

hadoop
Suresh Subbiah <sureshs@apache.org> committed d7793509b11200f542dd1e6e5382567a11b27973 (0 files)
Reviews: none

Merge [TRAFODION-2475] PR-965 Improved error handling for HBASE_OPTIONS clause

Suresh Subbiah <sureshs@apache.org> committed ded3fa21f06b16d5badffec282c374797f6a8ef0 (2 files)
Reviews: none

Fix testware
On HDP installs, HBase sanity checks arguments more extensively. This change makes the attribute values used in test table more reasonable.

Suresh Subbiah <sureshs@apache.org> committed 93b5baa16e746097a825cd60e853bf142672f7fe (5 files)
Liu Ming <ovis_poly@sina.com> committed 3d7a612587b3d65b0a40c2c16f276b36f502bd0e (0 files)
Reviews: none

Merge Fix for [TRAFODION-2471] pyinstaller: error command when grant privilege in HBase.

Eason <hfutmkby@gmail.com> committed 0c3cdbdf0f487c79728fdf1e17a63b9e3c07b571 (1 file)
Reviews: none

[TRAFODION-2471] pyinstaller: error command when grant privilege in HBase.

Dave Birdsall <dbirdsall@apache.org> committed bce610d7272c574a0b79c36a1342e14af49da1d1 (0 files)
Reviews: none

Merge [TRAFODION-2472] PR 947 Disallow transactional ALTER xx HBASE_OPTIONS