Source changes - FishEye

Shows the 20 most recent commits for HBase.

Suresh Subbiah <sureshs@apache.org> committed c66fca2502db7f75a285f93ca2a83169745b25e9 (0 files)
Reviews: none

Merge [TRAFODION-2775] PR-1267 Insert does not raise duplicate row error for hbase format table with defaulted first column

Suresh Subbiah <sureshs@apache.org> committed 921c0a5840a7f6f233166b1d745a4a7f65f49118 (10 files)
Reviews: none

[TRAFODION-2775] Insert does not raise duplicate row error for hbase format table with defaulted first column
There was a problem with column being checked during CheckAndPut call for such tables. We were always checking the first column in the row, which may not have existed in the previous version of this rowid. Please see JIRA for a complete explanation.

Suresh Subbiah <sureshs@apache.org> committed ab59ee768d51edfa80a0e258ca58c300273e658b (12 files)
Reviews: none

[TRAFODION-2758] Sort operator that uses TopN sort at runtime is not accurately determined at compile time
This commit includes fixes for these three issues. Please see JIRA for a description of each fix.

[TRAFODION-2758] Sort operator that uses TopN sort at runtime is not accurately determined at compile time
[TRAFODION-2759] Use of rowset with large memory leads to various errors at runtime
[TRAFODION-2760] hbase cache blocks is OFF for broad table with narrow index

[TRAFODION-2758]
comexe/ComTdbSort.cpp
comexe/ComTdbSort.h
executor/ex_sort.cpp
generator/GenExplain.cpp
generator/GenPreCode.cpp
generator/GenRelMisc.cpp
optimizer/RelExpr.cpp
[TRAFODION-2759]
bin/SqlciErrors.txt
generator/GenRelPackedRows.cpp
sqlcomp/DefaultConstants.h
sqlcomp/nadefaults.cpp
[TRAFODION-2760]
generator/GenRelScan.cpp

Paul Rogers <progers@maprtech.com> committed 42f7af22fc5d713aac07e057fd374ccd674e40df (23 files)
Reviews: none

DRILL-5830: Resolve regressions to MapR DB from DRILL-5546
- Back out HBase changes
- Code cleanup
- Test utilities
- Fix for DRILL-5829

closes #968

drill master
Suresh Subbiah <sureshs@apache.org> committed e68aa43ae2f99e571095763b6e7acd51114295da (9 files)
Reviews: none

[TRAFODION-2730] process loop when infer_charset cqd is ON and query invalidation is done
This PR also includes fixes for [TRAFODION-2747] and [TRAFODION-2748].
The changes are small and the problems not severe.
Please see JIRA for an explanation

Add aditional change has been made to explain output.
cache_blocks ON is now added to explain for HBase scans. Previously
explain would only show cache_blocks OFF, if blockcache was disabled for a
HBase scan.

Files
2730 - QCache.cpp
2747 - ExSequence.cpp, ExAtp.h
2748 - GenPreCode.cpp
cache_block - GenExplain.cpp, 4 regression EXPECTED files

Ray Chiang <rchiang@apache.org> committed 2ffe93ab609dfb54c3d1a53273ac2bc5ad15a5dd (1 file)
Reviews: none

HADOOP-14856. Fix AWS, Jetty, HBase, Ehcache entries for NOTICE.txt. (rchiang)

Ray Chiang <rchiang@apache.org> committed 99e7b264431ba59987693b3e39e392afbbe86302 (1 file)
Reviews: none

HADOOP-14856. Fix AWS, Jetty, HBase, Ehcache entries for NOTICE.txt. (rchiang)
(cherry picked from commit 2ffe93ab609dfb54c3d1a53273ac2bc5ad15a5dd)

selvaganesang <selva.govindarajan@esgyn.com> committed 00e3b87432feb9d84455ca8d20e9899611fdb7ff (45 files)
Reviews: none

Following changes are done in BMO memory quota
1) Enabled memory quota per node. The CQD BMO_MEMORY_LIMIT_PER_NODE
(renamed from EXE_MEMORY_LIMIT_PER_CPU) is set to 10240 MB by default.

Old attribute Old value Renamed Attribute New value
EXE_MEMORY_LIMIT_PER_CPU 0 BMO_MEMORY_LIMIT_PER_CPU 10240
EXE_MEMORY_LIMIT_LOWER_BOUND_HASHGROUPBY 10 BMO_MEMORY_LIMIT_LOWER_BOUND_HASHGROUPBY 25
EXE_MEMORY_LIMIT_LOWER_BOUND_HASHJOIN 10 BMO_MEMORY_LIMIT_LOWER_BOUND_HASHJOIN 25
EXE_MEMORY_LIMIT_LOWER_BOUND_PROBE_CACHE 10 BMO_MEMORY_LIMIT_LOWER_BOUND_PROBE_CACHE 25
EXE_MEMORY_LIMIT_LOWER_BOUND_SORT 10 BMO_MEMORY_LIMIT_LOWER_BOUND_SORT 200

2) Changes in EXPLAIN
   Estimated memory per node for all BMOs at ROOT operator
   Estimated memory per instance for every BMO operator
   Memory quota per instance for every BMO operator
3) BMO TDB contains the memory quota per esp instance now.
4) Root TDB now contains the limit per node and estimated memory per node.
   This can be used by WMS to change the memory allocation during
   runtime without compilation. - Not yet implemented.
4) Added a CQD BMO_MEMORY_LIMIT_UPPER_BOUND to gap the memory
   consumed by BMO by the same queries with less number of
   bmos.
5) The unused memory quota is yielded to other fragments in the process
   also.
6) Removed the code to limit the ESPs from being assigned to a fragement
   based on the BMO memory quota.
7) Added a new CQD BMO_MEMORY_ESTIMATE_RATIO_CAP to gap the memory
   estimate skew by any one BMO operator to 0.7.
8) To disable the memory quota per node, set BMO_MEMORY_LIMIT_PER_NODE to 0.
9) This memory quota is distributed proportionally based on the estimated memory
   taking into consideration the number of bmo instances per operator and
   the number of nodes available in the cluster to host these instances.
   Hence, this memory quota should be valid in multi-fragments independent of the
   number of fragments in an ESP.
10) Removed the CQD EXE_MEMORY_LIMIT_NONBMOS_PERCENT and EXE_MEMORY_RESERVED_FOR_MXOSRVR_IN_MB
11) Fixed BMO stats WM to be at least the allocated memory.
12) Changed the sort operator to account the bmo memory correctly.
(cherry picked from commit ba19c04a58890fdd845b03f8d915abdd487b6407)

Conflicts:

core/sqf/src/seatrans/hbase-trx/src/main/java/org/apache/hadoop/hbase/coprocessor/transactional/SplitBalanceHelper.java
core/sql/cli/Context.cpp
core/sql/executor/ex_frag_rt.cpp
core/sql/executor/ex_sort.cpp
core/sql/regress/executor/EXPECTED131
core/sql/regress/executor/EXPECTED140
core/sql/regress/hive/EXPECTED009
core/sql/regress/hive/EXPECTED030
core/sql/regress/hive/FILTER009
core/sql/regress/seabase/EXPECTED010
core/sql/regress/seabase/EXPECTED011
core/sql/regress/seabase/EXPECTED016
core/sql/sqlcomp/DefaultConstants.h

Dave Birdsall <dbirdsall@apache.org> committed edbe1cce9106efac40c0efa9cb2193baca1acf64 (1 file)
Reviews: none

[TRAFODION-2729] Fix bug reading region keys for hbase _ROW_ and _CELL_ access

Claus Ibsen <davsclaus@apache.org> committed 3c6a8293582e8bce70fb49490c03d2e2a2921a88 (1 file)
Reviews: none

Upgrade hbase. This fixes #1896

selvaganesang <selva@apache.org> committed a7a9b0f6b3d19f6632a7d9be333c63cf22a27516 (1 file)
Reviews: none

Merge PR 1211 [TRAFODION-2722] Change purgedata to use HBase truncate api

Anoop Sharma <anoop.sharma@esgyn.com> committed a3669571d6df1d6b6aa2ae8b521dd662f2a910a7 (5 files)
Reviews: none

TRAFODION-2722 Change purgedata to use HBase truncate api
Traf now uses HBase truncate api to delete all rows from
a table and indexes, instead of dropping/recreating underlying HBase
objects.

A second change has also been added to fix a bug where compiler cache
was being cleared whenever a DDL statement was issued.
This change was added a while back before Query Invalidation(QI) feature
was available and was not removed when QI was added.
It has now been removed.

Vrushali C <vrushali@apache.org> committed 71e59ff80f3bf2dee927cd4496aef7aeb52a8729 (5 files)
Reviews: none

YARN-6905 Multiple HBaseTimelineStorage test failures due to missing FastNumberFormat (Contributed by Haibo Chen)
(cherry picked from commit 608a06cca5d68b3155bd70a94bf29ae0942b9ca0)

Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/TestHBaseTimelineStorageEntities.java
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/HBaseTimelineStorageUtils.java

Vrushali C <vrushali@apache.org> committed a2598434ed9b2414e6028e5b2c1303419ebbf4ad (5 files)
Reviews: none

YARN-6905 Multiple HBaseTimelineStorage test failures due to missing FastNumberFormat (Contributed by Haibo Chen)
(cherry picked from commit 608a06cca5d68b3155bd70a94bf29ae0942b9ca0)

Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/TestHBaseTimelineStorageEntities.java
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/HBaseTimelineStorageUtils.java

Vrushali C <vrushali@apache.org> committed f735024e8d47d582220a8e849c7b49625f065029 (5 files)
Reviews: none

YARN-6905 Multiple HBaseTimelineStorage test failures due to missing FastNumberFormat (Contributed by Haibo Chen)
(cherry picked from commit 608a06cca5d68b3155bd70a94bf29ae0942b9ca0)

Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/TestHBaseTimelineStorageEntities.java
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/HBaseTimelineStorageUtils.java