Source changes - FishEye

Shows the 20 most recent commits for Hive.

Volodymyr Vysotskyi <vvovyk@gmail.com> committed 0049f17331462e0c7940bb5b5b79eab8e75d2a86 (1 file)
Reviews: none

DRILL-5857: Fix NumberFormatException in Hive unit tests
closes #980

Sandhya Sundaresan <sandhya@apache.org> committed f49b86f6c1c0ae17d5d4f3366b421e774f7ce56a (2 files)
Reviews: none

Fix to make hive hdfs filenames more unique

Hans Zeller <hzeller@apache.org> committed 4706888f1c6ba23f23340a3e4400e98310fbdf65 (0 files)
Reviews: none

Merge Fix for regression failure in hive/TEST007. PR-1256

Hans Zeller <hzeller@apache.org> committed 753ab825b8f4234691ea3d83c75dfc8b5c3e6c62 (1 file)
Reviews: none

Fix for regression failure in hive/TEST007.
Missing TRAFODION in metadata query.

Andy Yang <yongfeng.yang@esgyn.cn> committed e0e8890f08ec0fa6ed659b1b8461a4170cd00aed (1 file)
Reviews: none

[mantis-4973] add fix for hive table and table under seabase.

selvaganesang <selva.govindarajan@esgyn.com> committed 00e3b87432feb9d84455ca8d20e9899611fdb7ff (45 files)
Reviews: none

Following changes are done in BMO memory quota
1) Enabled memory quota per node. The CQD BMO_MEMORY_LIMIT_PER_NODE
(renamed from EXE_MEMORY_LIMIT_PER_CPU) is set to 10240 MB by default.

Old attribute Old value Renamed Attribute New value
EXE_MEMORY_LIMIT_PER_CPU 0 BMO_MEMORY_LIMIT_PER_CPU 10240
EXE_MEMORY_LIMIT_LOWER_BOUND_HASHGROUPBY 10 BMO_MEMORY_LIMIT_LOWER_BOUND_HASHGROUPBY 25
EXE_MEMORY_LIMIT_LOWER_BOUND_HASHJOIN 10 BMO_MEMORY_LIMIT_LOWER_BOUND_HASHJOIN 25
EXE_MEMORY_LIMIT_LOWER_BOUND_PROBE_CACHE 10 BMO_MEMORY_LIMIT_LOWER_BOUND_PROBE_CACHE 25
EXE_MEMORY_LIMIT_LOWER_BOUND_SORT 10 BMO_MEMORY_LIMIT_LOWER_BOUND_SORT 200

2) Changes in EXPLAIN
   Estimated memory per node for all BMOs at ROOT operator
   Estimated memory per instance for every BMO operator
   Memory quota per instance for every BMO operator
3) BMO TDB contains the memory quota per esp instance now.
4) Root TDB now contains the limit per node and estimated memory per node.
   This can be used by WMS to change the memory allocation during
   runtime without compilation. - Not yet implemented.
4) Added a CQD BMO_MEMORY_LIMIT_UPPER_BOUND to gap the memory
   consumed by BMO by the same queries with less number of
   bmos.
5) The unused memory quota is yielded to other fragments in the process
   also.
6) Removed the code to limit the ESPs from being assigned to a fragement
   based on the BMO memory quota.
7) Added a new CQD BMO_MEMORY_ESTIMATE_RATIO_CAP to gap the memory
   estimate skew by any one BMO operator to 0.7.
8) To disable the memory quota per node, set BMO_MEMORY_LIMIT_PER_NODE to 0.
9) This memory quota is distributed proportionally based on the estimated memory
   taking into consideration the number of bmo instances per operator and
   the number of nodes available in the cluster to host these instances.
   Hence, this memory quota should be valid in multi-fragments independent of the
   number of fragments in an ESP.
10) Removed the CQD EXE_MEMORY_LIMIT_NONBMOS_PERCENT and EXE_MEMORY_RESERVED_FOR_MXOSRVR_IN_MB
11) Fixed BMO stats WM to be at least the allocated memory.
12) Changed the sort operator to account the bmo memory correctly.
(cherry picked from commit ba19c04a58890fdd845b03f8d915abdd487b6407)

Conflicts:

core/sqf/src/seatrans/hbase-trx/src/main/java/org/apache/hadoop/hbase/coprocessor/transactional/SplitBalanceHelper.java
core/sql/cli/Context.cpp
core/sql/executor/ex_frag_rt.cpp
core/sql/executor/ex_sort.cpp
core/sql/regress/executor/EXPECTED131
core/sql/regress/executor/EXPECTED140
core/sql/regress/hive/EXPECTED009
core/sql/regress/hive/EXPECTED030
core/sql/regress/hive/FILTER009
core/sql/regress/seabase/EXPECTED010
core/sql/regress/seabase/EXPECTED011
core/sql/regress/seabase/EXPECTED016
core/sql/sqlcomp/DefaultConstants.h

Vitalii Diravka <vitalii.diravka@gmail.com> committed 4c99f0cdd98b1a0e789c5d11b353f88a2f6d54aa (3 files)
Steve Varnau <svarnau@apache.org> committed 250f60acb9b4a62fbdd581f6b3ab1da0e57806d9 (1 file)
Reviews: none

[TRAFODION-2697] Make sure trafodion is member of hbase, hive groups

Anoop Sharma <anoop.sharma@esgyn.com> committed 5a244d532425a6a4aff19a5cd2cd6ab637ea9b0a (20 files)
Reviews: none

Few fixes, details listed below.
-- fix an issue where multiple values inserted from a list would return
  error but each value inserted on its own would succeed.
  ex: create table ts (a timestamp);
      insert into ts values ('2017-01-01 10:10:10'), ('2018-01-01 10:10:10');

-- sometimes errors returned from child during hive inserts were not
   being returned. That has been fixed.

-- TRAFODION-2683 extension.
   added a 'p' (prune) option which would cleanse and filter unneeded
   explain output. This helps in reducing output especially
   for larger explains.
Ex:
>>explain option 'p' select * from dual;

------------------------------------------------------ PLAN SUMMARY
STATEMENT_NAME ........... NOT NAMED
STATEMENT ................ select * from dual;

------------------------------------------------------- NODE LISTING
ROOT ====================================== SEQ_NO 2 ONLY CHILD 1
DESCRIPTION
  fragment_id ............ 0
  parent_frag ............ (none)
  fragment_type .......... master
  xn_access_mode ......... read_only
  auto_query_retry ....... enabled
  embedded_arkcmp ........ used
  select_list ............ %(0)
  input_variables ........ %(0), %(0), %(0)

VALUES ==================================== SEQ_NO 1 NO CHILDREN
DESCRIPTION
  fragment_id ............ 0
  parent_frag ............ (none)
  fragment_type .......... master
  tuple_expr ............. %(0)

--- SQL operation complete.
>>

Anoop Sharma <anoop.sharma@esgyn.com> committed b21f2a946dc00ffade5d7f7e7452f3a5cdf6fd24 (11 files)
Reviews: none

removed obsolete code that used libhdfs to write to hive/text files
Traf had code to write to hive/text files using direct libhdfs
interface from C or through java interface. hive/sequence files
were added later and they are written using java interface.
libhdfs code was made obsolete a while back and is no longer used.
That code has now been removed.
Traf now uses common java interface to write to hive/text and
hive/sequence files.

Roberta Marton <rmarton@edev07.esgyn.local> committed 012d65405984358044853529184967b407c55457 (7 files)
Reviews: none

TRAFODION [2641] User who has MANAGE_STATISTICS privilege can't do update statistics on HIVE tables
TRAFODION [2175] a user should only see specific schemas/tables that he has
   privileges to
Updated the following get commands:
  get schemas (in catalog)
  get tables, indexes (in schema)
  get sequences, views (in schema, in catalog)
  get libraries, procedures, functions, table mapping functions

TRAFODION [1573] Additional GET commands for privileges
  get privileges on table
  get privileges on view
New regression test privs1/TEST125

Fixed bug: user granted MANAGE privilege does not have MANAGE sub-privs
Changed REGISTER_HIVE_OBJECT to be treated as a sub-priv under CREATE
Changed UNREGISTER_HIVE_OBJECT to be treaed as a sub-priv under DROP

Anoop Sharma <anoop.sharma@esgyn.com> committed 59926c845be4b9ee30228ed49511c6b69ba97e52 (0 files)
Reviews: none

PR-1120 [TRAFODION-2584] Add support to register hive schemas in traf MD

Anoop Sharma <anoop.sharma@esgyn.com> committed 8e67ca9e49dffc0d1ed1d5e744c9607b1155adcb (16 files)
Reviews: none

ra TRAFODION-2584 Add support to register hive schemas in traf MD
This checkin extends jira TRAFODION-2584 to add support to register
hive schemas(referred to as databases in hive) in trafodion metadata.
Until now, only hive tables and views were registered.
Registering hive schemas would allow certain operations, like
granting/revoking privileges, to be done on a hive schema/database.

Syntax to register/unregister hive schema
  -- register/unregister hive schema hive.sch;

Syntax to show registred hive schemas
  -- get hive registered schemas in catalog trafodion

"showddl schema hive.sch" will now show the ddl for hive database
creation and whether this schema is registered.

showddl and 'get tables in schema' will now return error if that
schema doesn't exist.

regress/hive/TEST007 has been extended with additional tests.

selvaganesang <selva@apache.org> committed cb0a7242eb4bdb8e40575a4caf258156f85c1963 (0 files)
Reviews: none

Merge PR 1108 [TRAFODION-2631] Disk IO counter is not populated for hdfs/hive IOs

Roberta Marton <rmarton@edev07.esgyn.local> committed 63b275d406e29bcaba1ab42e57a656bb8dd8ce31 (6 files)
Reviews: none

TRAFODION - 2632 Performing update statistics on metadata tables causes issues
Added code to return error and cleanup code a bit:
*** ERROR[9205] UPDATE STATISTICS is not supported for object <metadata table>

Also fixed a problem where privileges on Hive tables that included external
table gave false "no priv" error for the SHOWDDL <hive table> command.

selvaganesang <selva.govindarajan@esgyn.com> committed b3594b338b1a4acd3c0e2f8cb0f37482e42a4ef7 (6 files)
Anoop Sharma <anoop.sharma@esgyn.com> committed b109e2cf5883f8e763af853ab6fad7ce7110d9e8 (16 files)
Reviews: none

jira trafodion-2621, trafodion-2619 + others
-- trafodion-2621: add support to create like native hive or hbase tables
-- trafodion-2619: support to_date format 'yyyy/mm/dd'
-- alter table drop column on a table with user indexes would sometime
   fail. That has been fixed.
-- get schemas in a session where default catalog is set to hive,
   now correctly returns hive schemas
-- update stats on hive tables succeeds
-- update stats on hive view returns error as this operation
   is not supported

Liu Ming <ovis_poly@sina.com> committed ced104a79eab9ff886047f8c03b271db3335d958 (0 files)
Reviews: none

Merge Fix for [TRAFODION-2549] update sql reference manual about new hive data type