Details
-
Bug
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
v0.3.0
-
None
-
None
Description
From Capital One case
////////
It looks very interesting that Postgres has this behavior. Is Postgres your production setting for holding hive metadata?
If that is the case, I think we need extend Eagle a little bit to support Postgres. (Hive metadata tables use upper case for both table name and column name)
Thanks
Edward
On Mon, Apr 11, 2016 at 9:02 PM, Carnali, Matthew R. <Matthew.Carnali@capitalone.com> wrote:
Here are the correct queries:
1.
select "NAME" from "DBS”;
2.
select "t"."TBL_NAME" from "TBLS" t, "DBS" d where "t"."DB_ID"="d"."DB_ID" and "d"."NAME"='%s’;
3.
select "c"."COLUMN_NAME" from "DBS" d join "TBLS" t on "d"."DB_ID"="t"."DB_ID" join "SDS" s on "t"."SD_ID"="s"."SD_ID" join "COLUMNS_V2" c on "s"."CD_ID"="c"."CD_ID" where "d"."NAME"='%s' and "t"."TBL_NAME"='%s';
From: "Carnali, Matthew R." <Matthew.Carnali@capitalone.com>
Date: Monday, April 11, 2016 at 8:11 PM
To: Edward Zhang <yonzhang2012@gmail.com>
Cc: "qingwzhao@ebay.com" <qingwzhao@ebay.com>, Hao Chen <hao.chen3@ebay.com>, "liasu@ebay.com" <liasu@ebay.com>
Subject: Re: Fail to get WEB_CONFIG configurations for data classification
So what I found is the following:
http://stackoverflow.com/questions/6331504/omitting-the-double-quote-to-do-query-on-postgresql
I modified the queries to have quotes like so (but it still does not seem like it is working correctly, I am investigating further):
1. select ’NAME' from "DBS";
2. select 't.TBL_NAME' from "TBLS" t, "DBS" d where 't.db_id'='d.db_id' and 'd.name'='%s’;
3. select 'c.column_name' from "DBS" d join "TBLS" t on 'd.db_id'='t.db_id' join "SDS" s on 't.sd_id'='s.sd_id' join "COLUMNS_V2" c on 's.cd_id'='c.cd_id' where 'd.name'='%s' and 't.tbl_name'='%s’;
From: Edward Zhang <yonzhang2012@gmail.com>
Date: Monday, April 11, 2016 at 3:52 PM
To: "Carnali, Matthew R." <Matthew.Carnali@capitalone.com>
Cc: "qingwzhao@ebay.com" <qingwzhao@ebay.com>, Hao Chen <hao.chen3@ebay.com>, "liasu@ebay.com" <liasu@ebay.com>
Subject: Re: Fail to get WEB_CONFIG configurations for data classification
We use the following queries, let me know if any of queries can't be run directly against postgres?
1. to get databases
String sql = "select name from DBS";
2. to get tables
String sql_format = "select t.tbl_name from TBLS t, DBS d where t.db_id=d.db_id and d.name='%s'";
3. to get columns
String sql_format =
"select c.column_name " +
"from DBS d join TBLS t on d.db_id=t.db_id " +
"join SDS s on t.sd_id=s.sd_id " +
"join COLUMNS_V2 c on s.cd_id=c.cd_id " +
"where d.name='%s' and t.tbl_name='%s';";