[ATLAS-2447] Advanced Search DSL: Improve Select Queries Using De-duplication During Query Formation - ASF JIRA

XML

Word

Printable

JSON

In the provided Atlas instance , there are many 1000+ hive_table entities .

Few of them :
db2.table1
db1.table1
db1.table2
db3.table2

DSL query :

typename = hive_table
query = name > "table1"

listed all the hive_table entities whose name>"table1" ,

With limit 25 per page ,
db3.table2 is in 19th page (467th entity)
db1.table2 is in 21st page (521th entity)

Error scenario :
DSL query :

typename = hive_db
query = where name > "table1" select db

limit = 25 , lists only 1 db .

Expected that it would result db3 and db1 along with other db's since "table2" is greater than "table1" and "table2" is in db3 and db1.

when same query with limit =1000 , lists many results along with db1 and db3.

When limit is 25 , "select db" is done on the returned 25 results of "name > table1" and selects the db.

Since table2 doesn't appear in the first 25 entities , db3 and db1 didn't result.

Attaching the screenshots for limit 10000 and limit 25.

links to

Code review