[ATLAS-1822] Search Query Builder and Faceted Search in Atlas

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: atlas-core, atlas-webui
• Assignee: Madhan Neethiraj
• Reporter: Srikanth Venkat
• Created: 19/May/17 17:58
• Updated: 26/Sep/17 11:38

Description

Current search experience in Atlas UI is pretty basic and any advanced searches require knowledge of DSL that might be difficult for business users.

As a data steward or data consumer, I need an easy and intuitive way to perform faceted searches on entities, classifications, relationships, and business glossaries and filter the results based on a combination of parameters so I can navigate and locate assets for curation and in order to take any governance actions. I also need to be able to save my searches and filters and rerun them dynamically against the latest state of the enterprise metadata catalog in Atlas.
Implementation considerations:

1. For business users, there should be default filters provided based on commonly used criteria in the search experience
2. Any search results should only show metadata of data assets that user has permission to view, based upon authorization policies (via Ranger)
3. Search facets need to be based on search conditions and should allow for sequence of AND, OR and NOT operations with multiple search conditions
4. Search facets details (should allow for combinations of any or all of the following):

• Asset type (entity types)
• Attribute value checker (should allow for combinations of range checks, ordinality, exact match, numeric, string and date comparisons, single or muliple values, category checks, nulls)
• Created time
• Updated time
• Size (where relevant for e.g. HDFS files)
• Deleted/Active status
• Owner
• Custom property values
• File format
• Asset name
• Physical storage context: Datalake/cluster or datacenter holding the asset
• Classification
• Glossary/taxonomy association

Note:

1. Searches should be able to be saved and scoped to a particular business catalog or asset type or namespace (when multi-tenancy is supported by Atlas in the future)
2. Once search results are available, user should be able to filter by any of the facets.

Issue Links

• relates to: ATLAS-2165 Limit set to 0 while executing saved query hence no results are fetched.
• relates to: ATLAS-2152 Saved Search UI not able to load searches having OR conditions
• relates to: ATLAS-2164 Unable to execute the saved search query using its GUID
• relates to: ATLAS-2177 offset is not reset when search by type is changed, leads to undesired results.
• relates to: ATLAS-2154 UI save-search : No data message showing even after adding favorite search.
• relates to: ATLAS-2155 Attributes with null values are not filtered using search parameters
• relates to: ATLAS-2175 Infinite refresh scroll appears when a search query is saved.
• relates to: ATLAS-2161 UI : Save Search Query: Search button is not enabled when a saved search has only "Query" without typename / classification
• relates to: ATLAS-2163 Save Search Query : Saved search query having deleted types
• relates to: ATLAS-2156 UI save-search : allow user to edit name

Sub-Tasks

• Basic search improvement in use of Solr index for attribute filtering: Resolved
• Search API: Work with Tag attributes when query and entity filters are specified: Resolved
• basic-search: numeric tag-attribute filtering fails with java.lang.ClassCastException: Resolved
• Regression : Empty results fetched from GET Basic search query: Resolved
• Basic query with entity and tag filters : Providing invalid tag name for classification returns all entities matching typename/query: Resolved
• Regression : Entity,Tag filter fetches empty results when applied on an enum attribute.: Resolved
• Regression: Search fails when filter contains a boolean attribute and a string attribute with special characters: Resolved
• Regression: Basic search performance: Resolved
• Basic search issues due to Titan Solr schema: Resolved
• Basic search - Contains clause issues: Resolved
• Search using entity and trait attributes - "#" in string attribute filter doesn't fetch results: Resolved
• Basic query with entity and tag filters : Invalid filter keys for type and tags attributes are ignored and fetches all entities of the type / associated to the tag: Resolved
• Inconsistency in basic search results due to case sensitivity of type names: Resolved
• Support for Saving Searches: Resolved
• Saved Search: Introduce BASIC and ADVANCE Search Types During Save: Resolved
• UI to support search-history and favorites: Resolved