[HAWQ-459] Enhance Metadata Rest API to support table(s)

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0.0-incubating
• Component/s: Hcatalog, PXF
• Assignee: Shivram Mani
• Reporter: Shivram Mani
• Created: 25/Feb/16 21:40
• Updated: 14/Jul/16 17:46
• Resolved: 22/Mar/16 18:29

Description

Enhance the metadata endpoint using /Metdata/getMetadata to support retrieving metadata for generic datasources, i.e. Hive, HBase, HDFS, etc and support retrieving metadata for multiple objects based on wildcard pattern

Method supported

name
GET

Path mapped

pxf/<protocolVersion>/Metadata/getMetadata

Input parameters:

Name	Description	Is mandatory?	Could be a wildcard?	Valid input values
profile	Type of source to fetch data from, should support Hive.	Yes	No	hive
pattern	File or table name or pattern. Patterns are supported using wildcard *.	Yes	Yes	db.table or file path

In case of Hive, patterns corresponds to schema.table
In case of HBase, pattern=schema.table
In case of HDFS, pattern=absolute path to file

HTTP return codes:

Code	Meaning
200	found some result set
400	not all required parameters passed
404	didn't find any results for given parameters, empty result set
405	called method other than GET
500	unable to access underlying metastore(hive, hbase, hdfs etc), datasource not supported, some other exceptions

Return data format for 200 code:

{  
   "PXFMetadata":[  
      {  
         "item":{  
            "path":"default",
            "name":"abc"
         },
         "fields":[  
            {  
               "name":"a",
               "type":"int"
            },
            {  
               "name":"b",
               "type":"float"
            }
         ]
      },
      {  
         "item":{  
            "path":"default",
            "name":"abcdef"
         },
         "fields":[  
            {  
               "name":"a",
               "type":"int"
            },
            {  
               "name":"b",
               "type":"float"
            }
         ]
      },
      {  
         "item":{  
            "path":"default",
            "name":"abcdefghk"
         },
         "fields":[  
            {  
               "name":"a",
               "type":"int"
            },
            {  
               "name":"b",
               "type":"float"
            }
         ]
      }
   ]
}

Return data format for 404 code:

{  
   "PXFMetadata":[]
}

Return data format for 500 code:

{  
   "PXFError":{"code" : "XXX1", "message" : "Unable to reach Hive metastore"}
}

Examples

Request:

curl -i "http://localhost:51200/pxf/v14/Metadata/getMetadata?profile=hive&pattern=default.abc*" 

Response:

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: application/json
Transfer-Encoding: chunked
Date: Fri, 26 Feb 2016 03:33:53 GMT

{  
   "PXFMetadata":[  
      {  
         "item":{ 
            "path":"default",
            "name":"abc"
         },
         "fields":[  
            {  
               "name":"a",
               "type":"int"
            },
            {  
               "name":"b",
               "type":"float"
            }
         ]
      },
      {  
         "item":{  
            "path":"default",
            "name":"abcdef"
         },
         "fields":[  
            {  
               "name":"a",
               "type":"int"
            },
            {  
               "name":"b",
               "type":"float"
            }
         ]
      },
      {  
         "item":{  
            "path":"default",
            "item":"abcdefghk"
         },
         "fields":[  
            {  
               "name":"a",
               "type":"int"
            },
            {  
               "name":"b",
               "type":"float"
            }
         ]
      }
   ]
}

Issue Links

• is related to: HAWQ-491 External Database