Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-7014

Format plugin for LTSV files

    XMLWordPrintableJSON

    Details

      Description

      I would like to contribute this plugin to Drill.

      Abstract

      storage-plugins-override.conf

      "storage":{
        dfs: {
          type: "file",
          connection: "file:///",
          formats: {
            "ltsv": {
              "type": "ltsv",
              "extensions": [
                "ltsv"
              ]
            }
          },
          enabled: true
        }
      }
      

      sample.ltsv

      time:30/Nov/2016:00:55:08 +0900 host:xxx.xxx.xxx.xxx  forwardedfor:-  req:GET /v1/xxx HTTP/1.1  status:200  size:4968 referer:- ua:Java/1.8.0_131 reqtime:2.532 apptime:2.532 vhost:api.example.com
      time:30/Nov/2016:00:56:37 +0900 host:xxx.xxx.xxx.xxx  forwardedfor:-  req:GET /v1/yyy HTTP/1.1  status:200  size:412  referer:- ua:Java/1.8.0_201 reqtime:3.580 apptime:3.580 vhost:api.example.com
      

      Run query

      root@1805183e9b65:/apache-drill-1.15.0# ./bin/drill-embedded 
      Apache Drill 1.15.0
      "Drill must go on."
      0: jdbc:drill:zk=local> SELECT * FROM dfs.`/apache-drill-1.15.0/sample-data/sample.ltsv` WHERE reqtime > 3.0;
      +-----------------------------+------------------+---------------+-----------------------+---------+-------+----------+-----------------+----------+----------+------------------+
      |            time             |       host       | forwardedfor  |          req          | status  | size  | referer  |       ua        | reqtime  | apptime  |      vhost       |
      +-----------------------------+------------------+---------------+-----------------------+---------+-------+----------+-----------------+----------+----------+------------------+
      | 30/Nov/2016:00:56:37 +0900  | xxx.xxx.xxx.xxx  | -             | GET /v1/yyy HTTP/1.1  | 200     | 412   | -        | Java/1.8.0_201  | 3.580    | 3.580    | api.example.com  |
      +-----------------------------+------------------+---------------+-----------------------+---------+-------+----------+-----------------+----------+----------+------------------+
      1 row selected (6.074 seconds)
      0: jdbc:drill:zk=local> 
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                shimamoto Takako Shimamoto
                Reporter:
                shimamoto Takako Shimamoto
                Reviewer:
                Charles Givre
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: