Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-12278

[Rust][DataFusion]Use Timestamp(Nanosecond, None) for SQL TIMESTAMP Type

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      1. Rationale
        Running the query `CREATE EXTERNAL TABLE .. (c TIMESTAMP)` today in DataFusion will result in a data type pf "Date64" which means that anything more specific than the date will be ignored.

      This leads to strange behavior such as

      echo "Jorge,2018-12-13T12:12:10.011" >> /tmp/foo.csv
      echo "Andrew,2018-11-13T17:11:10.011" > /tmp/foo.csv
      
      cargo run -p datafusion --bin datafusion-cli
          Finished dev [unoptimized + debuginfo] target(s) in 0.23s
           Running `target/debug/datafusion-cli`
      > CREATE EXTERNAL TABLE t(a varchar, b TIMESTAMP)
      STORED AS CSV
      LOCATION '/tmp/foo.csv';
      
      0 rows in set. Query took 0 seconds.
      > select * from t;
      +--------+------------+
      | a      | b          |
      +--------+------------+
      | Andrew | 2018-11-13 |
      | Jorge  | 2018-12-13 |
      +--------+------------+
      

      (note how it is only a date, not a timestamp)

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            alamb Andrew Lamb
            alamb Andrew Lamb
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1h 20m
                1h 20m

                Issue deployment