Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Hide
      TIMESTAMP data type
      New primitive data type that supports traditional UNIX timestamp with optional nanosecond precision.

      Supported conversions:
      Integer numeric types: Interpreted as UNIX timestamp in seconds
      Floating point numeric types: Intepreted as UNIX timestamp in seconds with decimal precision
      Strings: JDBC compliant java.sql.Timestamp format "YYYY-MM-DD HH:MM:SS.fffffffff" (9 decimal place precision)

      Timestamps are interpreted to be timezoneless and stored as an offset from the UNIX epoch. Convenience UDFs for conversion to and from timezones are provided (to_utc_timestamp, from_utc_timestamp).
      All existing datetime UDFs (month, day, year, hour, etc.) will work with the TIMESTAMP data type.

      Supported SerDes:
      ColumnarSerDe
      LazyBinaryColumnarSerDe
      LazySimpleSerDe
      LazyBinarySerDe
      BinarySortableSerDe
      Show
      TIMESTAMP data type New primitive data type that supports traditional UNIX timestamp with optional nanosecond precision. Supported conversions: Integer numeric types: Interpreted as UNIX timestamp in seconds Floating point numeric types: Intepreted as UNIX timestamp in seconds with decimal precision Strings: JDBC compliant java.sql.Timestamp format "YYYY-MM-DD HH:MM:SS.fffffffff" (9 decimal place precision) Timestamps are interpreted to be timezoneless and stored as an offset from the UNIX epoch. Convenience UDFs for conversion to and from timezones are provided (to_utc_timestamp, from_utc_timestamp). All existing datetime UDFs (month, day, year, hour, etc.) will work with the TIMESTAMP data type. Supported SerDes: ColumnarSerDe LazyBinaryColumnarSerDe LazySimpleSerDe LazyBinarySerDe BinarySortableSerDe

      Description

      Add TIMESTAMP type to serde2 that supports unix timestamp (1970-01-01 00:00:01 UTC to 2038-01-19 03:14:07 UTC) with optional nanosecond precision using both LazyBinary and LazySimple SerDes.
      For LazySimpleSerDe, the data is stored in jdbc compliant java.sql.Timestamp parsable strings.

      1. hive-2272.1.patch
        198 kB
        Franklin Hu
      2. hive-2272.10.patch
        233 kB
        Franklin Hu
      3. hive-2272.11.patch
        230 kB
        Franklin Hu
      4. hive-2272.2.patch
        199 kB
        Franklin Hu
      5. hive-2272.3.patch
        210 kB
        Franklin Hu
      6. hive-2272.4.patch
        212 kB
        Franklin Hu
      7. hive-2272.5.patch
        212 kB
        Franklin Hu
      8. hive-2272.6.patch
        211 kB
        Franklin Hu
      9. hive-2272.7.patch
        230 kB
        Franklin Hu
      10. hive-2272.8.patch
        235 kB
        Franklin Hu
      11. hive-2272.9.patch
        236 kB
        Franklin Hu

        Issue Links

          Activity

          Hide
          Franklin Hu added a comment -

          fix fractional timestamp bug

          Show
          Franklin Hu added a comment - fix fractional timestamp bug
          Hide
          Franklin Hu added a comment -

          fix unit test problems

          Show
          Franklin Hu added a comment - fix unit test problems
          Hide
          Siying Dong added a comment -

          Can you add it to review board?

          Show
          Siying Dong added a comment - Can you add it to review board?
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/1135/
          -----------------------------------------------------------

          Review request for hive.

          Summary
          -------

          Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization.
          Supports SQL style jdbc timestamps of the format with nanosecond precision
          YYYY-MM-DD HH:MM:SS[.fff...]

          This addresses bug HIVE-2272.
          https://issues.apache.org/jira/browse/HIVE-2272

          Diffs


          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1146895
          trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION
          trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1146895
          trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION
          trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1146895
          trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1146895
          trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1146895
          trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1146895
          trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1146895
          trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1146895
          trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1146895
          trunk/ql/src/test/results/clientpositive/show_functions.q.out 1146895
          trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION
          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1146895

          Diff: https://reviews.apache.org/r/1135/diff

          Testing
          -------

          Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent

          Thanks,

          Franklin

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1135/ ----------------------------------------------------------- Review request for hive. Summary ------- Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization. Supports SQL style jdbc timestamps of the format with nanosecond precision YYYY-MM-DD HH:MM:SS [.fff...] This addresses bug HIVE-2272 . https://issues.apache.org/jira/browse/HIVE-2272 Diffs trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1146895 trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1146895 trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1146895 trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1146895 trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1146895 trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1146895 trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1146895 trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1146895 trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1146895 trunk/ql/src/test/results/clientpositive/show_functions.q.out 1146895 trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1146895 Diff: https://reviews.apache.org/r/1135/diff Testing ------- Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent Thanks, Franklin
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/1135/
          -----------------------------------------------------------

          (Updated 2011-07-18 19:12:08.745755)

          Review request for hive.

          Summary (updated)
          -------

          Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization.
          Supports SQL style jdbc timestamps of the format with nanosecond precision
          YYYY-MM-DD HH:MM:SS[.fff...]

          This addresses bug HIVE-2272.
          https://issues.apache.org/jira/browse/HIVE-2272

          Diffs


          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1146895
          trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION
          trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1146895
          trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION
          trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1146895
          trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1146895
          trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1146895
          trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1146895
          trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1146895
          trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1146895
          trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1146895
          trunk/ql/src/test/results/clientpositive/show_functions.q.out 1146895
          trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION
          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1146895

          Diff: https://reviews.apache.org/r/1135/diff

          Testing
          -------

          Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent

          Thanks,

          Franklin

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1135/ ----------------------------------------------------------- (Updated 2011-07-18 19:12:08.745755) Review request for hive. Summary (updated) ------- Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization. Supports SQL style jdbc timestamps of the format with nanosecond precision YYYY-MM-DD HH:MM:SS [.fff...] This addresses bug HIVE-2272 . https://issues.apache.org/jira/browse/HIVE-2272 Diffs trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1146895 trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1146895 trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1146895 trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1146895 trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1146895 trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1146895 trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1146895 trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1146895 trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1146895 trunk/ql/src/test/results/clientpositive/show_functions.q.out 1146895 trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1146895 Diff: https://reviews.apache.org/r/1135/diff Testing ------- Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent Thanks, Franklin
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/1135/
          -----------------------------------------------------------

          (Updated 2011-07-18 19:13:31.962359)

          Review request for hive.

          Summary
          -------

          Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization.
          Supports SQL style jdbc timestamps of the format with nanosecond precision
          YYYY-MM-DD HH:MM:SS[.fff...]

          This addresses bug HIVE-2272.
          https://issues.apache.org/jira/browse/HIVE-2272

          Diffs (updated)


          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1146895
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1146895
          trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION
          trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1146895
          trunk/ql/src/test/results/clientpositive/show_functions.q.out 1146895
          trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION
          trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1146895
          trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1146895
          trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1146895
          trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1146895
          trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION
          trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1146895
          trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION
          trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1146895
          trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1146895
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1146895

          Diff: https://reviews.apache.org/r/1135/diff

          Testing
          -------

          Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent

          Thanks,

          Franklin

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1135/ ----------------------------------------------------------- (Updated 2011-07-18 19:13:31.962359) Review request for hive. Summary ------- Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization. Supports SQL style jdbc timestamps of the format with nanosecond precision YYYY-MM-DD HH:MM:SS [.fff...] This addresses bug HIVE-2272 . https://issues.apache.org/jira/browse/HIVE-2272 Diffs (updated) trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1146895 trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1146895 trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1146895 trunk/ql/src/test/results/clientpositive/show_functions.q.out 1146895 trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1146895 trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1146895 trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1146895 trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1146895 trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1146895 trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1146895 trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1146895 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1146895 Diff: https://reviews.apache.org/r/1135/diff Testing ------- Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent Thanks, Franklin
          Hide
          Franklin Hu added a comment -

          remove debug statements

          Show
          Franklin Hu added a comment - remove debug statements
          Hide
          Franklin Hu added a comment -

          rebase

          Show
          Franklin Hu added a comment - rebase
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/1135/
          -----------------------------------------------------------

          (Updated 2011-07-26 21:11:35.218104)

          Review request for hive.

          Changes
          -------

          Rebase

          Summary
          -------

          Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization.
          Supports SQL style jdbc timestamps of the format with nanosecond precision
          YYYY-MM-DD HH:MM:SS[.fff...]

          This addresses bug HIVE-2272.
          https://issues.apache.org/jira/browse/HIVE-2272

          Diffs (updated)


          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1151189
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION
          trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1151189
          trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION
          trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1151189
          trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1151189
          trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1151189
          trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1151189
          trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1151189
          trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1151189
          trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1151189
          trunk/ql/src/test/results/clientpositive/show_functions.q.out 1151189
          trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1151189
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1151189
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1151189
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1151189
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1151189
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1151189
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1151189
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1151189
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1151189
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1151189
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1151189
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1151189
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1151189
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1151189
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION

          Diff: https://reviews.apache.org/r/1135/diff

          Testing
          -------

          Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent

          Thanks,

          Franklin

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1135/ ----------------------------------------------------------- (Updated 2011-07-26 21:11:35.218104) Review request for hive. Changes ------- Rebase Summary ------- Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization. Supports SQL style jdbc timestamps of the format with nanosecond precision YYYY-MM-DD HH:MM:SS [.fff...] This addresses bug HIVE-2272 . https://issues.apache.org/jira/browse/HIVE-2272 Diffs (updated) trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1151189 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1151189 trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1151189 trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1151189 trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1151189 trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1151189 trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1151189 trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1151189 trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1151189 trunk/ql/src/test/results/clientpositive/show_functions.q.out 1151189 trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1151189 trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1151189 trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1151189 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1151189 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1151189 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1151189 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1151189 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1151189 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1151189 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1151189 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1151189 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1151189 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1151189 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1151189 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION Diff: https://reviews.apache.org/r/1135/diff Testing ------- Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent Thanks, Franklin
          Hide
          Franklin Hu added a comment -

          Decided to store timestamps as UTC on disk and add offsets (localize) if user needs through UDF. This simplifies use cases (compared to MySQL) where users from different time zones querying the same data would receive different results.

          Will file a follow up JIRA ticket to add table level timezone defaults. This would allow a user to import existing data that is already offset from UTC and write new data to the table with that offset.

          Summary:

          -Added config var for default timezone
          -Added GenericUDF for converting from UTC to localized timezone (either default, or supplied from list of Java timezones)
          Ex: utc_to_localized_timestamp(<timestamp>) OR utc_to_localized_timestamp(<timestamp>, "America/Chicago")
          -Fixed DateDiff bug I introduced

          Show
          Franklin Hu added a comment - Decided to store timestamps as UTC on disk and add offsets (localize) if user needs through UDF. This simplifies use cases (compared to MySQL) where users from different time zones querying the same data would receive different results. Will file a follow up JIRA ticket to add table level timezone defaults. This would allow a user to import existing data that is already offset from UTC and write new data to the table with that offset. Summary: -Added config var for default timezone -Added GenericUDF for converting from UTC to localized timezone (either default, or supplied from list of Java timezones) Ex: utc_to_localized_timestamp(<timestamp>) OR utc_to_localized_timestamp(<timestamp>, "America/Chicago") -Fixed DateDiff bug I introduced
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/1135/
          -----------------------------------------------------------

          (Updated 2011-07-28 19:17:59.868991)

          Review request for hive.

          Changes
          -------

          Decided to store timestamps as UTC on disk and add offsets (localize) if user needs through UDF. This simplifies use cases (compared to MySQL) where users from different time zones querying the same data would receive different results.

          Will file a follow up JIRA ticket to add table level timezone defaults. This would allow a user to import existing data that is already offset from UTC and write new data to the table with that offset.

          Summary:

          -Added config var for default timezone
          -Added GenericUDF for converting from UTC to localized timezone (either default, or supplied from list of Java timezones)
          Ex: utc_to_localized_timestamp(<timestamp>) OR utc_to_localized_timestamp(<timestamp>, "America/Chicago")
          -Fixed DateDiff bug I introduced

          Summary
          -------

          Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization.
          Supports SQL style jdbc timestamps of the format with nanosecond precision
          YYYY-MM-DD HH:MM:SS[.fff...]

          This addresses bug HIVE-2272.
          https://issues.apache.org/jira/browse/HIVE-2272

          Diffs (updated)


          trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1151571
          trunk/conf/hive-default.xml 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFUtcToLocalizedTimestamp.java PRE-CREATION
          trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1151571
          trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION
          trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1151571
          trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1151571
          trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1151571
          trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1151571
          trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1151571
          trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1151571
          trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1151571
          trunk/ql/src/test/results/clientpositive/show_functions.q.out 1151571
          trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1151571

          Diff: https://reviews.apache.org/r/1135/diff

          Testing
          -------

          Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent

          Thanks,

          Franklin

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1135/ ----------------------------------------------------------- (Updated 2011-07-28 19:17:59.868991) Review request for hive. Changes ------- Decided to store timestamps as UTC on disk and add offsets (localize) if user needs through UDF. This simplifies use cases (compared to MySQL) where users from different time zones querying the same data would receive different results. Will file a follow up JIRA ticket to add table level timezone defaults. This would allow a user to import existing data that is already offset from UTC and write new data to the table with that offset. Summary: -Added config var for default timezone -Added GenericUDF for converting from UTC to localized timezone (either default, or supplied from list of Java timezones) Ex: utc_to_localized_timestamp(<timestamp>) OR utc_to_localized_timestamp(<timestamp>, "America/Chicago") -Fixed DateDiff bug I introduced Summary ------- Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization. Supports SQL style jdbc timestamps of the format with nanosecond precision YYYY-MM-DD HH:MM:SS [.fff...] This addresses bug HIVE-2272 . https://issues.apache.org/jira/browse/HIVE-2272 Diffs (updated) trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1151571 trunk/conf/hive-default.xml 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFUtcToLocalizedTimestamp.java PRE-CREATION trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1151571 trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1151571 trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1151571 trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1151571 trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1151571 trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1151571 trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1151571 trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1151571 trunk/ql/src/test/results/clientpositive/show_functions.q.out 1151571 trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1151571 Diff: https://reviews.apache.org/r/1135/diff Testing ------- Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent Thanks, Franklin
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/1135/
          -----------------------------------------------------------

          (Updated 2011-07-28 20:51:48.675604)

          Review request for hive, Yongqiang He, Ning Zhang, and Siying Dong.

          Summary
          -------

          Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization.
          Supports SQL style jdbc timestamps of the format with nanosecond precision
          YYYY-MM-DD HH:MM:SS[.fff...]

          This addresses bug HIVE-2272.
          https://issues.apache.org/jira/browse/HIVE-2272

          Diffs


          trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1151571
          trunk/conf/hive-default.xml 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1151571
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFUtcToLocalizedTimestamp.java PRE-CREATION
          trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1151571
          trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION
          trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1151571
          trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1151571
          trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1151571
          trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1151571
          trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1151571
          trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1151571
          trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1151571
          trunk/ql/src/test/results/clientpositive/show_functions.q.out 1151571
          trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1151571
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1151571

          Diff: https://reviews.apache.org/r/1135/diff

          Testing
          -------

          Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent

          Thanks,

          Franklin

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1135/ ----------------------------------------------------------- (Updated 2011-07-28 20:51:48.675604) Review request for hive, Yongqiang He, Ning Zhang, and Siying Dong. Summary ------- Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization. Supports SQL style jdbc timestamps of the format with nanosecond precision YYYY-MM-DD HH:MM:SS [.fff...] This addresses bug HIVE-2272 . https://issues.apache.org/jira/browse/HIVE-2272 Diffs trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1151571 trunk/conf/hive-default.xml 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1151571 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFUtcToLocalizedTimestamp.java PRE-CREATION trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1151571 trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1151571 trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1151571 trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1151571 trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1151571 trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1151571 trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1151571 trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1151571 trunk/ql/src/test/results/clientpositive/show_functions.q.out 1151571 trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1151571 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1151571 Diff: https://reviews.apache.org/r/1135/diff Testing ------- Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent Thanks, Franklin
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/1135/
          -----------------------------------------------------------

          (Updated 2011-07-28 21:59:38.146393)

          Review request for hive, Yongqiang He, Ning Zhang, and Siying Dong.

          Changes
          -------

          Add corresponding UDF for converting to UTC from a localized timestamp

          Summary
          -------

          Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization.
          Supports SQL style jdbc timestamps of the format with nanosecond precision
          YYYY-MM-DD HH:MM:SS[.fff...]

          This addresses bug HIVE-2272.
          https://issues.apache.org/jira/browse/HIVE-2272

          Diffs (updated)


          trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1152003
          trunk/conf/hive-default.xml 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1152003
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java PRE-CREATION
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUtcTimestamp.java PRE-CREATION
          trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1152003
          trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION
          trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1152003
          trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1152003
          trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1152003
          trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1152003
          trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1152003
          trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1152003
          trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1152003
          trunk/ql/src/test/results/clientpositive/show_functions.q.out 1152003
          trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1152003
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1152003
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1152003
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1152003
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1152003
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1152003
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1152003
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1152003
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1152003
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1152003
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1152003
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1152003
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1152003
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1152003
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1152003

          Diff: https://reviews.apache.org/r/1135/diff

          Testing
          -------

          Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent

          Thanks,

          Franklin

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1135/ ----------------------------------------------------------- (Updated 2011-07-28 21:59:38.146393) Review request for hive, Yongqiang He, Ning Zhang, and Siying Dong. Changes ------- Add corresponding UDF for converting to UTC from a localized timestamp Summary ------- Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization. Supports SQL style jdbc timestamps of the format with nanosecond precision YYYY-MM-DD HH:MM:SS [.fff...] This addresses bug HIVE-2272 . https://issues.apache.org/jira/browse/HIVE-2272 Diffs (updated) trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1152003 trunk/conf/hive-default.xml 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUtcTimestamp.java PRE-CREATION trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1152003 trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1152003 trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1152003 trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1152003 trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1152003 trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1152003 trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1152003 trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1152003 trunk/ql/src/test/results/clientpositive/show_functions.q.out 1152003 trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1152003 Diff: https://reviews.apache.org/r/1135/diff Testing ------- Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent Thanks, Franklin
          Hide
          Franklin Hu added a comment -

          rename UDFs, add to_utc_timestamp UDF

          Show
          Franklin Hu added a comment - rename UDFs, add to_utc_timestamp UDF
          Hide
          Franklin Hu added a comment -

          fix fractional timestamp precision bug

          Show
          Franklin Hu added a comment - fix fractional timestamp precision bug
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/1135/#review1277
          -----------------------------------------------------------

          trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          <https://reviews.apache.org/r/1135/#comment2918>

          I think we should put default timezone to be UTC and we internally in facebook have PST as default.

          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
          <https://reviews.apache.org/r/1135/#comment2922>

          Only loading it in client side is sufficient?

          • Siying

          On 2011-07-28 21:59:38, Franklin Hu wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/1135/

          -----------------------------------------------------------

          (Updated 2011-07-28 21:59:38)

          Review request for hive, Yongqiang He, Ning Zhang, and Siying Dong.

          Summary

          -------

          Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization.

          Supports SQL style jdbc timestamps of the format with nanosecond precision

          YYYY-MM-DD HH:MM:SS[.fff...]

          This addresses bug HIVE-2272.

          https://issues.apache.org/jira/browse/HIVE-2272

          Diffs

          -----

          trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1152003

          trunk/conf/hive-default.xml 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1152003

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java PRE-CREATION

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUtcTimestamp.java PRE-CREATION

          trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1152003

          trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION

          trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION

          trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION

          trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION

          trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION

          trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1152003

          trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1152003

          trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1152003

          trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1152003

          trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1152003

          trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1152003

          trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1152003

          trunk/ql/src/test/results/clientpositive/show_functions.q.out 1152003

          trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION

          trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION

          trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION

          trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION

          trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1152003

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1152003

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1152003

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1152003

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1152003

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1152003

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1152003

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1152003

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1152003

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1152003

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1152003

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1152003

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1152003

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1152003

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1152003

          Diff: https://reviews.apache.org/r/1135/diff

          Testing

          -------

          Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent

          Thanks,

          Franklin

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1135/#review1277 ----------------------------------------------------------- trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java < https://reviews.apache.org/r/1135/#comment2918 > I think we should put default timezone to be UTC and we internally in facebook have PST as default. trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java < https://reviews.apache.org/r/1135/#comment2922 > Only loading it in client side is sufficient? Siying On 2011-07-28 21:59:38, Franklin Hu wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1135/ ----------------------------------------------------------- (Updated 2011-07-28 21:59:38) Review request for hive, Yongqiang He, Ning Zhang, and Siying Dong. Summary ------- Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization. Supports SQL style jdbc timestamps of the format with nanosecond precision YYYY-MM-DD HH:MM:SS [.fff...] This addresses bug HIVE-2272 . https://issues.apache.org/jira/browse/HIVE-2272 Diffs ----- trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1152003 trunk/conf/hive-default.xml 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1152003 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUtcTimestamp.java PRE-CREATION trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1152003 trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1152003 trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1152003 trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1152003 trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1152003 trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1152003 trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1152003 trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1152003 trunk/ql/src/test/results/clientpositive/show_functions.q.out 1152003 trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1152003 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1152003 Diff: https://reviews.apache.org/r/1135/diff Testing ------- Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent Thanks, Franklin
          Hide
          Franklin Hu added a comment -

          address siying's comments from review board

          Show
          Franklin Hu added a comment - address siying's comments from review board
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/1135/
          -----------------------------------------------------------

          (Updated 2011-08-08 18:19:27.210647)

          Review request for hive, Yongqiang He, Ning Zhang, and Siying Dong.

          Changes
          -------

          Addressed Siying's comments.
          Removed references to a "default" timezone. All times are treated as UTC.

          Summary
          -------

          Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization.
          Supports SQL style jdbc timestamps of the format with nanosecond precision
          YYYY-MM-DD HH:MM:SS[.fff...]

          This addresses bug HIVE-2272.
          https://issues.apache.org/jira/browse/HIVE-2272

          Diffs (updated)


          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1155026
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java PRE-CREATION
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUtcTimestamp.java PRE-CREATION
          trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1155026
          trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION
          trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1155026
          trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1155026
          trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1155026
          trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1155026
          trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1155026
          trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1155026
          trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1155026
          trunk/ql/src/test/results/clientpositive/show_functions.q.out 1155026
          trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1155026
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1155026
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1155026
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1155026
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1155026
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1155026
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1155026
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1155026
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1155026
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1155026
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1155026
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1155026
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1155026
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1155026
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1155026

          Diff: https://reviews.apache.org/r/1135/diff

          Testing
          -------

          Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent

          Thanks,

          Franklin

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1135/ ----------------------------------------------------------- (Updated 2011-08-08 18:19:27.210647) Review request for hive, Yongqiang He, Ning Zhang, and Siying Dong. Changes ------- Addressed Siying's comments. Removed references to a "default" timezone. All times are treated as UTC. Summary ------- Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization. Supports SQL style jdbc timestamps of the format with nanosecond precision YYYY-MM-DD HH:MM:SS [.fff...] This addresses bug HIVE-2272 . https://issues.apache.org/jira/browse/HIVE-2272 Diffs (updated) trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUtcTimestamp.java PRE-CREATION trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1155026 trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1155026 trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1155026 trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1155026 trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1155026 trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1155026 trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1155026 trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1155026 trunk/ql/src/test/results/clientpositive/show_functions.q.out 1155026 trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1155026 Diff: https://reviews.apache.org/r/1135/diff Testing ------- Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent Thanks, Franklin
          Hide
          Siying Dong added a comment -

          +1, please open a follow up JIRA for setting timezones.

          Show
          Siying Dong added a comment - +1, please open a follow up JIRA for setting timezones.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/1135/#review1391
          -----------------------------------------------------------

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java
          <https://reviews.apache.org/r/1135/#comment3175>

          Parameterize the WritableComparable there to eliminate warning.

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java
          <https://reviews.apache.org/r/1135/#comment3172>

          Do we really need two booleans? It seems at a time data will be either in byte[] or in timestamp, so one bool will suffice?

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java
          <https://reviews.apache.org/r/1135/#comment3174>

          Arrays.fill() might be better here.

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java
          <https://reviews.apache.org/r/1135/#comment3173>

          I think you want Arrays.equals() there.

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java
          <https://reviews.apache.org/r/1135/#comment3177>

          System.arrayCopy() will be more efficient here.

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java
          <https://reviews.apache.org/r/1135/#comment3178>

          System.arrayCopy() will be more efficient here.

          • Ashutosh

          On 2011-08-08 18:19:27, Franklin Hu wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/1135/

          -----------------------------------------------------------

          (Updated 2011-08-08 18:19:27)

          Review request for hive, Yongqiang He, Ning Zhang, and Siying Dong.

          Summary

          -------

          Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization.

          Supports SQL style jdbc timestamps of the format with nanosecond precision

          YYYY-MM-DD HH:MM:SS[.fff...]

          This addresses bug HIVE-2272.

          https://issues.apache.org/jira/browse/HIVE-2272

          Diffs

          -----

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java PRE-CREATION

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUtcTimestamp.java PRE-CREATION

          trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1155026

          trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION

          trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION

          trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION

          trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION

          trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION

          trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1155026

          trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1155026

          trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1155026

          trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1155026

          trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1155026

          trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1155026

          trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1155026

          trunk/ql/src/test/results/clientpositive/show_functions.q.out 1155026

          trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION

          trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION

          trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION

          trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION

          trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1155026

          Diff: https://reviews.apache.org/r/1135/diff

          Testing

          -------

          Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent

          Thanks,

          Franklin

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1135/#review1391 ----------------------------------------------------------- trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java < https://reviews.apache.org/r/1135/#comment3175 > Parameterize the WritableComparable there to eliminate warning. trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java < https://reviews.apache.org/r/1135/#comment3172 > Do we really need two booleans? It seems at a time data will be either in byte[] or in timestamp, so one bool will suffice? trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java < https://reviews.apache.org/r/1135/#comment3174 > Arrays.fill() might be better here. trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java < https://reviews.apache.org/r/1135/#comment3173 > I think you want Arrays.equals() there. trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java < https://reviews.apache.org/r/1135/#comment3177 > System.arrayCopy() will be more efficient here. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java < https://reviews.apache.org/r/1135/#comment3178 > System.arrayCopy() will be more efficient here. Ashutosh On 2011-08-08 18:19:27, Franklin Hu wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1135/ ----------------------------------------------------------- (Updated 2011-08-08 18:19:27) Review request for hive, Yongqiang He, Ning Zhang, and Siying Dong. Summary ------- Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization. Supports SQL style jdbc timestamps of the format with nanosecond precision YYYY-MM-DD HH:MM:SS [.fff...] This addresses bug HIVE-2272 . https://issues.apache.org/jira/browse/HIVE-2272 Diffs ----- trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUtcTimestamp.java PRE-CREATION trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1155026 trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1155026 trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1155026 trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1155026 trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1155026 trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1155026 trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1155026 trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1155026 trunk/ql/src/test/results/clientpositive/show_functions.q.out 1155026 trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1155026 Diff: https://reviews.apache.org/r/1135/diff Testing ------- Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent Thanks, Franklin
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-08-10 21:30:23, Ashutosh Chauhan wrote:

          > trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java, line 134

          > <https://reviews.apache.org/r/1135/diff/6/?file=31093#file31093line134>

          >

          > I think you want Arrays.equals() there.

          currentBytes always points to the correct byte[], if any, so in this case I want to compare the pointers to check whether we are using externalBytes to hold the data.

          On 2011-08-10 21:30:23, Ashutosh Chauhan wrote:

          > trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java, lines 73-74

          > <https://reviews.apache.org/r/1135/diff/6/?file=31093#file31093line73>

          >

          > Do we really need two booleans? It seems at a time data will be either in byte[] or in timestamp, so one bool will suffice?

          We may have conditions when the data is in both the byte[] and timestamp (ex: needed to instantiate timestamp object from byte[] to get the string representation) so with two booleans we can make sure we don't do repeated work if there are repeated calls.

          • Franklin

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/1135/#review1391
          -----------------------------------------------------------

          On 2011-08-08 18:19:27, Franklin Hu wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/1135/

          -----------------------------------------------------------

          (Updated 2011-08-08 18:19:27)

          Review request for hive, Yongqiang He, Ning Zhang, and Siying Dong.

          Summary

          -------

          Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization.

          Supports SQL style jdbc timestamps of the format with nanosecond precision

          YYYY-MM-DD HH:MM:SS[.fff...]

          This addresses bug HIVE-2272.

          https://issues.apache.org/jira/browse/HIVE-2272

          Diffs

          -----

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1155026

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java PRE-CREATION

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION

          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUtcTimestamp.java PRE-CREATION

          trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1155026

          trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION

          trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION

          trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION

          trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION

          trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION

          trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1155026

          trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1155026

          trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1155026

          trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1155026

          trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1155026

          trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1155026

          trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1155026

          trunk/ql/src/test/results/clientpositive/show_functions.q.out 1155026

          trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION

          trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION

          trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION

          trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION

          trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1155026

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION

          trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1155026

          Diff: https://reviews.apache.org/r/1135/diff

          Testing

          -------

          Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent

          Thanks,

          Franklin

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-08-10 21:30:23, Ashutosh Chauhan wrote: > trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java, line 134 > < https://reviews.apache.org/r/1135/diff/6/?file=31093#file31093line134 > > > I think you want Arrays.equals() there. currentBytes always points to the correct byte[], if any, so in this case I want to compare the pointers to check whether we are using externalBytes to hold the data. On 2011-08-10 21:30:23, Ashutosh Chauhan wrote: > trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java, lines 73-74 > < https://reviews.apache.org/r/1135/diff/6/?file=31093#file31093line73 > > > Do we really need two booleans? It seems at a time data will be either in byte[] or in timestamp, so one bool will suffice? We may have conditions when the data is in both the byte[] and timestamp (ex: needed to instantiate timestamp object from byte[] to get the string representation) so with two booleans we can make sure we don't do repeated work if there are repeated calls. Franklin ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1135/#review1391 ----------------------------------------------------------- On 2011-08-08 18:19:27, Franklin Hu wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1135/ ----------------------------------------------------------- (Updated 2011-08-08 18:19:27) Review request for hive, Yongqiang He, Ning Zhang, and Siying Dong. Summary ------- Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization. Supports SQL style jdbc timestamps of the format with nanosecond precision YYYY-MM-DD HH:MM:SS [.fff...] This addresses bug HIVE-2272 . https://issues.apache.org/jira/browse/HIVE-2272 Diffs ----- trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1155026 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUtcTimestamp.java PRE-CREATION trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1155026 trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1155026 trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1155026 trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1155026 trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1155026 trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1155026 trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1155026 trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1155026 trunk/ql/src/test/results/clientpositive/show_functions.q.out 1155026 trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1155026 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1155026 Diff: https://reviews.apache.org/r/1135/diff Testing ------- Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent Thanks, Franklin
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/1135/
          -----------------------------------------------------------

          (Updated 2011-08-10 23:53:23.261081)

          Review request for hive, Yongqiang He, Ning Zhang, and Siying Dong.

          Changes
          -------

          -Fix q.out files for timestamp_udf.q and show_functions.q
          -Incorporate Ashutosh's comments

          Summary
          -------

          Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization.
          Supports SQL style jdbc timestamps of the format with nanosecond precision
          YYYY-MM-DD HH:MM:SS[.fff...]

          This addresses bug HIVE-2272.
          https://issues.apache.org/jira/browse/HIVE-2272

          Diffs (updated)


          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1156396
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java PRE-CREATION
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION
          trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUtcTimestamp.java PRE-CREATION
          trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1156396
          trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION
          trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1156396
          trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1156396
          trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1156396
          trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1156396
          trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1156396
          trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1156396
          trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1156396
          trunk/ql/src/test/results/clientpositive/show_functions.q.out 1156396
          trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1156396
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1156396
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1156396
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1156396
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1156396
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1156396
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1156396
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1156396
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1156396
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1156396
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1156396
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1156396
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1156396
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1156396
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1156396

          Diff: https://reviews.apache.org/r/1135/diff

          Testing
          -------

          Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent

          Thanks,

          Franklin

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1135/ ----------------------------------------------------------- (Updated 2011-08-10 23:53:23.261081) Review request for hive, Yongqiang He, Ning Zhang, and Siying Dong. Changes ------- -Fix q.out files for timestamp_udf.q and show_functions.q -Incorporate Ashutosh's comments Summary ------- Adds TIMESTAMP type to serde2 with both string (LazySimple) and binary (LazyBinary) serialization. Supports SQL style jdbc timestamps of the format with nanosecond precision YYYY-MM-DD HH:MM:SS [.fff...] This addresses bug HIVE-2272 . https://issues.apache.org/jira/browse/HIVE-2272 Diffs (updated) trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java 1156396 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUtcTimestamp.java PRE-CREATION trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q 1156396 trunk/ql/src/test/queries/clientpositive/timestamp_1.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_2.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/timestamp_udf.q PRE-CREATION trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out 1156396 trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out 1156396 trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out 1156396 trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out 1156396 trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out 1156396 trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out 1156396 trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out 1156396 trunk/ql/src/test/results/clientpositive/show_functions.q.out 1156396 trunk/ql/src/test/results/clientpositive/timestamp_1.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_2.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_3.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1156396 trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1156396 trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1156396 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1156396 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1156396 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1156396 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1156396 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1156396 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1156396 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1156396 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1156396 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1156396 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1156396 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1156396 trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1156396 Diff: https://reviews.apache.org/r/1135/diff Testing ------- Wrote some unit tests for string/binary serialization. Also ran against test data and looked consistent Thanks, Franklin
          Hide
          Franklin Hu added a comment -

          Fix .out files, incorporate comments from review board

          Show
          Franklin Hu added a comment - Fix .out files, incorporate comments from review board
          Hide
          Siying Dong added a comment -

          Thanks Franklin!

          Show
          Siying Dong added a comment - Thanks Franklin!
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #892 (See https://builds.apache.org/job/Hive-trunk-h0.21/892/)
          HIVE-2272. add TIMESTAMP data type (Franklin Hu via Siying Dong)

          sdong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1157222
          Files :

          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java
          • /hive/trunk/ql/src/test/results/clientpositive/timestamp_1.q.out
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java
          • /hive/trunk/ql/src/test/results/clientpositive/timestamp_3.q.out
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java
          • /hive/trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java
          • /hive/trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java
          • /hive/trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUtcTimestamp.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java
          • /hive/trunk/ql/src/test/queries/clientpositive/timestamp_1.q
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java
          • /hive/trunk/ql/src/test/queries/clientpositive/timestamp_3.q
          • /hive/trunk/ql/src/test/queries/clientpositive/timestamp_udf.q
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java
          • /hive/trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java
          • /hive/trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java
          • /hive/trunk/ql/src/test/results/clientpositive/timestamp_2.q.out
          • /hive/trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java
          • /hive/trunk/ql/src/test/results/clientpositive/show_functions.q.out
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java
          • /hive/trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java
          • /hive/trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java
          • /hive/trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java
          • /hive/trunk/ql/src/test/results/clientnegative/invalid_t_create3.q.out
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java
          • /hive/trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java
          • /hive/trunk/ql/src/test/queries/clientpositive/timestamp_2.q
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java
          • /hive/trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #892 (See https://builds.apache.org/job/Hive-trunk-h0.21/892/ ) HIVE-2272 . add TIMESTAMP data type (Franklin Hu via Siying Dong) sdong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1157222 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileApprox.java /hive/trunk/ql/src/test/results/clientpositive/timestamp_1.q.out /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java /hive/trunk/ql/src/test/results/clientpositive/timestamp_3.q.out /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVariance.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java /hive/trunk/ql/src/test/results/clientpositive/timestamp_udf.q.out /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java /hive/trunk/ql/src/test/queries/clientnegative/invalid_t_create3.q /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryTimestamp.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFYear.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java /hive/trunk/ql/src/test/results/clientnegative/invalid_t_create2.q.out /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateSub.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStdSample.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStd.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableTimestampObjectInspector.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCorrelation.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFWeekOfYear.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUtcTimestamp.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java /hive/trunk/ql/src/test/queries/clientpositive/timestamp_1.q /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableTimestampObjectInspector.java /hive/trunk/ql/src/test/queries/clientpositive/timestamp_3.q /hive/trunk/ql/src/test/queries/clientpositive/timestamp_udf.q /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java /hive/trunk/ql/src/test/results/clientpositive/timestamp_comparison.q.out /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java /hive/trunk/ql/src/test/results/clientnegative/invalid_t_alter2.q.out /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateDiff.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java /hive/trunk/ql/src/test/results/clientpositive/timestamp_2.q.out /hive/trunk/ql/src/test/results/clientnegative/invalid_create_tbl1.q.out /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDate.java /hive/trunk/ql/src/test/results/clientpositive/show_functions.q.out /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMonth.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java /hive/trunk/ql/src/test/queries/clientpositive/timestamp_comparison.q /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java /hive/trunk/ql/src/test/results/clientnegative/invalid_t_transform.q.out /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovarianceSample.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java /hive/trunk/ql/src/test/results/clientnegative/invalid_t_create1.q.out /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java /hive/trunk/ql/src/test/results/clientnegative/invalid_t_create3.q.out /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDateAdd.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCovariance.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFMinute.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFDayOfMonth.java /hive/trunk/ql/src/test/results/clientnegative/wrong_column_type.q.out /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFHistogramNumeric.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaTimestampObjectInspector.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/TimestampObjectInspector.java /hive/trunk/ql/src/test/queries/clientpositive/timestamp_2.q /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFVarianceSample.java /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyTimestampObjectInspector.java /hive/trunk/ql/src/test/results/clientnegative/invalid_t_alter1.q.out /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSecond.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java

            People

            • Assignee:
              Franklin Hu
              Reporter:
              Franklin Hu
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development