Uploaded image for project: 'Sqoop (Retired)'
  1. Sqoop (Retired)
  2. SQOOP-1350

Sqoop2: Support all supported data types in the CSV Intermediate Data Format implementation

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.99.3
    • 1.99.5
    • sqoop2-framework
    • None

    Description

      http://issues.apache.org/jira/secure/attachment/12589331/Sqoop2Datatypes.pdf

      Link to the original IDF proposal on the column types to support the IDF Schema.

      The updated design doc is in

      https://cwiki.apache.org/confluence/display/SQOOP/Intermediate+Data+Format+API

      Attachments

        1. SQOOP-1709.patch
          29 kB
          Veena Basavaraj

        Issue Links

          1.
          Sqoop2: rename Type to Column Type + size to length + minor doc fixes Sub-task Resolved Veena Basavaraj
          2.
          Sqoop2: Allow null as a dummy Schema Sub-task Resolved Veena Basavaraj
          3.
          Sqoop2: Remove Data class Sub-task Resolved Veena Basavaraj
          4.
          Sqoop2: Use configurable writable to get Intermediate Data Format Sub-task Resolved Veena Basavaraj
          5.
          Add IDF API doc/wiki for the IDF interface and Schema -> ColumnTypes Sub-task Resolved Veena Basavaraj
          6.
          SQOOP2: Address the validate method in Column class Sub-task Resolved Veena Basavaraj
          7.
          Rename Unsupported Column type to Unknown and add java doc Sub-task Resolved Veena Basavaraj
          8.
          Column Type enhancements for complex types Sub-task Resolved Veena Basavaraj
          9.
          Make name for column required ( fix the corr tests) Sub-task Resolved Veena Basavaraj
          10.
          Add Options as a field in the Enum object ( so it can be used for validation) Sub-task Resolved Veena Basavaraj
          11.
          Support List Type in CSV IDF Sub-task Resolved Veena Basavaraj
          12.
          Support Map Type in CSV IDF Sub-task Resolved Veena Basavaraj
          13.
          Sqoop2: Remove Data class from docs Sub-task Resolved Veena Basavaraj
          14.
          Sqoop2: Unit tests for different Column sub classes Array/Set and Map types Sub-task Resolved Veena Basavaraj
          15.
          Sqoop2: Time/Timestamp format support for CSV IDF Sub-task Resolved Veena Basavaraj
          16.
          Fix Enum to no inherit from list Sub-task Resolved Veena Basavaraj
          17.
          Investigation CSV IDF FORMAT of the Array/NestedArray/ Set/ Map in Postgres and HIVE. Sub-task Resolved Veena Basavaraj
          18.
          Sqoop2: IDF API changes Sub-task Resolved Veena Basavaraj
          19.
          Sqoop2: Add SqoopIDFUtils class and unit tests Sub-task Resolved Veena Basavaraj
          20.
          Sqoop2: Date and DateTime is not encoded in Single Quotes Sub-task Resolved Veena Basavaraj
          21.
          Support Enum in CSVIDF ( + add unit tests) Sub-task Resolved Veena Basavaraj
          22.
          Sqoop2: Handle NULLs for all types in CSV Intermediate Data Format Sub-task Resolved Veena Basavaraj
          23.
          Sqoop2: Update CSVIntermediate BIT data type Sub-task Resolved Veena Basavaraj
          24.
          Sqoop2: Define IDF object model Sub-task Resolved Veena Basavaraj
          25.
          Sqoop2: DateTime support in CSV IDF and iso8601 Sub-task Resolved Veena Basavaraj
          26.
          Sqoop2: Make DateTime Column type support datetime with and without timezone Sub-task Resolved Veena Basavaraj
          27.
          Using JODA for datetime means we only have 3 digit millisecond representation for fraction Sub-task Resolved Veena Basavaraj

          Activity

            People

              vybs Veena Basavaraj
              abec Abraham Elmahrek
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: