Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.99.5
    • sqoop2-framework
    • None

    Description

      I saw this test case I partially was responsible for and it does not make any sense to me anymore...

      @Test
        public void testDateTimePrecisionWithCSVTextInObjectArrayOut() {
          Schema schema = new Schema("test");
          schema.addColumn(new DateTime("1"));
          dataFormat.setSchema(schema);
          dataFormat.setTextData("'2014-10-01 12:00:00.000'");
          assertEquals("2014-10-01T12:00:00.000-07:00", dataFormat.getObjectData()[0].toString());
        }
      

      the spec in https://cwiki.apache.org/confluence/display/SQOOP/Sqoop2+CSV+Intermediate+representation does not mention / say that the date time has to be in iso8601 format. but with us using JODA for parsing datetime objects, it becomes imminent that the text representation of date-time has to have iso8601 format, see details here

      https://github.com/JodaOrg/joda-time/issues/11

      FIrst, are we ok with this?

      If so, then the following code replacing space to T in CSVIDF should be removed.

       case DATE_TIME:
            // A datetime string with a space as date-time separator will not be
            // parsed expectedly. The expected separator is "T". See also:
            // https://github.com/JodaOrg/joda-time/issues/11
            String dateTime = removeQuotes(fieldString).replace(" ", "T");
        
      
      

      then the test case will be like this, that makes sense now, because the FROM connector will set data in iso8601 format and it can expect it to be returned bacl in the dateTime format with the timezone that sqoop is prescribing it will support.

      @Test
        public void testDateTimePrecisionWithCSVTextInObjectArrayOut() {
          Schema schema = new Schema("test");
          schema.addColumn(new DateTime("1"));
          dataFormat.setSchema(schema);
          dataFormat.setTextData("'2014-10-01T12:00:00.000'");
          assertEquals("2014-10-01T12:00:00.000-07:00", dataFormat.getObjectData()[0].toString());
        }
      

      Attachments

        1. SQOOP-1846.patch
          4 kB
          Veena Basavaraj
        2. SQOOP-1846-v1.patch
          2 kB
          Veena Basavaraj
        3. SQOOP-1846-v2.patch
          2 kB
          Veena Basavaraj

        Issue Links

          Activity

            People

              vybs Veena Basavaraj
              vybs Veena Basavaraj
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: