Sqoop
  1. Sqoop
  2. SQOOP-536

Blob import doesn't work with avro 1.7.1

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.1-incubating
    • Fix Version/s: 1.4.2
    • Component/s: None
    • Labels:
      None

      Description

      I see the following test failures when updating avro dependency from 1.5.3 to 1.7.1:

      • MySQLLobAvroImportTest
      • OracleLobAvroImportTest

      The reason is because byte[] fails to be mapped to avro bytes type (see below the isBytes() method of GenericData in avro 1.7.1).

      /** Called by the default implementation of {@link #instanceOf}.*/
      protected boolean isBytes(Object datum) {
        return datum instanceof ByteBuffer;
      }
      

      The fix is to wrap byte[] as ByteBuffer so that avro GenericData can map it to bytes type.

      1. SQOOP-536.patch
        0.8 kB
        Cheolsoo Park

        Activity

        Cheolsoo Park created issue -
        Cheolsoo Park made changes -
        Field Original Value New Value
        Description I see the following test failures when updating avro dependency from 1.5.3 to 1.7.1:

        - MySQLLobAvroImportTest
        - OracleLobAvroImportTest

        The reason is because byte[] fails to be mapped to avro bytes type (see below the isBytes() method of GenericData in avro 1.7.1).
        {code}
        /** Called by the default implementation of {@link #instanceOf}.*/
        protected boolean isBytes(Object datum) {
          return datum instanceof ByteBuffer;
        }
        {code}

        The fix is to wrap byte[] as ByteWrapper so that avro GenericData can map it to bytes type.
        I see the following test failures when updating avro dependency from 1.5.3 to 1.7.1:

        - MySQLLobAvroImportTest
        - OracleLobAvroImportTest

        The reason is because byte[] fails to be mapped to avro bytes type (see below the isBytes() method of GenericData in avro 1.7.1).
        {code}
        /** Called by the default implementation of {@link #instanceOf}.*/
        protected boolean isBytes(Object datum) {
          return datum instanceof ByteBuffer;
        }
        {code}

        The fix is to wrap byte[] as ByteBuffer so that avro GenericData can map it to bytes type.
        Cheolsoo Park made changes -
        Attachment SQOOP-536.patch [ 12539003 ]
        Cheolsoo Park made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Jarek Jarcec Cecho made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]

          People

          • Assignee:
            Cheolsoo Park
            Reporter:
            Cheolsoo Park
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development