Sqoop
  1. Sqoop
  2. SQOOP-536

Blob import doesn't work with avro 1.7.1

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.1-incubating
    • Fix Version/s: 1.4.2
    • Component/s: None
    • Labels:
      None

      Description

      I see the following test failures when updating avro dependency from 1.5.3 to 1.7.1:

      • MySQLLobAvroImportTest
      • OracleLobAvroImportTest

      The reason is because byte[] fails to be mapped to avro bytes type (see below the isBytes() method of GenericData in avro 1.7.1).

      /** Called by the default implementation of {@link #instanceOf}.*/
      protected boolean isBytes(Object datum) {
        return datum instanceof ByteBuffer;
      }
      

      The fix is to wrap byte[] as ByteBuffer so that avro GenericData can map it to bytes type.

      1. SQOOP-536.patch
        0.8 kB
        Cheolsoo Park

        Activity

        Hide
        Cheolsoo Park added a comment -

        Attached is a patch that wraps byte[] as ByteBuffer.

        Show
        Cheolsoo Park added a comment - Attached is a patch that wraps byte[] as ByteBuffer.
        Hide
        Jarek Jarcec Cecho added a comment -

        Committed revision 1369193.

        Thank you for your contribution Cheolsoo!

        Jarcec

        Show
        Jarek Jarcec Cecho added a comment - Committed revision 1369193. Thank you for your contribution Cheolsoo! Jarcec
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6 #132 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6/132/)
        SQOOP-536. Blob import doesn't work with avro 1.7.1.

        (Cheolsoo Park via Jarek Jarcec Cecho) (Revision 1369193)

        Result = SUCCESS
        jarcec :
        Files :

        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/AvroImportMapper.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6 #132 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6/132/ ) SQOOP-536 . Blob import doesn't work with avro 1.7.1. (Cheolsoo Park via Jarek Jarcec Cecho) (Revision 1369193) Result = SUCCESS jarcec : Files : /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/AvroImportMapper.java

          People

          • Assignee:
            Cheolsoo Park
            Reporter:
            Cheolsoo Park
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development