Uploaded image for project: 'Sqoop (Retired)'
  1. Sqoop (Retired)
  2. SQOOP-1350 Sqoop2: Support all supported data types in the CSV Intermediate Data Format implementation
  3. SQOOP-1349

Sqoop2: Use configurable writable to get Intermediate Data Format

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.99.3
    • 1.99.5
    • sqoop2-framework
    • None

    Description

      The below code performs an unnecessary string copy.

      toDataFormat.setObjectData( matcher.getMatchingData( fromDataFormat.getObjectData() ) );
writable.setString(toDataFormat.getTextData());
context.write(writable, NullWritable.get());

      It would be better to just pass IDFs around.

      Attachments

        1. SQOOP-1349.abe.patch
          8 kB
          Abraham Elmahrek
        2. SQOOP-1349-tested.patch
          16 kB
          Veena Basavaraj

        Issue Links

          depends upon

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-10686 Writables are not always configured

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. SQOOP-777 Sqoop2: Implement intermediate data format representation policy

          • Major - Major loss of function.
          • Resolved
          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. SQOOP-1452 Sqoop2: Upgrade to Hadoop 2.5.1

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link RB

          Activity

            People

              vybs Veena Basavaraj
              abec Abraham Elmahrek
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: