[SQOOP-1717] Sqoop2: Remove Data class from docs

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.99.5
• Fix Version/s: 1.99.5
• Component/s: docs
• Assignee: Veena Basavaraj
• Reporter: Abraham Elmahrek
• Created: 12/Nov/14 22:51
• Updated: 19/Nov/14 00:58
• Resolved: 18/Nov/14 23:54

Description

From ConnectorDevelopment.java:

The diagram below describes the initialization phase of a job.
``SqoopInputFormat`` create splits using ``Partitioner``.
::

      ,----------------.          ,-----------.
      |SqoopInputFormat|          |Partitioner|
      `-------+--------'          `-----+-----'
   getSplits  |                         |
  ----------->|                         |
              |      getPartitions      |
              |------------------------>|
              |                         |         ,---------.
              |                         |-------> |Partition|
              |                         |         `----+----'
              |<- - - - - - - - - - - - |              |
              |                         |              |          ,----------.
              |-------------------------------------------------->|SqoopSplit|
              |                         |              |          `----+-----'

The diagram below describes the map phase of a job.
``SqoopMapper`` invokes ``From`` connector's extractor's ``extract`` method.
::

      ,-----------.
      |SqoopMapper|
      `-----+-----'
     run    |
  --------->|                                   ,------------------.
            |---------------------------------->|SqoopMapDataWriter|
            |                                   `------+-----------'
            |                ,---------.               |
            |--------------> |Extractor|               |
            |                `----+----'               |
            |      extract        |                    |
            |-------------------->|                    |
            |                     |                    |
           read from DB           |                    |
  <-------------------------------|      write*        |
            |                     |------------------->|
            |                     |                    |           ,----.
            |                     |                    |---------->|Data|
            |                     |                    |           `-+--'
            |                     |                    |
            |                     |                    |      context.write
            |                     |                    |-------------------------->

The diagram below decribes the reduce phase of a job.
``OutputFormat`` invokes ``To`` connector's loader's ``load`` method (via ``SqoopOutputFormatLoadExecutor`` ).
::

    ,------------.  ,---------------------.
    |SqoopReducer|  |SqoopNullOutputFormat|
    `---+--------'  `----------+----------'
        |                 |   ,-----------------------------.
        |                 |-> |SqoopOutputFormatLoadExecutor|
        |                 |   `--------------+--------------'        ,----.
        |                 |                  |---------------------> |Data|
        |                 |                  |                       `-+--'
        |                 |                  |   ,-----------------.   |
        |                 |                  |-> |SqoopRecordWriter|   |
      getRecordWriter     |                  |   `--------+--------'   |
  ----------------------->| getRecordWriter  |            |            |
        |                 |----------------->|            |            |     ,--------------.
        |                 |                  |-----------------------------> |ConsumerThread|
        |                 |                  |            |            |     `------+-------'
        |                 |<- - - - - - - - -|            |            |            |    ,------.
  <- - - - - - - - - - - -|                  |            |            |            |--->|Loader|
        |                 |                  |            |            |            |    `--+---'
        |                 |                  |            |            |            |       |
        |                 |                  |            |            |            | load  |
   run  |                 |                  |            |            |            |------>|
  ----->|                 |     write        |            |            |            |       |
        |------------------------------------------------>| setContent |            | read* |
        |                 |                  |            |----------->| getContent |<------|
        |                 |                  |            |            |<-----------|       |
        |                 |                  |            |            |            | - - ->|
        |                 |                  |            |            |            |       | write into DB
        |                 |                  |            |            |            |       |-------------->



.. _`Intermediate representation`: https://cwiki.apache.org/confluence/display/SQOOP/Sqoop2+Intermediate+representation

Notice the use of the "Data" entity.

Issue Links

• is cloned by: SQOOP-1753 Sqoop2: 5 minute demo docs have formatting issues
• relates to: SQOOP-1348 Sqoop2: Remove Data class
• links to: RB