[SQOOP-1938] DOC:update the sqoop MR engine implementation details

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.99.5
• Assignee: Veena Basavaraj
• Reporter: Veena Basavaraj
• Created: 22/Dec/14 23:31
• Updated: 20/Jan/15 21:57
• Resolved: 13/Jan/15 21:05

Description

https://cwiki.apache.org/confluence/display/SQOOP/Sqoop+MR+Execution+Engine

1. Why we need SqoopWritable, what can be done in future?
2. Even though we call sqoop as a map only, is that how it always works? what happend when numLoaders is non zero

      // Set number of reducers as number of configured loaders  or suppress
      // reduce phase entirely if loaders are not set at all.
      if(request.getLoaders() != null) {
        job.setNumReduceTasks(request.getLoaders());
      } else {
        job.setNumReduceTasks(0);
      }

3. Internals of SqoopNullOutputFormat and how SqoopWritable is used in it

Issue Links

• is related to: SQOOP-1602 Sqoop2: Fix the current balancing across Loaders internal to Sqoop