Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-5544

Hadoop jobs assigns only one mapper in task

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Fix Version/s: 1.2.6
    • Component/s: None
    • Labels:
      None
    • Environment:

      Red hat linux 5.4, Hadoop 1.0.3, pig 0.11.1

      Description

      We have got very strange beheviour of hadoop cluster after upgrading
      Cassandra from 1.1.5 to Cassandra 1.2.1. We have 5 nodes cluster of Cassandra, where three of them are hodoop slaves. Now when we are submitting job through Pig script, only one map assigns in task running on one of the hadoop slaves regardless of
      volume of data (already tried with more than million rows).
      Configure of pig as follows:
      export PIG_HOME=/oracle/pig-0.10.0
      export PIG_CONF_DIR=${HADOOP_HOME}/conf
      export PIG_INITIAL_ADDRESS=192.168.157.103
      export PIG_RPC_PORT=9160
      export PIG_PARTITIONER=org.apache.cassandra.dht.Murmur3Partitioner

      Also we have these following properties in hadoop:
      <property>
      <name>mapred.tasktracker.map.tasks.maximum</name>
      <value>10</value>
      </property>
      <property>
      <name>mapred.map.tasks</name>
      <value>4</value>
      </property>

        Attachments

        1. 5544-3.txt
          3 kB
          Alex Liu
        2. 5544-2.txt
          4 kB
          Alex Liu
        3. 5544-1.txt
          2 kB
          Alex Liu
        4. 5544.txt
          0.7 kB
          Alex Liu
        5. Screen Shot 2013-05-26 at 4.49.48 PM.png
          82 kB
          Shamim Ahmed

          Issue Links

            Activity

              People

              • Assignee:
                alexliu68 Alex Liu
                Reporter:
                shamim_ru Shamim Ahmed
                Reviewer:
                Brandon Williams
              • Votes:
                4 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: