[CASSANDRA-4813] Problem using BulkOutputFormat while streaming several SSTables simultaneously from a given node.

• Type: Bug
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 1.2.0 beta 3
• Labels: Bulkoutputformat Hadoop SSTables

Environment

I am using SLES 10 SP3, Java 6, 4 Cassandra + Hadoop nodes, 3 Hadoop only nodes (datanodes/tasktrackers), 1 namenode/jobtracker. The machines used are Six-Core AMD Opteron(tm) Processor 8431, 24 cores and 33 GB of RAM. I get the issue on both cassandra 1.1.3, 1.1.5 and I am using Hadoop 0.20.2.

• Severity: Low
• Assignee: Yuki Morishita
• Reporter: Ralph Romanos

Authors

Yuki Morishita

Reviewers

Michael Kjellman

• Created: 16/Oct/12 08:58
• Updated: 16/Apr/19 09:32
• Resolved: 13/Nov/12 22:39

Description

The issue occurs when streaming simultaneously SSTables from the same node to a cassandra cluster using SSTableloader. It seems to me that Cassandra cannot handle receiving simultaneously SSTables from the same node. However, when it receives simultaneously SSTables from two different nodes, everything works fine. As a consequence, when using BulkOutputFormat to generate SSTables and stream them to a cassandra cluster, I cannot use more than one reducer per node otherwise I get a java.io.EOFException in the tasktracker's logs and a java.io.IOException: Broken pipe in the Cassandra logs.

Issue Links

• duplicates: CASSANDRA-4922 TTransportException thrown many times while Hadoop streams in or out from cluster