[CASSANDRA-8934] COPY command has inherent 128KB field size limit

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 2.0.15, 2.1.5
• Component/s: Legacy/Tools
• Labels: cqlsh
• Severity: Normal
• Impacts: Docs
• Assignee: Philip Thompson
• Reporter: Brian Hess

Authors

Philip Thompson

Reviewers

Tom Hobbs

• Created: 09/Mar/15 18:44
• Updated: 06/Nov/19 22:05
• Resolved: 24/Mar/15 16:45

Description

In using the COPY command as follows:
cqlsh -e "COPY test.test1mb(pkey, ccol, data) FROM 'in/data1MB/data1MB_9.csv'"
the following error is thrown:
<stdin>:1:field larger than field limit (131072)

The data file contains a field that is greater than 128KB (it's more like almost 1MB).

A work-around (thanks to Jeremiah Jordan and Tom Hobbs is to modify the cqlsh script and add the line
csv.field_size_limit(1000000000)
anywhere after the line
import csv