Cassandra
  1. Cassandra
  2. CASSANDRA-4421

Support cql3 table definitions in Hadoop InputFormat

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Fix Version/s: 1.2.6
    • Component/s: API
    • Labels:
    • Environment:

      Debian Squeeze

      Description

      Hello,

      i faced a bug while writing composite column values and following validation on server side.

      This is the setup for reproduction:

      1. create a keyspace

      create keyspace test with strategy_class = 'SimpleStrategy' and strategy_options:replication_factor = 1;

      2. create a cf via cql (3.0)

      create table test1 (
      a int,
      b int,
      c int,
      primary key (a, b)
      );

      If i have a look at the schema in cli i noticed that there is no column metadata for columns not part of primary key.

      create column family test1
      with column_type = 'Standard'
      and comparator = 'CompositeType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type)'
      and default_validation_class = 'UTF8Type'
      and key_validation_class = 'Int32Type'
      and read_repair_chance = 0.1
      and dclocal_read_repair_chance = 0.0
      and gc_grace = 864000
      and min_compaction_threshold = 4
      and max_compaction_threshold = 32
      and replicate_on_write = true
      and compaction_strategy = 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
      and caching = 'KEYS_ONLY'
      and compression_options =

      {'sstable_compression' : 'org.apache.cassandra.io.compress.SnappyCompressor'}

      ;

      Please notice the default validation class: UTF8Type

      Now i would like to insert value > 127 via cassandra client (no cql, part of mr-jobs). Have a look at the attachement.

      Batch mutate fails:

      InvalidRequestException(why:(String didn't validate.) [test][test1][1:c] failed validation)

      A validator for column value is fetched in ThriftValidation::validateColumnData which returns always the default validator which is UTF8Type as described above (The ColumnDefinition for given column name "c" is always null)

      In UTF8Type there is a check for

      if (b > 127)
      return false;

      Anyway, maybe i'm doing something wrong, but i used cql 3.0 for table creation. I assigned data types to all columns, but i can not set values for a composite column because the default validation class is used.

      I think the schema should know the correct validator even for composite columns. The usage of the default validation class does not make sense.

      Best Regards

      Bert Passek

      1. 4421-10-patch-for-1.2.txt
        171 kB
        Alex Liu
      2. 4421-10-patch-for-trunk.txt
        169 kB
        Alex Liu
      3. 4421-9-je.txt
        169 kB
        Alex Liu
      4. 4421-8-je.txt
        167 kB
        Alex Liu
      5. 4421-8-cb.txt
        166 kB
        Colin B.
      6. 4421-7-je.txt
        20 kB
        Alex Liu
      7. 4421-6-je.txt
        20 kB
        Alex Liu
      8. 4421-6.cb.txt
        169 kB
        Colin B.
      9. 4421-5.txt
        169 kB
        Alex Liu
      10. 4421-4.txt
        169 kB
        Alex Liu
      11. 4421-3.txt
        168 kB
        Alex Liu
      12. 4421-2.txt
        167 kB
        Alex Liu
      13. 4421-1.txt
        167 kB
        Alex Liu
      14. 4421.txt
        162 kB
        Alex Liu

        Activity

          People

          • Assignee:
            Alex Liu
            Reporter:
            bert Passek
            Reviewer:
            Jonathan Ellis
            Tester:
            Ryan McGuire
          • Votes:
            15 Vote for this issue
            Watchers:
            18 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development