Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-4421

Support cql3 table definitions in Hadoop InputFormat

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Normal
    • Resolution: Fixed
    • 1.2.6
    • Legacy/CQL
    • Debian Squeeze

    Description

      Hello,

      i faced a bug while writing composite column values and following validation on server side.

      This is the setup for reproduction:

      1. create a keyspace

      create keyspace test with strategy_class = 'SimpleStrategy' and strategy_options:replication_factor = 1;

      2. create a cf via cql (3.0)

      create table test1 (
      a int,
      b int,
      c int,
      primary key (a, b)
      );

      If i have a look at the schema in cli i noticed that there is no column metadata for columns not part of primary key.

      create column family test1
      with column_type = 'Standard'
      and comparator = 'CompositeType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type)'
      and default_validation_class = 'UTF8Type'
      and key_validation_class = 'Int32Type'
      and read_repair_chance = 0.1
      and dclocal_read_repair_chance = 0.0
      and gc_grace = 864000
      and min_compaction_threshold = 4
      and max_compaction_threshold = 32
      and replicate_on_write = true
      and compaction_strategy = 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
      and caching = 'KEYS_ONLY'
      and compression_options =

      {'sstable_compression' : 'org.apache.cassandra.io.compress.SnappyCompressor'}

      ;

      Please notice the default validation class: UTF8Type

      Now i would like to insert value > 127 via cassandra client (no cql, part of mr-jobs). Have a look at the attachement.

      Batch mutate fails:

      InvalidRequestException(why:(String didn't validate.) [test][test1][1:c] failed validation)

      A validator for column value is fetched in ThriftValidation::validateColumnData which returns always the default validator which is UTF8Type as described above (The ColumnDefinition for given column name "c" is always null)

      In UTF8Type there is a check for

      if (b > 127)
      return false;

      Anyway, maybe i'm doing something wrong, but i used cql 3.0 for table creation. I assigned data types to all columns, but i can not set values for a composite column because the default validation class is used.

      I think the schema should know the correct validator even for composite columns. The usage of the default validation class does not make sense.

      Best Regards

      Bert Passek

      Attachments

        1. 4421.txt
          162 kB
          Alex Liu
        2. 4421-1.txt
          167 kB
          Alex Liu
        3. 4421-2.txt
          167 kB
          Alex Liu
        4. 4421-3.txt
          168 kB
          Alex Liu
        5. 4421-4.txt
          169 kB
          Alex Liu
        6. 4421-5.txt
          169 kB
          Alex Liu
        7. 4421-6.cb.txt
          169 kB
          Colin B.
        8. 4421-6-je.txt
          20 kB
          Alex Liu
        9. 4421-7-je.txt
          20 kB
          Alex Liu
        10. 4421-8-cb.txt
          166 kB
          Colin B.
        11. 4421-8-je.txt
          167 kB
          Alex Liu
        12. 4421-9-je.txt
          169 kB
          Alex Liu
        13. 4421-10-patch-for-trunk.txt
          169 kB
          Alex Liu
        14. 4421-10-patch-for-1.2.txt
          171 kB
          Alex Liu

        Activity

          People

            alexliu68 Alex Liu
            bertpassek bert Passek
            Alex Liu
            Jonathan Ellis
            Ryan McGuire Ryan McGuire
            Votes:
            15 Vote for this issue
            Watchers:
            18 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: