Cassandra
  1. Cassandra
  2. CASSANDRA-1603

cli: allow entering binary data to columns with no metadata

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Fix Version/s: 0.7 beta 3
    • Component/s: Tools
    • Labels:
      None

      Description

      CASSANDRA-1578 adds support for when we have metadata for the column in the schema, but we should also allow manually turning a given string into a binary format.

      one option would be to allow something like

      set CF[column]=timeuuid(12345678-1234-5678-1234-567812345678)
      

      but this only allows data entry, and would not help when doing gets.

      maybe adding a cli command "assume column family with ..." would be best, allowing the operator to create local metadata that unlike "update columnfamily" doesn't get sent to the server. (saving the local metadata between cli sessions would be good.)

      1. 1603-v2.txt
        19 kB
        Jonathan Ellis
      2. CASSANDRA-1603.patch
        18 kB
        Pavel Yaskevich
      3. CASSANDRA-1603-v3.patch
        19 kB
        Pavel Yaskevich

        Activity

        Jonathan Ellis created issue -
        Hide
        Pavel Yaskevich added a comment - - edited

        What if we make a function for each of the marshal types, e.g. long(), integer(), timeuuid() etc. However, we also dynamically change the column definition according to an assigned value. The reason being, we can know who is who e.g. :

        1). column CF['hello'] does not have validation_class in column_metadata
        2).

        set CF['hello'] = integer(1039459847598)

        3). according to the function we convert the value to the right type and create/edit ColumnDef with right validation_class then set a value to the column
        4).

        get CF['hello']

        without problem we can covert the value according to the validation_class set on the previous step...

        What do you think? Am I misunderstanding anything here?

        Show
        Pavel Yaskevich added a comment - - edited What if we make a function for each of the marshal types, e.g. long(), integer(), timeuuid() etc. However, we also dynamically change the column definition according to an assigned value. The reason being, we can know who is who e.g. : 1). column CF ['hello'] does not have validation_class in column_metadata 2). set CF['hello'] = integer(1039459847598) 3). according to the function we convert the value to the right type and create/edit ColumnDef with right validation_class then set a value to the column 4). get CF['hello'] without problem we can covert the value according to the validation_class set on the previous step... What do you think? Am I misunderstanding anything here?
        Hide
        Jonathan Ellis added a comment -

        what if we want to do a get, before doing a set?

        Show
        Jonathan Ellis added a comment - what if we want to do a get, before doing a set?
        Hide
        Pavel Yaskevich added a comment -

        If there were data before this kind of "typed" set it will be stored using default CF validator which is BytesType (if column specific validator was not set) or even column specific validator will be used. So if you "get" it will just use default CF or column validator (is previously set) to covert stored data into string (.getString())...

        Show
        Pavel Yaskevich added a comment - If there were data before this kind of "typed" set it will be stored using default CF validator which is BytesType (if column specific validator was not set) or even column specific validator will be used. So if you "get" it will just use default CF or column validator (is previously set) to covert stored data into string (.getString())...
        Hide
        Jonathan Ellis added a comment -

        right. i'm saying we need to support the use case of getting properly-formatted data back, without having to mutate data first.

        Show
        Jonathan Ellis added a comment - right. i'm saying we need to support the use case of getting properly-formatted data back, without having to mutate data first.
        Hide
        Pavel Yaskevich added a comment -

        How about:

         get CF['hello'] as <type>
        

        Where <type> could be long, integer, timeuuid etc.

        What do you think?

        Show
        Pavel Yaskevich added a comment - How about: get CF['hello'] as <type> Where <type> could be long, integer, timeuuid etc. What do you think?
        Hide
        Jonathan Ellis added a comment -

        well, it would really have to be

        get CF[key][column] as <type>
        

        then if we remember that type so that

        get CF[key]
        

        shows column correctly, that should be fine.

        Show
        Jonathan Ellis added a comment - well, it would really have to be get CF[key][column] as <type> then if we remember that type so that get CF[key] shows column correctly, that should be fine.
        Hide
        Pavel Yaskevich added a comment -

        That is what I was thinking... Great!

        Show
        Pavel Yaskevich added a comment - That is what I was thinking... Great!
        Hide
        Pavel Yaskevich added a comment -

        Examples:

        set Category['my task'][time_spent] = lexicaluuid(00048df6-faef-2bb5-2772-4e30d6b86086)
        get Category['my task'][time_spent] as TimeUUIDType

        set Foo[bar][name] = utf8('this is utf8 string')

        Show
        Pavel Yaskevich added a comment - Examples: set Category ['my task'] [time_spent] = lexicaluuid(00048df6-faef-2bb5-2772-4e30d6b86086) get Category ['my task'] [time_spent] as TimeUUIDType set Foo [bar] [name] = utf8('this is utf8 string')
        Pavel Yaskevich made changes -
        Field Original Value New Value
        Attachment CASSANDRA-1603.patch [ 12457304 ]
        Pavel Yaskevich made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Reviewer jbellis
        Hide
        Jonathan Ellis added a comment -

        updated to have getFormatTypeForColumn support Function names.

        still needs to allow fixing errors – if i set type long for instance but that turns out to not be the kind of data that is in the column i need to be able to override that later. i.e. we need to handle overriding when hasValueValidator(columnNameInBytes, metadata) is true.

        Show
        Jonathan Ellis added a comment - updated to have getFormatTypeForColumn support Function names. still needs to allow fixing errors – if i set type long for instance but that turns out to not be the kind of data that is in the column i need to be able to override that later. i.e. we need to handle overriding when hasValueValidator(columnNameInBytes, metadata) is true.
        Jonathan Ellis made changes -
        Attachment 1603-v2.txt [ 12457306 ]
        Hide
        Pavel Yaskevich added a comment -

        get with `as <type>` always changing column value validator.

        Show
        Pavel Yaskevich added a comment - get with `as <type>` always changing column value validator.
        Pavel Yaskevich made changes -
        Attachment CASSANDRA-1603-v3.patch [ 12457310 ]
        Hide
        Jonathan Ellis added a comment -

        committed

        Show
        Jonathan Ellis added a comment - committed
        Jonathan Ellis made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Pavel Yaskevich added a comment -

        Can you please check if it is commited? I can't see it in git/svn right now...

        Show
        Pavel Yaskevich added a comment - Can you please check if it is commited? I can't see it in git/svn right now...
        Hide
        Jonathan Ellis added a comment -

        really committed now.

        Show
        Jonathan Ellis added a comment - really committed now.
        Hide
        Hudson added a comment -

        Integrated in Cassandra #567 (See https://hudson.apache.org/hudson/job/Cassandra/567/)
        add cli support for binary data withoutCfDef hints. patch by Pavel Yaskevich; reviewed by jbellis for CASSANDRA-1603

        Show
        Hudson added a comment - Integrated in Cassandra #567 (See https://hudson.apache.org/hudson/job/Cassandra/567/ ) add cli support for binary data withoutCfDef hints. patch by Pavel Yaskevich; reviewed by jbellis for CASSANDRA-1603
        Gavin made changes -
        Workflow no-reopen-closed, patch-avail [ 12523895 ] patch-available, re-open possible [ 12750679 ]
        Gavin made changes -
        Workflow patch-available, re-open possible [ 12750679 ] reopen-resolved, no closed status, patch-avail, testing [ 12757212 ]

          People

          • Assignee:
            Pavel Yaskevich
            Reporter:
            Jonathan Ellis
            Reviewer:
            Jonathan Ellis
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development