Details

    • Type: Task
    • Status: Open
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
    • old issue number:
      346

      Description

      Although supporting an ARRAY type helps, there are some use cases in which data is not homogenous. We should support a Struct data type for these cases.

        Activity

        Hide
        tzolkin Vaclav Loffelmann added a comment -

        What about to use google protocol buffers? It is fast and space efficient.
        Related PHOENIX-1165

        Show
        tzolkin Vaclav Loffelmann added a comment - What about to use google protocol buffers? It is fast and space efficient. Related PHOENIX-1165
        Hide
        alexdl alex kamil added a comment -

        any idea which release is going to support this?

        related: PHOENIX-370, PHOENIX-150

        Show
        alexdl alex kamil added a comment - any idea which release is going to support this? related: PHOENIX-370 , PHOENIX-150
        Hide
        pctony Tony Stevenson added a comment -

        Comment:ramkrishna:12/20/13 04:57:03 PM:

        assigned

        Show
        pctony Tony Stevenson added a comment - Comment:ramkrishna:12/20/13 04:57:03 PM: assigned
        Hide
        pctony Tony Stevenson added a comment -

        Comment:jamestaylor:12/20/13 04:56:53 PM:

        We should investigate using [Parquet](http://parquet.io/) as our underlying storage format for these structs (and potentially for JSON as well, -497)

        Show
        pctony Tony Stevenson added a comment - Comment:jamestaylor:12/20/13 04:56:53 PM: We should investigate using [Parquet] ( http://parquet.io/ ) as our underlying storage format for these structs (and potentially for JSON as well, -497)
        Hide
        pctony Tony Stevenson added a comment -

        Comment:jamestaylor:08/13/13 09:38:26 PM:

        This one is a dup of -239. I think it'd be similar in concept kiji-schema, as it would define the structure of a single KeyValue column in your schema (i.e. an instantiation of the struct would be stored in a single KeyValue), but there's could be other sibling KeyValue columns that aren't structs.

        I think the schema of the struct would be defined in the Phoenix metadata table (SYSTEM.TABLE) using a new struct type to differentiate it. We'd need to allow references in queries using a dotted notation. At upsert/insert time, you'd need to provide the struct in it's entirety.

        Other than using less space, since you don't have the overhead of an entire KeyValue with each value. I'm not convinced this adds a whole lot of value. You can essentially model the same thing with multiple columns. I'd rather see HBase come up with better/more condensed block encodings and have a condensed memory model that can better leverage these encodings.

        As far as -19, that one is different. It's for cases where you'd want to have very wide rows in which value information is encoded in the column qualifier. In this case, you'd define these set of columns as a "nested table" which you could join against the row that contains them. So a set of column qualifiers would look like another row to Phoenix.

        Show
        pctony Tony Stevenson added a comment - Comment:jamestaylor:08/13/13 09:38:26 PM: This one is a dup of -239. I think it'd be similar in concept kiji-schema, as it would define the structure of a single KeyValue column in your schema (i.e. an instantiation of the struct would be stored in a single KeyValue), but there's could be other sibling KeyValue columns that aren't structs. I think the schema of the struct would be defined in the Phoenix metadata table (SYSTEM.TABLE) using a new struct type to differentiate it. We'd need to allow references in queries using a dotted notation. At upsert/insert time, you'd need to provide the struct in it's entirety. Other than using less space, since you don't have the overhead of an entire KeyValue with each value. I'm not convinced this adds a whole lot of value. You can essentially model the same thing with multiple columns. I'd rather see HBase come up with better/more condensed block encodings and have a condensed memory model that can better leverage these encodings. As far as -19, that one is different. It's for cases where you'd want to have very wide rows in which value information is encoded in the column qualifier. In this case, you'd define these set of columns as a "nested table" which you could join against the row that contains them. So a set of column qualifiers would look like another row to Phoenix.
        Hide
        pctony Tony Stevenson added a comment -

        Comment:apurtell:08/13/13 06:55:18 PM:

        > how would this work with -19 and -239

        I have the same question.

        To what extent could this be similar to (or borrow from?) [kiji-schema](https://github.com/kijiproject/kiji-schema)?

        Show
        pctony Tony Stevenson added a comment - Comment:apurtell:08/13/13 06:55:18 PM: > how would this work with -19 and -239 I have the same question. To what extent could this be similar to (or borrow from?) [kiji-schema] ( https://github.com/kijiproject/kiji-schema)?
        Hide
        pctony Tony Stevenson added a comment -

        Comment:nmaillard:08/04/13 11:31:42 AM:

        +1
        structs would be great
        how would this work with -19 and -239

        Show
        pctony Tony Stevenson added a comment - Comment:nmaillard:08/04/13 11:31:42 AM: +1 structs would be great how would this work with -19 and -239
        Hide
        pctony Tony Stevenson added a comment -

        Comment:elevine:08/02/13 05:42:36 PM:

        +1
        This would extend Phoenix's support for semi-structured data.

        Show
        pctony Tony Stevenson added a comment - Comment:elevine:08/02/13 05:42:36 PM: +1 This would extend Phoenix's support for semi-structured data.
        Hide
        pctony Tony Stevenson added a comment -

        Comment:anoopsamjohn:07/30/13 09:41:57 PM:

        assigned

        Show
        pctony Tony Stevenson added a comment - Comment:anoopsamjohn:07/30/13 09:41:57 PM: assigned
        Hide
        pctony Tony Stevenson added a comment -

        Comment:anoopsamjohn:07/28/13 06:05:11 PM:

        Thanks James for filing this.

        Show
        pctony Tony Stevenson added a comment - Comment:anoopsamjohn:07/28/13 06:05:11 PM: Thanks James for filing this.

          People

          • Assignee:
            Unassigned
            Reporter:
            jamestaylor James Taylor
          • Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:

              Development