Cassandra
  1. Cassandra
  2. CASSANDRA-3252

IntegerType columns cause exceptions when use in Pig.

    Details

      Description

      CassandraStorage uses the validator classes to determine how to marshal column values, if the type is IntegerType a BigInteger is returned and Pig throws an exception on writing the value to intermediate storage. The exception is thrown because BigInteger is not one of the data types that Pig can handle.

      1. CASSANDRA-3252.patch
        3 kB
        Dana H. P'Simer, Jr.

        Activity

        Hide
        Dana H. P'Simer, Jr. added a comment -

        A somewhat kludgey fix for IntegerType data in cassandra.

        Show
        Dana H. P'Simer, Jr. added a comment - A somewhat kludgey fix for IntegerType data in cassandra.
        Hide
        Dana H. P'Simer, Jr. added a comment -

        Fixed this by adding a special case for IntegerType data that will attempt to convert the BigInteger to a format that Pig can handle.

        Show
        Dana H. P'Simer, Jr. added a comment - Fixed this by adding a special case for IntegerType data that will attempt to convert the BigInteger to a format that Pig can handle.
        Hide
        Jonathan Ellis added a comment -

        Wouldn't a better fix be "use longtype or int32type if that's what you want instead?"

        Show
        Jonathan Ellis added a comment - Wouldn't a better fix be "use longtype or int32type if that's what you want instead?"
        Hide
        Brandon Williams added a comment -

        This is already handles by CASSANDRA-2810

        Show
        Brandon Williams added a comment - This is already handles by CASSANDRA-2810
        Hide
        Dana H. P'Simer, Jr. added a comment -

        @Jonathan, I did not see any int32type and I just checked again and there is no such type in the org.apache.cassandra.db.marshal package. Where is this type? On the other hand, LongType is not appropriate since it expectes 8 bytes and there are only 4 in the data.

        @Brandon, even the latest version of that patch does not handle the case where the BigInteger is larger than 32 bits. I assume that BigInteger was used to allow for arbitrarily large integers. Your code would break if the data were larger than 32 bits. However, I do like moving the logic into a separate function.

        Show
        Dana H. P'Simer, Jr. added a comment - @Jonathan, I did not see any int32type and I just checked again and there is no such type in the org.apache.cassandra.db.marshal package. Where is this type? On the other hand, LongType is not appropriate since it expectes 8 bytes and there are only 4 in the data. @Brandon, even the latest version of that patch does not handle the case where the BigInteger is larger than 32 bits. I assume that BigInteger was used to allow for arbitrarily large integers. Your code would break if the data were larger than 32 bits. However, I do like moving the logic into a separate function.
        Hide
        Jonathan Ellis added a comment -

        int32type is new in 1.0

        Show
        Jonathan Ellis added a comment - int32type is new in 1.0

          People

          • Assignee:
            Unassigned
            Reporter:
            Dana H. P'Simer, Jr.
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development