[BEAM-13270] Google Spanner dependency is too old; cannot deal with JSON type code

• Type: Bug
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Affects Version/s: 2.34.0
• Component/s: io-py-gcp
• Labels: types
• Environment: MacOs BigSur, python 3.7.9

Language

Python

• Assignee: Unassigned
• Reporter: Laura
• Created: 17/Nov/21 21:24
• Updated: 04/Jun/22 21:50

Description

Apache Beam is using google-cloud-spanner version 1.19.1, while current version of google-cloud-spanner is 3.11.1.

I was attempting to use ReadFromSpanner when I ran into  a ValueError - it was getting a type code 11 from Spanner and couldn't parse it. 

Here is (some) relevant code from v3.11.1 of Spanner: 

class TypeCode(proto.Enum):    
  r"""``TypeCode`` is used as part of [Type][google.spanner.v1.Type] to
  indicate the type of a Cloud Spanner value.    

  Each legal value of a type can be encoded to or decoded from a JSON   
  value, using the encodings described below. All Cloud Spanner values    
  can be ``null``, regardless of type; ``null``\ s are always encoded    
  as a JSON ``null``. 
  """
  TYPE_CODE_UNSPECIFIED = 0    
  BOOL = 1    
  INT64 = 2    
  FLOAT64 = 3    
  TIMESTAMP = 4    
  DATE = 5    
  STRING = 6    
  BYTES = 7    
  ARRAY = 8   
  STRUCT = 9    
  NUMERIC = 10    
  JSON = 11

and the same from v1.19.1:

_sym_db.RegisterEnumDescriptor(_TYPECODE)
TypeCode = enum_type_wrapper.EnumTypeWrapper(_TYPECODE)
TYPE_CODE_UNSPECIFIED = 0
BOOL = 1
INT64 = 2
FLOAT64 = 3T
IMESTAMP = 4
DATE = 5
STRING = 6
BYTES = 7
ARRAY = 8
STRUCT = 9
NUMERIC = 10 

There is no support for type code 11, aka JSON in Apache Beam right now.