Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-2310

PhoenixConfigurationUtil.getUpsertColumnMetadataList() in Phoenix Mapreduce integration generates incorrect upsert statement for view immediately after issue view DDL

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 4.5.2
    • 4.6.0, 4.5.3
    • None
    • None

    Description

      We ran into what I believe is a corner case that was causing a M/R job using the Phoenix / Pig integration to fail by virtue of a incorrect UPSERT statement being generated.

      The issue was intermittent. The bug is that the UPSERT statement generated by PhoenixConfigurationUtil.getUpsertColumnMetadataList() when invoked from PhoenixRecordWriter would, instead of the cf.column_name, would contain for certain columns the result class name + hashcode as generated by Java's Object.toString(). Since this was not a valid column name the Pig Script would blow-up.

      This only occurs if we are attempting to insert data into a Phoenix View and the DDL for the Phoenix View was issued recently such that the MetadataClient cache was for this view was populated by MetaDataClient.createTable().

      What is occurring is in this case we wrap the PColumn in a Delegate at lines 1898 and 1909. The DelegateColumn class used to wrap PColumn doesn't implement toString() and so the default Object toString() is used. If you restart the JVM and force Phoenix to re-read the metadata from SYSTEM.CATALOG this doesn't occur as in this case we don't wrap the PColumn instance.

      I have a test to repro and a possible patch I'll attach shortly.

      Attachments

        1. PHOENIX-2310_v2.patch
          7 kB
          Jan Fernando
        2. PHOENIX-2310-4.5.patch
          6 kB
          Jan Fernando
        3. PHOENIX-2310-v1.patch
          6 kB
          Jan Fernando

        Activity

          People

            jfernando_sfdc Jan Fernando
            jfernando_sfdc Jan Fernando
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: