Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-2310

PhoenixConfigurationUtil.getUpsertColumnMetadataList() in Phoenix Mapreduce integration generates incorrect upsert statement for view immediately after issue view DDL

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 4.5.2
    • Fix Version/s: 4.6.0, 4.5.3
    • Component/s: None
    • Labels:
      None

      Description

      We ran into what I believe is a corner case that was causing a M/R job using the Phoenix / Pig integration to fail by virtue of a incorrect UPSERT statement being generated.

      The issue was intermittent. The bug is that the UPSERT statement generated by PhoenixConfigurationUtil.getUpsertColumnMetadataList() when invoked from PhoenixRecordWriter would, instead of the cf.column_name, would contain for certain columns the result class name + hashcode as generated by Java's Object.toString(). Since this was not a valid column name the Pig Script would blow-up.

      This only occurs if we are attempting to insert data into a Phoenix View and the DDL for the Phoenix View was issued recently such that the MetadataClient cache was for this view was populated by MetaDataClient.createTable().

      What is occurring is in this case we wrap the PColumn in a Delegate at lines 1898 and 1909. The DelegateColumn class used to wrap PColumn doesn't implement toString() and so the default Object toString() is used. If you restart the JVM and force Phoenix to re-read the metadata from SYSTEM.CATALOG this doesn't occur as in this case we don't wrap the PColumn instance.

      I have a test to repro and a possible patch I'll attach shortly.

        Attachments

        1. PHOENIX-2310-v1.patch
          6 kB
          Jan Fernando
        2. PHOENIX-2310-4.5.patch
          6 kB
          Jan Fernando
        3. PHOENIX-2310_v2.patch
          7 kB
          Jan Fernando

          Activity

            People

            • Assignee:
              jfernando_sfdc Jan Fernando
              Reporter:
              jfernando_sfdc Jan Fernando
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: