Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-1093

ConvertJSONToSQL incorrectly detects required columns

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.5.0
    • Component/s: Extensions
    • Labels:
      None

      Description

      create table device_pings(id varchar not null, ts varchar not null, bssid varchar, ssid varchar, noise integer, signal integer, constraint pk primary key (id, ts))
      

      With the example DDL above, neither SSID nor BSSID are required fields, yet ConvertJSONToSQL throws an exception if the input JSON lacks those fields:

      2015-11-01 17:47:10,373 ERROR [Timer-Driven Process Thread-6] o.a.n.p.standard.ConvertJSONToSQL ConvertJSONToSQL[id=a336eb2b-fc63-4118-b098-c0ded1dd5520] Failed to convert StandardFlowFileRecord[uuid=5d2c05f0-982e-4feb-94b1-d9946be730d4,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1446416358991-2, container=default, section=2], offset=203306, length=614],offset=0,name=1446418019645796000,size=614] to a SQL INSERT statement due to org.apache.nifi.processor.exception.ProcessException: JSON does not have a value for the Required column 'BSSID'; routing to failure: org.apache.nifi.processor.exception.ProcessException: JSON does not have a value for the Required column 'BSSID'
      2015-11-01 17:47:10,381 ERROR [Timer-Driven Process Thread-6] o.a.n.p.standard.ConvertJSONToSQL ConvertJSONToSQL[id=a336eb2b-fc63-4118-b098-c0ded1dd5520] Failed to convert StandardFlowFileRecord[uuid=727b8c3c-66c5-4d6a-8cdc-602da8b80132,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1446416358991-2, container=default, section=2], offset=203920, length=674],offset=0,name=1446418019645796000,size=674] to a SQL INSERT statement due to org.apache.nifi.processor.exception.ProcessException: JSON does not have a value for the Required column 'SSID'; routing to failure: org.apache.nifi.processor.exception.ProcessException: JSON does not have a value for the Required column 'SSID'
      

      The processor has "Unmatched Field Behavior" property. Should there be an additional property "Unmatched Column Behavior" property that allows SQL statements to be generated from the set of columns actually available in the JSON?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ozhurakousky Oleg Zhurakousky
                Reporter:
                randerzander Randy Gelhausen
              • Votes:
                0 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: