Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-7345

Multiple entity is created in Atlas for one Hive table if table name contains uppercase characters

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.11.4
    • Fix Version/s: 1.12.0
    • Component/s: Extensions
    • Labels:
      None

      Description

      Both Hive and NiFi create new hive_table entities in Atlas if table name contains uppercase character(s) in NiFi and lineage is reported into the entity that NiFi created. Hive is case insensitive, so it would be nice if Hive table names are treated like that within NiFi as well.

      Steps to reproduce
      1. Create a table in Hive (I did it in Hive CLI) - it doesn't matter if it's uppercase or lowercase, Hive's output will show it lowercase anyway
      2. A new entity in Atlas should be created and the Audits tab on Atlas UI should show that it is created by Hive
      3. Insert something into the Hive table
      4. I created the following flow in NiFi: SelectHiveQL -> LogAttribute - configure SelectHiveQL in a way that the Hive table it uses contains uppercase character(s)
      5. Start the flow
      6. A new hive_table entity in Atlas should be created (the table's name is what is in SelectHiveQL) and the Audits tab should show that it was created by NiFi

      There is no issue if the table name used in NiFi only contains lowercase characters.
      I could reproduce it using Hive 3 processors as well.

        Attachments

          Activity

            People

            • Assignee:
              turcsanyip Peter Turcsanyi
              Reporter:
              tamasfulop Tamas Fulop
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 10m
                10m