Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-6945

Use minExtInfo=true to reduce the amount of data queried from Atlas

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.11.0
    • Component/s: Extensions
    • Labels:
      None

      Description

      When ReportLineageToAtlas queries the entities from Atlas, the AtlasEntityWithExtInfo response object also contains all the referred entities with all their data by default which leads to huge data transfers between Atlas and NiFi. On the other hand, this extended data of the related entities are not used by the reporting task, so they could be excluded from the query (using minExtInfo=true parameter) in order to reduce the response size.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                turcsanyip Peter Turcsanyi
                Reporter:
                turcsanyip Peter Turcsanyi
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h