Uploaded image for project: 'Atlas'
  1. Atlas
  2. ATLAS-3718

Update Hive hook to create ADLS-Gen2 entities for ABFS path references

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.0
    • 2.1.0
    • atlas-core
    • None

    Description

      Entity types for ADLS-Gen2 have been added recently via ATLAS-3660. Hive hook should be updated to create ADLS-Gen2 entities for references to ABFS paths - for example in 'insert overwrite' operation. This will capture the lineage between Hive tables and ABFS paths.

      Attachments

        1. adls-gen2-directory-entity.png
          168 kB
          Sarath Subramanian
        2. adls-gen2-container-entity.png
          165 kB
          Sarath Subramanian
        3. adles-gen2-account-entity.png
          225 kB
          Sarath Subramanian
        4. external-table-lineage.png
          171 kB
          Sarath Subramanian

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            sarath Sarath Subramanian
            sarath Sarath Subramanian
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment