Uploaded image for project: 'Atlas'
  1. Atlas
  2. ATLAS-3718

Update Hive hook to create ADLS-Gen2 entities for ABFS path references

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.0
    • 2.1.0
    • atlas-core
    • None

    Description

      Entity types for ADLS-Gen2 have been added recently via ATLAS-3660. Hive hook should be updated to create ADLS-Gen2 entities for references to ABFS paths - for example in 'insert overwrite' operation. This will capture the lineage between Hive tables and ABFS paths.

      Attachments

        1. adles-gen2-account-entity.png
          225 kB
          Sarath Subramanian
        2. adls-gen2-container-entity.png
          165 kB
          Sarath Subramanian
        3. adls-gen2-directory-entity.png
          168 kB
          Sarath Subramanian
        4. external-table-lineage.png
          171 kB
          Sarath Subramanian

        Issue Links

          Activity

            People

              sarath Sarath Subramanian
              sarath Sarath Subramanian
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: