Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-5920

Add support to execute OpenCypher query against Neo4j

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Won't Do
    • Affects Version/s: 1.10.7
    • Fix Version/s: None
    • Component/s: hooks, operators
    • Labels:
      None

      Description

      As a DAG developer

      I want to create DAG tasks to execute OpenCypher queries against a graph database

      So that the output can be used elsewhere in a DAG / business

      UPDATE:

      Feedback through the development effort has highlighted some considerations:

      • AIP-21 and any other future plans for the Apache Airflow project need to be taken into consideration
      • A complete featrue set should be implemented considering all use cases:
        • Export to common formats
        • Importing data from common formats
        • DML queries to manipulate the graph
      • A suitable connection definition should be implmented for Neo4J (not JDBC)
        • Connection should support bolt and bolt+routing protocols
        • Connection pooling should be managed by the hook to ensure correct transation bounaries within the session
      • A detailed failure mode analysis should also be submitted including:
        • Failed connection handling
        • Query execution handling
        • Transaction handling for multiple queries
        • Data integrity checking (import/export validation)
      • Considerations to ensure export does not fill the local disk

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                tfindlay Timothy Findlay
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 48h
                  48h
                  Remaining:
                  Remaining Estimate - 48h
                  48h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified