Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8707 Implement an async pure c++ HDFS client
  3. HDFS-12427

libhdfs++: Prevent Requests from holding dangling pointer to RpcEngine

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 3.2.0
    • hdfs-client
    • None

    Description

      The lifetime of Request objects is tied to the worker thread(s) in the async event loop. In the current code there's nothing that prevents a request from outliving the RpcEngine (bound to FileSystem) while it's waiting for IO. If the Request, or a task that makes a new request, outlives the RpcEngine it attempts to dereference a dangling pointer and either crashes or continues to run with bad data.

      Proposed fix is to reference count the RpcEngine via shared_ptr so that Requests can hold a weak_ptr to it. When a request or RpcConnection attempting to make a request needs something from the RpcEngine like a call id number it can promote the weak_ptr to a shared_ptr. If it's unable to promote because the RpcEngine has been destroyed the Request's handler can be invoked with an appropriate error message. A weak_ptr must be used rather than a shared_ptr to avoid reference cycles.

      Attachments

        1. HDFS-12427.HDFS-8707.004.patch
          52 kB
          James Clampffer
        2. HDFS-12427.HDFS-8707.003.patch
          51 kB
          James Clampffer
        3. HDFS-12427.HDFS-8707.002.patch
          44 kB
          James Clampffer
        4. HDFS-12427.HDFS-8707.001.patch
          44 kB
          James Clampffer
        5. HDFS-12427.HDFS-8707.000.patch
          45 kB
          James Clampffer

        Activity

          People

            James C James Clampffer
            James C James Clampffer
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: