Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8707 Implement an async pure c++ HDFS client
  3. HDFS-12427

libhdfs++: Prevent Requests from holding dangling pointer to RpcEngine



    • Type: Sub-task
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.2.0
    • Component/s: hdfs-client
    • Labels:


      The lifetime of Request objects is tied to the worker thread(s) in the async event loop. In the current code there's nothing that prevents a request from outliving the RpcEngine (bound to FileSystem) while it's waiting for IO. If the Request, or a task that makes a new request, outlives the RpcEngine it attempts to dereference a dangling pointer and either crashes or continues to run with bad data.

      Proposed fix is to reference count the RpcEngine via shared_ptr so that Requests can hold a weak_ptr to it. When a request or RpcConnection attempting to make a request needs something from the RpcEngine like a call id number it can promote the weak_ptr to a shared_ptr. If it's unable to promote because the RpcEngine has been destroyed the Request's handler can be invoked with an appropriate error message. A weak_ptr must be used rather than a shared_ptr to avoid reference cycles.


        1. HDFS-12427.HDFS-8707.000.patch
          45 kB
          James Clampffer
        2. HDFS-12427.HDFS-8707.001.patch
          44 kB
          James Clampffer
        3. HDFS-12427.HDFS-8707.002.patch
          44 kB
          James Clampffer
        4. HDFS-12427.HDFS-8707.003.patch
          51 kB
          James Clampffer
        5. HDFS-12427.HDFS-8707.004.patch
          52 kB
          James Clampffer



            • Assignee:
              James C James Clampffer
              James C James Clampffer
            • Votes:
              0 Vote for this issue
              3 Start watching this issue


              • Created: