XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • hdfs-client
    • None

    Description

      I propose implementing a set of memory new/delete pairs with additional checking to detect double deletes, read-after-delete, and write-after-deletes to help debug resource ownership issues and prevent new ones from entering the library.

      One of the most common issues we have is use-after-free issues. The continuation pattern makes these really tricky to debug because by the time a segsegv is raised the context of what has caused the error is long gone.

      The plan is to add allocators that can be turned on that can do the following, in order of runtime cost.
      1: no-op, forward through to default new/delete
      2: make sure the memory given to the constructor is dirty, memset free'd memory to 0
      3: implement operator new with mmap, lock that region of memory once it's been deleted; obviously this can't be left to run forever because the memory is never unmapped

      This should also put some groundwork in place for implementing specialized allocators for tiny objects that we churn through like std::string.

      Attachments

        1. HDFS-10188.HDFS-8707.000.patch
          4 kB
          Xiaowei Zhu
        2. HDFS-10188.HDFS-8707.001.patch
          4 kB
          Xiaowei Zhu
        3. HDFS-10188.HDFS-8707.002.patch
          8 kB
          Xiaowei Zhu
        4. HDFS-10188.HDFS-8707.003.patch
          8 kB
          Xiaowei Zhu
        5. HDFS-10188.HDFS-8707.004.patch
          8 kB
          James Clampffer

        Activity

          People

            xiaowei.zhu Xiaowei Zhu
            James Clampffer James Clampffer
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: