Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-1282

Large memory reallocation by Arrow causes hang in jemalloc

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: C++
    • Labels:
      None

      Description

      When reallocating a large amount of memory, Arrow is either triggering a bug in jemalloc or has a bug itself in the memory manager (many different applications reporting same issue but not clear from jemalloc issue description if they're sure it's in jemalloc or caused by other issues like using multiple memory allocation libraries in the same process, multithreaded access, etc).

      Link to stack trace is here: https://gist.github.com/jeffknupp/73879feacf9c560afd4f1a20213dc6ef

      Link to issue in jemalloc GitHub is here: https://github.com/jemalloc/jemalloc/issues/802

      Originally observed in redis, discussed with jemalloc maintainer here: https://github.com/antirez/redis/issues/3799

      This is entirely reproducible on Ubuntu 16.04 xenial, which uses version 3.6.0 according to `apt` metadata.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                xhochy Uwe L. Korn
                Reporter:
                jeffknupp Jeff Knupp
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: