Uploaded image for project: 'Subversion'
  1. Subversion
  2. SVN-1899

svndumpfilter holds an entire revision of dumpstream in memory

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: all
    • Fix Version/s: 1.2.0
    • Component/s: svndumpfilter
    • Labels:
      None

      Description

      Hi
      
      I'm trying to use svndumpfilter (which doesn't seem to be listed as a 
      subcomponent btw.) to extract parts of my repository to load into a
      separate repository on another machine. It is dying however with the following
      strace:
      
      read(0, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 69632) = 
      69632
      mmap2(NULL, 1073750016, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) 
      = -1 ENOMEM (Cannot allocate memory)
      brk(0)                                  = 0x8729000
      brk(0x4872a000)                         = 0x8729000
      mmap2(NULL, 1073881088, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) 
      = -1 ENOMEM (Cannot allocate memory)
      mmap2(NULL, 2097152, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) 
      = 0x26bd7000
      munmap(0x26bd7000, 167936)              = 0
      munmap(0x26d00000, 880640)              = 0
      mprotect(0x26c00000, 135168, PROT_READ|PROT_WRITE) = 0
      mmap2(NULL, 1073750016, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) 
      = -1 ENOMEM (Cannot allocate memory)
      rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
      tgkill(1278, 1278, SIGABRT)             = 0
      --- SIGABRT (Aborted) @ 0 (0) ---
      +++ killed by SIGABRT +++
      
      It would appear to be attempting to mmap just over 1Gig of memory on stdin. I'm
      running it as follows:
      
      [root@svn root]# strace svndumpfilter include projects/2g include jonathan/2g 
      include livia/bcu include livia/vb < 2gstuff > 2gstuff.filtered 2>run.out
      
      2gstuff is a dump of the original repository.
      
      [root@svn root]# ls -l /proc/1306/fd/
      total 3
      lr-x------  1 root root 64 May 27 08:46 0 -> /root/2gstuff
      l-wx------  1 root root 64 May 27 08:46 1 -> /root/2gstuff.filtered
      l-wx------  1 root root 64 May 27 08:46 2 -> /root/run.out
      
      My system is running Fedora core 2 using 1.04 rpms supplied in the dev/testing
      repositories. It is dumping the 511th revision of the repository when it
      dies which contained three 256M files and one 128M file in the commit. Yes
      that really is 896Megs in a single commit which is probably why the memory
      usage spikes up over the 1Gig mark ... the machine has about a gig of memory
      and a fair amount of swap:
      [root@svn root]# free
                   total       used       free     shared    buffers     cached
      Mem:       1034900     166308     868592          0       9804     118976
      -/+ buffers/cache:      37528     997372
      Swap:      1052248      24528    1027720
      
      I'd be happy to supply any additional information someone may need on
      request.
      
      Thank you!
      
      --Craig
      

      Original issue reported by schlenterc

        Attachments

        1. 1_test.sh
          0.7 kB
          Shlomi Fish

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              subversion-importer Subversion Importer
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: