Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: ---
    • Fix Version/s: 1.8.0
    • Component/s: libsvn_ra_serf
    • Labels:
      None

      Description

      While timing checkouts over https I get occassional stalls with serf, I don't
      get stalls with neon.  I'm using Subversion trunk as a dataset, I'm running
      trunk mod_dav_svn and svn.  The client backtrace is:
      
      #0  0x00007f863094fc6a in link_requests ()
         from /usr/local/serf/lib/libserf-2.so.0
      #1  0x00007f8630950076 in reset_connection ()
         from /usr/local/serf/lib/libserf-2.so.0
      #2  0x00007f8630950e8a in read_from_connection ()
         from /usr/local/serf/lib/libserf-2.so.0
      #3  0x00007f8630950f50 in serf__process_connection ()
         from /usr/local/serf/lib/libserf-2.so.0
      #4  0x00007f863094eebc in serf_event_trigger ()
         from /usr/local/serf/lib/libserf-2.so.0
      #5  0x00007f863094f02e in serf_context_run ()
         from /usr/local/serf/lib/libserf-2.so.0
      #6  0x00007f86337c8536 in finish_report (report_baton=0x1ec0b30, 
          pool=0x1e58358) at ../src/subversion/libsvn_ra_serf/update.c:2372
      #7  0x00007f863594ad65 in svn_wc_crawl_revisions5 (wc_ctx=0x1e30d50, 
          local_abspath=0x1e58498 "/home/pm/sw/subversion/obj/xx", 
          reporter=0x7f86339d99e0, report_baton=0x1ec0b30, restore_files=1, 
          depth=svn_depth_unknown, honor_depth_exclude=1, 
          depth_compatibility_trick=0, use_commit_times=0, 
          cancel_func=0x41383b <svn_cl__check_cancel>, cancel_baton=0x0, 
          notify_func=0x4184d3 <notify>, notify_baton=0x1e5e428, 
          scratch_pool=0x1e58358) at ../src/subversion/libsvn_wc/adm_crawler.c:861
      #8  0x00007f8635c71b16 in update_internal (result_rev=0x0, 
          local_abspath=0x1e58498 "/home/pm/sw/subversion/obj/xx", 
          anchor_abspath=0x1e59cf0 "/home/pm/sw/subversion/obj/xx", 
          revision=0x7fff91099990, depth=svn_depth_unknown, depth_is_sticky=0, 
          ignore_externals=0, allow_unver_obstructions=0, adds_as_modification=1, 
          timestamp_sleep=0x7fff91099ac4, notify_summary=1, ctx=0x1e30c98, 
          pool=0x1e58358) at ../src/subversion/libsvn_client/update.c:399
      #9  0x00007f8635c720fc in svn_client__update_internal (result_rev=0x0, 
          local_abspath=0x1e58498 "/home/pm/sw/subversion/obj/xx", 
          revision=0x7fff91099c60, depth=svn_depth_unknown, depth_is_sticky=1, 
          ignore_externals=0, allow_unver_obstructions=0, adds_as_modification=1, 
          make_parents=0, innerupdate=0, timestamp_sleep=0x7fff91099ac4, 
          ctx=0x1e30c98, pool=0x1e58358)
          at ../src/subversion/libsvn_client/update.c:541
      #10 0x00007f8635c173b2 in svn_client__checkout_internal (result_rev=0x0, 
          url=0x1e5efe0 "https://localhost:8887/mirror/subversion/trunk", 
          local_abspath=0x1e58498 "/home/pm/sw/subversion/obj/xx", 
          peg_revision=0x7fff91099c50, revision=0x7fff91099c60, ra_cache=0x0, 
          depth=svn_depth_unknown, ignore_externals=0, allow_unver_obstructions=0, 
          timestamp_sleep=0x0, ctx=0x1e30c98, pool=0x1e58358)
          at ../src/subversion/libsvn_client/checkout.c:215
      #11 0x00007f8635c174fe in svn_client_checkout3 (result_rev=0x0, 
          URL=0x1e5efe0 "https://localhost:8887/mirror/subversion/trunk", 
          path=0x1e5f080 "xx", peg_revision=0x7fff91099c50, revision=0x7fff91099c60, 
          depth=svn_depth_unknown, ignore_externals=0, allow_unver_obstructions=0, 
          ctx=0x1e30c98, pool=0x1e58358)
          at ../src/subversion/libsvn_client/checkout.c:255
      #12 0x0000000000408971 in svn_cl__checkout (os=0x1e304c8, 
          baton=0x7fff91099e50, pool=0x1e30278)
          at ../src/subversion/svn/checkout-cmd.c:161
      #13 0x0000000000416750 in main (argc=6, argv=0x7fff9109a218)
          at ../src/subversion/svn/main.c:2586
      
      
      I'll attach the server backtrace.
      

      1. 1_gdb.txt
        24 kB
        Philip Martin
      2. 2_gdb.txt
        34 kB
        Philip Martin
      3. 3_3981-serf.patch
        1 kB
        Justin Erenkrantz

        Issue Links

          Activity

          Hide
          philipm Philip Martin added a comment -

          Attachment 1_gdb.txt has been added with description: server backtrace

          Show
          philipm Philip Martin added a comment - Attachment 1_gdb.txt has been added with description: server backtrace
          Hide
          philipm Philip Martin added a comment -

          Created an attachment (id=1181)
          server backtrace
          
          

          Show
          philipm Philip Martin added a comment - Created an attachment (id=1181) server backtrace
          Hide
          philipm Philip Martin added a comment -

          These occurred with a debug build with pool debugging enabled. I'll try a more
          normal build.
          

          Show
          philipm Philip Martin added a comment - These occurred with a debug build with pool debugging enabled. I'll try a more normal build.
          Hide
          philipm Philip Martin added a comment -

          This is the client stacktrace for a build without pool debugging:
          
          #0  0x00007fdce5c2601d in reset_connection ()
             from /usr/local/serf/lib/libserf-2.so.0
          #1  0x00007fdce5c26e8a in read_from_connection ()
             from /usr/local/serf/lib/libserf-2.so.0
          #2  0x00007fdce5c26f50 in serf__process_connection ()
             from /usr/local/serf/lib/libserf-2.so.0
          #3  0x00007fdce5c24ebc in serf_event_trigger ()
             from /usr/local/serf/lib/libserf-2.so.0
          #4  0x00007fdce5c2502e in serf_context_run ()
             from /usr/local/serf/lib/libserf-2.so.0
          #5  0x00007fdce8a98861 in finish_report (report_baton=0x16a2b30, 
              pool=0x163a358) at ../src/subversion/libsvn_ra_serf/update.c:2372
          #6  0x00007fdceac0bcd0 in svn_wc_crawl_revisions5 (wc_ctx=0x1612d50, 
              local_abspath=0x163a498 "/home/pm/sw/subversion/obj/xx", 
              reporter=0x7fdce8ca7980, report_baton=0x16a2b30, restore_files=1, 
              depth=svn_depth_unknown, honor_depth_exclude=1, 
              depth_compatibility_trick=0, use_commit_times=0, 
              cancel_func=0x41372b <svn_cl__check_cancel>, cancel_baton=0x0, 
              notify_func=0x418383 <notify>, notify_baton=0x1640428, 
              scratch_pool=0x163a358) at ../src/subversion/libsvn_wc/adm_crawler.c:861
          #7  0x00007fdceaf2e226 in update_internal (result_rev=0x0, 
              local_abspath=0x163a498 "/home/pm/sw/subversion/obj/xx", 
              anchor_abspath=0x163bcf0 "/home/pm/sw/subversion/obj/xx", 
              revision=0x7fff0d4af940, depth=svn_depth_unknown, depth_is_sticky=0, 
              ignore_externals=0, allow_unver_obstructions=0, adds_as_modification=1, 
              timestamp_sleep=0x7fff0d4afa74, notify_summary=1, ctx=0x1612c98, 
              pool=0x163a358) at ../src/subversion/libsvn_client/update.c:399
          #8  0x00007fdceaf2e80c in svn_client__update_internal (result_rev=0x0, 
              local_abspath=0x163a498 "/home/pm/sw/subversion/obj/xx", 
              revision=0x7fff0d4afc10, depth=svn_depth_unknown, depth_is_sticky=1, 
              ignore_externals=0, allow_unver_obstructions=0, adds_as_modification=1, 
              make_parents=0, innerupdate=0, timestamp_sleep=0x7fff0d4afa74, 
              ctx=0x1612c98, pool=0x163a358)
              at ../src/subversion/libsvn_client/update.c:541
          #9  0x00007fdceaed4280 in svn_client__checkout_internal (result_rev=0x0, 
              url=0x1640fe0 "https://localhost:8887/mirror/subversion/trunk", 
              local_abspath=0x163a498 "/home/pm/sw/subversion/obj/xx", 
              peg_revision=0x7fff0d4afc00, revision=0x7fff0d4afc10, ra_cache=0x0, 
              depth=svn_depth_unknown, ignore_externals=0, allow_unver_obstructions=0, 
              timestamp_sleep=0x0, ctx=0x1612c98, pool=0x163a358)
              at ../src/subversion/libsvn_client/checkout.c:215
          #10 0x00007fdceaed43cc in svn_client_checkout3 (result_rev=0x0, 
              URL=0x1640fe0 "https://localhost:8887/mirror/subversion/trunk", 
              path=0x1641080 "xx", peg_revision=0x7fff0d4afc00, revision=0x7fff0d4afc10, 
              depth=svn_depth_unknown, ignore_externals=0, allow_unver_obstructions=0, 
              ctx=0x1612c98, pool=0x163a358)
              at ../src/subversion/libsvn_client/checkout.c:255
          #11 0x0000000000408903 in svn_cl__checkout (os=0x16124c8, 
              baton=0x7fff0d4afe00, pool=0x1612278)
              at ../src/subversion/svn/checkout-cmd.c:161
          #12 0x0000000000416601 in main (argc=6, argv=0x7fff0d4b01c8)
              at ../src/subversion/svn/main.c:2586
          

          Show
          philipm Philip Martin added a comment - This is the client stacktrace for a build without pool debugging: #0 0x00007fdce5c2601d in reset_connection () from /usr/local/serf/lib/libserf-2.so.0 #1 0x00007fdce5c26e8a in read_from_connection () from /usr/local/serf/lib/libserf-2.so.0 #2 0x00007fdce5c26f50 in serf__process_connection () from /usr/local/serf/lib/libserf-2.so.0 #3 0x00007fdce5c24ebc in serf_event_trigger () from /usr/local/serf/lib/libserf-2.so.0 #4 0x00007fdce5c2502e in serf_context_run () from /usr/local/serf/lib/libserf-2.so.0 #5 0x00007fdce8a98861 in finish_report (report_baton=0x16a2b30, pool=0x163a358) at ../src/subversion/libsvn_ra_serf/update.c:2372 #6 0x00007fdceac0bcd0 in svn_wc_crawl_revisions5 (wc_ctx=0x1612d50, local_abspath=0x163a498 "/home/pm/sw/subversion/obj/xx", reporter=0x7fdce8ca7980, report_baton=0x16a2b30, restore_files=1, depth=svn_depth_unknown, honor_depth_exclude=1, depth_compatibility_trick=0, use_commit_times=0, cancel_func=0x41372b <svn_cl__check_cancel>, cancel_baton=0x0, notify_func=0x418383 <notify>, notify_baton=0x1640428, scratch_pool=0x163a358) at ../src/subversion/libsvn_wc/adm_crawler.c:861 #7 0x00007fdceaf2e226 in update_internal (result_rev=0x0, local_abspath=0x163a498 "/home/pm/sw/subversion/obj/xx", anchor_abspath=0x163bcf0 "/home/pm/sw/subversion/obj/xx", revision=0x7fff0d4af940, depth=svn_depth_unknown, depth_is_sticky=0, ignore_externals=0, allow_unver_obstructions=0, adds_as_modification=1, timestamp_sleep=0x7fff0d4afa74, notify_summary=1, ctx=0x1612c98, pool=0x163a358) at ../src/subversion/libsvn_client/update.c:399 #8 0x00007fdceaf2e80c in svn_client__update_internal (result_rev=0x0, local_abspath=0x163a498 "/home/pm/sw/subversion/obj/xx", revision=0x7fff0d4afc10, depth=svn_depth_unknown, depth_is_sticky=1, ignore_externals=0, allow_unver_obstructions=0, adds_as_modification=1, make_parents=0, innerupdate=0, timestamp_sleep=0x7fff0d4afa74, ctx=0x1612c98, pool=0x163a358) at ../src/subversion/libsvn_client/update.c:541 #9 0x00007fdceaed4280 in svn_client__checkout_internal (result_rev=0x0, url=0x1640fe0 "https://localhost:8887/mirror/subversion/trunk", local_abspath=0x163a498 "/home/pm/sw/subversion/obj/xx", peg_revision=0x7fff0d4afc00, revision=0x7fff0d4afc10, ra_cache=0x0, depth=svn_depth_unknown, ignore_externals=0, allow_unver_obstructions=0, timestamp_sleep=0x0, ctx=0x1612c98, pool=0x163a358) at ../src/subversion/libsvn_client/checkout.c:215 #10 0x00007fdceaed43cc in svn_client_checkout3 (result_rev=0x0, URL=0x1640fe0 "https://localhost:8887/mirror/subversion/trunk", path=0x1641080 "xx", peg_revision=0x7fff0d4afc00, revision=0x7fff0d4afc10, depth=svn_depth_unknown, ignore_externals=0, allow_unver_obstructions=0, ctx=0x1612c98, pool=0x163a358) at ../src/subversion/libsvn_client/checkout.c:255 #11 0x0000000000408903 in svn_cl__checkout (os=0x16124c8, baton=0x7fff0d4afe00, pool=0x1612278) at ../src/subversion/svn/checkout-cmd.c:161 #12 0x0000000000416601 in main (argc=6, argv=0x7fff0d4b01c8) at ../src/subversion/svn/main.c:2586
          Hide
          philipm Philip Martin added a comment -

          Attachment 2_gdb.txt has been added with description: server backtrace no pool debugging

          Show
          philipm Philip Martin added a comment - Attachment 2_gdb.txt has been added with description: server backtrace no pool debugging
          Hide
          philipm Philip Martin added a comment -

          Created an attachment (id=1182)
          server backtrace no pool debugging
          
          

          Show
          philipm Philip Martin added a comment - Created an attachment (id=1182) server backtrace no pool debugging
          Hide
          philipm Philip Martin added a comment -

          I also get hangs with 'svn export'.  Here is one such, with a debug libserf:
          
          #0  reset_connection (conn=0x26ca8a8, requeue_requests=1) at ./outgoing.c:440
          #1  0x00007f43edbeee8a in read_from_connection (conn=0x26ca8a8)
              at ./outgoing.c:1027
          #2  0x00007f43edbeef50 in serf__process_connection (conn=0x26ca8a8, events=1)
              at ./outgoing.c:1071
          #3  0x00007f43edbecebc in serf_event_trigger (s=0x24de400, 
              serf_baton=0x26ca8b8, desc=0x24de5e8) at ./context.c:233
          #4  0x00007f43edbed02e in serf_context_run (ctx=0x24de400, 
              duration=-694967296, pool=0x43d1608) at ./context.c:296
          #5  0x00007f43f0a60d82 in finish_report (report_baton=0x2519b28, 
              pool=0x24ac278) at ../src/subversion/libsvn_ra_serf/update.c:2379
          #6  0x00007f43f2ebc210 in svn_client_export5 (result_rev=0x0, 
              from_path_or_url=0x24dafe0 "https://localhost:8887/mirror/subversion/trunk",
          to_path=0x24db080 "xx", peg_revision=0x7f43f3102ef0, revision=0x7f43f3102ef0, 
              overwrite=0, ignore_externals=0, ignore_keywords=0, 
              depth=svn_depth_infinity, native_eol=0x0, ctx=0x24acc98, pool=0x24ac278)
              at ../src/subversion/libsvn_client/export.c:1202
          #7  0x000000000040d9b9 in svn_cl__export (os=0x24ac4c8, baton=0x7fffdd49d920, 
              pool=0x24ac278) at ../src/subversion/svn/export-cmd.c:106
          #8  0x0000000000416601 in main (argc=6, argv=0x7fffdd49dce8)
              at ../src/subversion/svn/main.c:2586
          
          

          Show
          philipm Philip Martin added a comment - I also get hangs with 'svn export'. Here is one such, with a debug libserf: #0 reset_connection (conn=0x26ca8a8, requeue_requests=1) at ./outgoing.c:440 #1 0x00007f43edbeee8a in read_from_connection (conn=0x26ca8a8) at ./outgoing.c:1027 #2 0x00007f43edbeef50 in serf__process_connection (conn=0x26ca8a8, events=1) at ./outgoing.c:1071 #3 0x00007f43edbecebc in serf_event_trigger (s=0x24de400, serf_baton=0x26ca8b8, desc=0x24de5e8) at ./context.c:233 #4 0x00007f43edbed02e in serf_context_run (ctx=0x24de400, duration=-694967296, pool=0x43d1608) at ./context.c:296 #5 0x00007f43f0a60d82 in finish_report (report_baton=0x2519b28, pool=0x24ac278) at ../src/subversion/libsvn_ra_serf/update.c:2379 #6 0x00007f43f2ebc210 in svn_client_export5 (result_rev=0x0, from_path_or_url=0x24dafe0 "https://localhost:8887/mirror/subversion/trunk", to_path=0x24db080 "xx", peg_revision=0x7f43f3102ef0, revision=0x7f43f3102ef0, overwrite=0, ignore_externals=0, ignore_keywords=0, depth=svn_depth_infinity, native_eol=0x0, ctx=0x24acc98, pool=0x24ac278) at ../src/subversion/libsvn_client/export.c:1202 #7 0x000000000040d9b9 in svn_cl__export (os=0x24ac4c8, baton=0x7fffdd49d920, pool=0x24ac278) at ../src/subversion/svn/export-cmd.c:106 #8 0x0000000000416601 in main (argc=6, argv=0x7fffdd49dce8) at ../src/subversion/svn/main.c:2586
          Hide
          philipm Philip Martin added a comment -

          So serf is stuck in this loop in outgoing.c:reset_connection:
          
              while (old_reqs) {
                  /* If we haven't started to write the connection, bring it over
                   * unchanged to our new socket.  Otherwise, call the cancel function.
                   */
                  if (requeue_requests && !old_reqs->written) {
                      serf_request_t *req = old_reqs;
                      old_reqs = old_reqs->next;
                      req->next = NULL;
                      link_requests(&conn->requests, &conn->requests_tail, req);
                  }
                  else {
                      cancel_request(old_reqs, &old_reqs, requeue_requests);
                  }
              }
          
          It's looping calling link_requests forever.
          

          Show
          philipm Philip Martin added a comment - So serf is stuck in this loop in outgoing.c:reset_connection: while (old_reqs) { /* If we haven't started to write the connection, bring it over * unchanged to our new socket. Otherwise, call the cancel function. */ if (requeue_requests && !old_reqs->written) { serf_request_t *req = old_reqs; old_reqs = old_reqs->next; req->next = NULL; link_requests(&conn->requests, &conn->requests_tail, req); } else { cancel_request(old_reqs, &old_reqs, requeue_requests); } } It's looping calling link_requests forever.
          Hide
          philipm Philip Martin added a comment -

          Sometimes the command runs to completion without a problem and sometimes it goes
          into an infinite loop in serf. Running under gdb I observe that infinite loops
          are always preceeded by a SIGPIPE. I see no SIGPIPE when the command runs to
          completion, but I do see one shortly before each infinite loop.  The client
          generally checkouts-out/exports a few files after the SIGPIPE before looping.
          
          The stacktrace for SIGPIPE is:
          
          #0  0x00007ffff64a0f0b in __libc_writev (fd=<value optimized out>, 
              vector=0x7870c8, count=<value optimized out>)
              at ../sysdeps/unix/sysv/linux/writev.c:51
          #1  0x00007ffff6ba3ed1 in apr_socket_sendv () from /usr/lib/libapr-1.so.0
          #2  0x00007ffff28a8225 in socket_writev (conn=0x786ff8) at ./outgoing.c:494
          #3  0x00007ffff28a88f7 in write_to_connection (conn=0x786ff8)
              at ./outgoing.c:756
          #4  0x00007ffff28a8feb in serf__process_connection (conn=0x786ff8, events=5)
              at ./outgoing.c:1101
          #5  0x00007ffff28a6ebc in serf_event_trigger (s=0x682400, serf_baton=0x787008, 
              desc=0x682608) at ./context.c:233
          #6  0x00007ffff28a702e in serf_context_run (ctx=0x682400, duration=-694967296, 
              pool=0x6e4338) at ./context.c:296
          #7  0x00007ffff5720b0b in finish_report (report_baton=0x6bdb28, pool=0x650278)
              at ../src/subversion/libsvn_ra_serf/update.c:2379
          #8  0x00007ffff7b91674 in svn_client_export5 (result_rev=0x0, 
              from_path_or_url=0x67efe0 "https://localhost:8887/mirror/subversion/trunk",
          to_path=0x67f080 "xx", peg_revision=0x7ffff7dde730, revision=0x7ffff7dde730, 
              overwrite=0, ignore_externals=0, ignore_keywords=0, 
              depth=svn_depth_infinity, native_eol=0x0, ctx=0x650c98, pool=0x650278)
              at ../src/subversion/libsvn_client/export.c:1202
          #9  0x000000000040da89 in svn_cl__export (os=0x6504c8, baton=0x7fffffffe610, 
              pool=0x650278) at ../src/subversion/svn/export-cmd.c:106
          #10 0x00000000004168e4 in main (argc=6, argv=0x7fffffffe9d8)
              at ../src/subversion/svn/main.c:2586
          
          

          Show
          philipm Philip Martin added a comment - Sometimes the command runs to completion without a problem and sometimes it goes into an infinite loop in serf. Running under gdb I observe that infinite loops are always preceeded by a SIGPIPE. I see no SIGPIPE when the command runs to completion, but I do see one shortly before each infinite loop. The client generally checkouts-out/exports a few files after the SIGPIPE before looping. The stacktrace for SIGPIPE is: #0 0x00007ffff64a0f0b in __libc_writev (fd=<value optimized out>, vector=0x7870c8, count=<value optimized out>) at ../sysdeps/unix/sysv/linux/writev.c:51 #1 0x00007ffff6ba3ed1 in apr_socket_sendv () from /usr/lib/libapr-1.so.0 #2 0x00007ffff28a8225 in socket_writev (conn=0x786ff8) at ./outgoing.c:494 #3 0x00007ffff28a88f7 in write_to_connection (conn=0x786ff8) at ./outgoing.c:756 #4 0x00007ffff28a8feb in serf__process_connection (conn=0x786ff8, events=5) at ./outgoing.c:1101 #5 0x00007ffff28a6ebc in serf_event_trigger (s=0x682400, serf_baton=0x787008, desc=0x682608) at ./context.c:233 #6 0x00007ffff28a702e in serf_context_run (ctx=0x682400, duration=-694967296, pool=0x6e4338) at ./context.c:296 #7 0x00007ffff5720b0b in finish_report (report_baton=0x6bdb28, pool=0x650278) at ../src/subversion/libsvn_ra_serf/update.c:2379 #8 0x00007ffff7b91674 in svn_client_export5 (result_rev=0x0, from_path_or_url=0x67efe0 "https://localhost:8887/mirror/subversion/trunk", to_path=0x67f080 "xx", peg_revision=0x7ffff7dde730, revision=0x7ffff7dde730, overwrite=0, ignore_externals=0, ignore_keywords=0, depth=svn_depth_infinity, native_eol=0x0, ctx=0x650c98, pool=0x650278) at ../src/subversion/libsvn_client/export.c:1202 #9 0x000000000040da89 in svn_cl__export (os=0x6504c8, baton=0x7fffffffe610, pool=0x650278) at ../src/subversion/svn/export-cmd.c:106 #10 0x00000000004168e4 in main (argc=6, argv=0x7fffffffe9d8) at ../src/subversion/svn/main.c:2586
          Hide
          philipm Philip Martin added a comment -

          This is happening about 1 in 2 or 1 in 3 checkouts over https at present, and
          that's enough to make it a 1.7 issue.
          

          Show
          philipm Philip Martin added a comment - This is happening about 1 in 2 or 1 in 3 checkouts over https at present, and that's enough to make it a 1.7 issue.
          Hide
          philipm Philip Martin added a comment -

          Doing the checkouts/exports over plain HTTP doesn't appear to trigger the bug.
          The client doesn't appear to receive any SIGPIPE either.
          

          Show
          philipm Philip Martin added a comment - Doing the checkouts/exports over plain HTTP doesn't appear to trigger the bug. The client doesn't appear to receive any SIGPIPE either.
          Hide
          gstein Greg Stein added a comment -

          Shift to 1.8.0, after ungating per r1158522.
          

          Show
          gstein Greg Stein added a comment - Shift to 1.8.0, after ungating per r1158522.
          Hide
          philipm Philip Martin added a comment -

          This still occurs using trunk@1333080.  During checkout with serf over https the
          client sometimes gets SIGPIPE and then shortly afterwards the client spins using
          100% CPU.  It doesn't seem to happen if Keepalive is disabled on the server.
          

          Show
          philipm Philip Martin added a comment - This still occurs using trunk@1333080. During checkout with serf over https the client sometimes gets SIGPIPE and then shortly afterwards the client spins using 100% CPU. It doesn't seem to happen if Keepalive is disabled on the server.
          Hide
          philipm Philip Martin added a comment -

          Using trunk@1349357 shows that recent serf changes have had some effect. 
          Checkouts that don't involve a SIGPIPE still succeed.  Checkouts that involve
          SIGPIPE now hang in poll() rather than spin in an infinite loop.  The checkout
          used to spin shortly after receiving SIGPIPE but now it carries on for a much
          longer time before hanging.  Sometimes long enough to receive multiple SIGPIPE.
           The backtrace during the hang is:
          
          #0  0x00007ffff649cf03 in __epoll_wait_nocancel ()
              at ../sysdeps/unix/syscall-template.S:82
          #1  0x00007ffff6b9cdcf in impl_pollset_poll () from /usr/lib/libapr-1.so.0
          #2  0x00007ffff6b9c350 in apr_pollset_poll () from /usr/lib/libapr-1.so.0
          #3  0x00007ffff288712d in serf_context_run ()
             from /usr/local/serf/lib/libserf-1.so.0
          #4  0x00007ffff571144e in finish_report (report_baton=0x65a760, pool=0x6582a8)
              at ../src/subversion/libsvn_ra_serf/update.c:2419
          #5  0x00007ffff78a4b76 in svn_wc_crawl_revisions5 (wc_ctx=0x652d50, 
              local_abspath=0x6583e8 "/home/pm/sw/subversion/obj/wc", 
              reporter=0x7ffff5924ec0, report_baton=0x65a760, restore_files=1, 
              depth=svn_depth_unknown, honor_depth_exclude=1, 
              depth_compatibility_trick=0, use_commit_times=0, 
              cancel_func=0x413943 <svn_cl__check_cancel>, cancel_baton=0x0, 
              notify_func=0x418bcf <notify>, notify_baton=0x6a6ae8, 
              scratch_pool=0x6582a8) at ../src/subversion/libsvn_wc/adm_crawler.c:858
          #6  0x00007ffff7bca7cb in update_internal (result_rev=0x0, 
              local_abspath=0x6583e8 "/home/pm/sw/subversion/obj/wc", 
              anchor_abspath=0x6599a0 "/home/pm/sw/subversion/obj/wc", 
              revision=0x7fffffffe180, depth=svn_depth_unknown, depth_is_sticky=0, 
              ignore_externals=0, allow_unver_obstructions=0, adds_as_modification=1, 
              timestamp_sleep=0x7fffffffe29c, notify_summary=1, ctx=0x652c98, 
              pool=0x6582a8) at ../src/subversion/libsvn_client/update.c:395
          #7  0x00007ffff7bcad6c in svn_client__update_internal (result_rev=0x0, 
              local_abspath=0x6583e8 "/home/pm/sw/subversion/obj/wc", 
              revision=0x7fffffffe400, depth=svn_depth_unknown, depth_is_sticky=1, 
              ignore_externals=0, allow_unver_obstructions=0, adds_as_modification=1, 
              make_parents=0, innerupdate=0, timestamp_sleep=0x7fffffffe29c, 
              ctx=0x652c98, pool=0x6582a8)
              at ../src/subversion/libsvn_client/update.c:537
          #8  0x00007ffff7b71466 in svn_client__checkout_internal (result_rev=0x0, 
              url=0x6a77a8 "https://localhost:8887/repos/asf/subversion/trunk", 
              local_abspath=0x6583e8 "/home/pm/sw/subversion/obj/wc", 
              peg_revision=0x7fffffffe3f0, revision=0x7fffffffe400, 
              depth=svn_depth_unknown, ignore_externals=0, allow_unver_obstructions=0, 
              timestamp_sleep=0x0, ctx=0x652c98, pool=0x6582a8)
              at ../src/subversion/libsvn_client/checkout.c:165
          #9  0x00007ffff7b71585 in svn_client_checkout3 (result_rev=0x0, 
              URL=0x6a77a8 "https://localhost:8887/repos/asf/subversion/trunk", 
              path=0x6a7850 "wc", peg_revision=0x7fffffffe3f0, revision=0x7fffffffe400, 
              depth=svn_depth_unknown, ignore_externals=0, allow_unver_obstructions=0, 
              ctx=0x652c98, pool=0x6582a8)
              at ../src/subversion/libsvn_client/checkout.c:205
          #10 0x0000000000408cb9 in svn_cl__checkout (os=0x6524c8, baton=0x7fffffffe600, 
              pool=0x652278) at ../src/subversion/svn/checkout-cmd.c:161
          #11 0x00000000004168be in main (argc=6, argv=0x7fffffffe9a8)
              at ../src/subversion/svn/main.c:2733
          
          

          Show
          philipm Philip Martin added a comment - Using trunk@1349357 shows that recent serf changes have had some effect. Checkouts that don't involve a SIGPIPE still succeed. Checkouts that involve SIGPIPE now hang in poll() rather than spin in an infinite loop. The checkout used to spin shortly after receiving SIGPIPE but now it carries on for a much longer time before hanging. Sometimes long enough to receive multiple SIGPIPE. The backtrace during the hang is: #0 0x00007ffff649cf03 in __epoll_wait_nocancel () at ../sysdeps/unix/syscall-template.S:82 #1 0x00007ffff6b9cdcf in impl_pollset_poll () from /usr/lib/libapr-1.so.0 #2 0x00007ffff6b9c350 in apr_pollset_poll () from /usr/lib/libapr-1.so.0 #3 0x00007ffff288712d in serf_context_run () from /usr/local/serf/lib/libserf-1.so.0 #4 0x00007ffff571144e in finish_report (report_baton=0x65a760, pool=0x6582a8) at ../src/subversion/libsvn_ra_serf/update.c:2419 #5 0x00007ffff78a4b76 in svn_wc_crawl_revisions5 (wc_ctx=0x652d50, local_abspath=0x6583e8 "/home/pm/sw/subversion/obj/wc", reporter=0x7ffff5924ec0, report_baton=0x65a760, restore_files=1, depth=svn_depth_unknown, honor_depth_exclude=1, depth_compatibility_trick=0, use_commit_times=0, cancel_func=0x413943 <svn_cl__check_cancel>, cancel_baton=0x0, notify_func=0x418bcf <notify>, notify_baton=0x6a6ae8, scratch_pool=0x6582a8) at ../src/subversion/libsvn_wc/adm_crawler.c:858 #6 0x00007ffff7bca7cb in update_internal (result_rev=0x0, local_abspath=0x6583e8 "/home/pm/sw/subversion/obj/wc", anchor_abspath=0x6599a0 "/home/pm/sw/subversion/obj/wc", revision=0x7fffffffe180, depth=svn_depth_unknown, depth_is_sticky=0, ignore_externals=0, allow_unver_obstructions=0, adds_as_modification=1, timestamp_sleep=0x7fffffffe29c, notify_summary=1, ctx=0x652c98, pool=0x6582a8) at ../src/subversion/libsvn_client/update.c:395 #7 0x00007ffff7bcad6c in svn_client__update_internal (result_rev=0x0, local_abspath=0x6583e8 "/home/pm/sw/subversion/obj/wc", revision=0x7fffffffe400, depth=svn_depth_unknown, depth_is_sticky=1, ignore_externals=0, allow_unver_obstructions=0, adds_as_modification=1, make_parents=0, innerupdate=0, timestamp_sleep=0x7fffffffe29c, ctx=0x652c98, pool=0x6582a8) at ../src/subversion/libsvn_client/update.c:537 #8 0x00007ffff7b71466 in svn_client__checkout_internal (result_rev=0x0, url=0x6a77a8 "https://localhost:8887/repos/asf/subversion/trunk", local_abspath=0x6583e8 "/home/pm/sw/subversion/obj/wc", peg_revision=0x7fffffffe3f0, revision=0x7fffffffe400, depth=svn_depth_unknown, ignore_externals=0, allow_unver_obstructions=0, timestamp_sleep=0x0, ctx=0x652c98, pool=0x6582a8) at ../src/subversion/libsvn_client/checkout.c:165 #9 0x00007ffff7b71585 in svn_client_checkout3 (result_rev=0x0, URL=0x6a77a8 "https://localhost:8887/repos/asf/subversion/trunk", path=0x6a7850 "wc", peg_revision=0x7fffffffe3f0, revision=0x7fffffffe400, depth=svn_depth_unknown, ignore_externals=0, allow_unver_obstructions=0, ctx=0x652c98, pool=0x6582a8) at ../src/subversion/libsvn_client/checkout.c:205 #10 0x0000000000408cb9 in svn_cl__checkout (os=0x6524c8, baton=0x7fffffffe600, pool=0x652278) at ../src/subversion/svn/checkout-cmd.c:161 #11 0x00000000004168be in main (argc=6, argv=0x7fffffffe9a8) at ../src/subversion/svn/main.c:2733
          Hide
          philipm Philip Martin added a comment -

          r1349699 enables running the regression tests over https. The tests would PASS
          with neon (before it was disabled) but hang with serf.
          

          Show
          philipm Philip Martin added a comment - r1349699 enables running the regression tests over https. The tests would PASS with neon (before it was disabled) but hang with serf.
          Hide
          philipm Philip Martin added a comment -

          I've managed to run the regression tests over SSL without a hang using
          trunk@1349736.  A large checkout still hangs after receiving SIGPIPE. The most
          recent stacktrace is back in link_requests:
          
          #0  0x00007ffff3f46d7a in link_requests ()
             from /usr/local/serf/lib/libserf-1.so.0
          #1  0x00007ffff3f471a6 in reset_connection ()
             from /usr/local/serf/lib/libserf-1.so.0
          #2  0x00007ffff3f47fba in read_from_connection ()
             from /usr/local/serf/lib/libserf-1.so.0
          #3  0x00007ffff3f48080 in serf__process_connection ()
             from /usr/local/serf/lib/libserf-1.so.0
          #4  0x00007ffff3f45fec in serf_event_trigger ()
             from /usr/local/serf/lib/libserf-1.so.0
          #5  0x00007ffff3f4615e in serf_context_run ()
             from /usr/local/serf/lib/libserf-1.so.0
          #6  0x00007ffff5b6573b in finish_report (report_baton=0x65a760, pool=0x6582a8)
              at ../src/subversion/libsvn_ra_serf/update.c:2427
          #7  0x00007ffff78a2902 in svn_wc_crawl_revisions5 (wc_ctx=0x652d50, 
              local_abspath=0x6583e8 "/home/pm/sw/subversion/obj/wc", 
              reporter=0x7ffff5d79680, report_baton=0x65a760, restore_files=1, 
              depth=svn_depth_unknown, honor_depth_exclude=1, 
              depth_compatibility_trick=0, use_commit_times=0, 
              cancel_func=0x41396b <svn_cl__check_cancel>, cancel_baton=0x0, 
              notify_func=0x418bf7 <notify>, notify_baton=0x6a6ae8, 
              scratch_pool=0x6582a8) at ../src/subversion/libsvn_wc/adm_crawler.c:858
          
          SIGPIPE is Apache closing a socket but I'm not sure exactly what config
          directive causes it to occur.  I can reproduce it over SSL with keep-alive
          enabled and MaxKeepAliveRequests set to 100.  Without SSL I don't see SIGPIPE.
          

          Show
          philipm Philip Martin added a comment - I've managed to run the regression tests over SSL without a hang using trunk@1349736. A large checkout still hangs after receiving SIGPIPE. The most recent stacktrace is back in link_requests: #0 0x00007ffff3f46d7a in link_requests () from /usr/local/serf/lib/libserf-1.so.0 #1 0x00007ffff3f471a6 in reset_connection () from /usr/local/serf/lib/libserf-1.so.0 #2 0x00007ffff3f47fba in read_from_connection () from /usr/local/serf/lib/libserf-1.so.0 #3 0x00007ffff3f48080 in serf__process_connection () from /usr/local/serf/lib/libserf-1.so.0 #4 0x00007ffff3f45fec in serf_event_trigger () from /usr/local/serf/lib/libserf-1.so.0 #5 0x00007ffff3f4615e in serf_context_run () from /usr/local/serf/lib/libserf-1.so.0 #6 0x00007ffff5b6573b in finish_report (report_baton=0x65a760, pool=0x6582a8) at ../src/subversion/libsvn_ra_serf/update.c:2427 #7 0x00007ffff78a2902 in svn_wc_crawl_revisions5 (wc_ctx=0x652d50, local_abspath=0x6583e8 "/home/pm/sw/subversion/obj/wc", reporter=0x7ffff5d79680, report_baton=0x65a760, restore_files=1, depth=svn_depth_unknown, honor_depth_exclude=1, depth_compatibility_trick=0, use_commit_times=0, cancel_func=0x41396b <svn_cl__check_cancel>, cancel_baton=0x0, notify_func=0x418bf7 <notify>, notify_baton=0x6a6ae8, scratch_pool=0x6582a8) at ../src/subversion/libsvn_wc/adm_crawler.c:858 SIGPIPE is Apache closing a socket but I'm not sure exactly what config directive causes it to occur. I can reproduce it over SSL with keep-alive enabled and MaxKeepAliveRequests set to 100. Without SSL I don't see SIGPIPE.
          Hide
          philipm Philip Martin added a comment -

          I don't think the MaxKeepAliveRequests value matters: I've seen the problem when
          the value is 100 and 10000 but not on every checkout. Checkout of Subversion
          trunk obviously involves more that 100 requests but fewer than 100 connections.
          I don't understand the connection between the config directives and SIGPIPE.
          Perhaps it is something outside apache? Machine load? Memory pressure?
          

          Show
          philipm Philip Martin added a comment - I don't think the MaxKeepAliveRequests value matters: I've seen the problem when the value is 100 and 10000 but not on every checkout. Checkout of Subversion trunk obviously involves more that 100 requests but fewer than 100 connections. I don't understand the connection between the config directives and SIGPIPE. Perhaps it is something outside apache? Machine load? Memory pressure?
          Hide
          philipm Philip Martin added a comment -

          The SIGPIPE effect might be a side-effect of using the debugger (the svn client
          ignores SIGPIPE). On Debian/stable using SSL over localhost I still see the
          original problem: an infinite loop in link_requests.  I have tried Ubuntu/12.04
          and not managed to reproduce the problem.
          

          Show
          philipm Philip Martin added a comment - The SIGPIPE effect might be a side-effect of using the debugger (the svn client ignores SIGPIPE). On Debian/stable using SSL over localhost I still see the original problem: an infinite loop in link_requests. I have tried Ubuntu/12.04 and not managed to reproduce the problem.
          Hide
          philipm Philip Martin added a comment -

          I can trigger the hang by checking with a client on my Debian box from a server
          on my Ubuntu box over my LAN. The client spins using 100% CPU, the stack is:
          
          #0  0x00007f4cc141881e in write_to_connection ()
             from /usr/local/serf/lib/libserf-1.so.0
          #1  0x00007f4cc141911b in serf__process_connection ()
             from /usr/local/serf/lib/libserf-1.so.0
          #2  0x00007f4cc1416fec in serf_event_trigger ()
             from /usr/local/serf/lib/libserf-1.so.0
          #3  0x00007f4cc141715e in serf_context_run ()
             from /usr/local/serf/lib/libserf-1.so.0
          #4  0x00007f4cc303673b in finish_report (report_baton=0x8ee760, pool=0x8ec2a8)
              at ../src/subversion/libsvn_ra_serf/update.c:2427
          #5  0x00007f4cc4d73902 in svn_wc_crawl_revisions5 (wc_ctx=0x8e6c68, 
              local_abspath=0x8ec3e8 "/home/pm/sw/subversion/obj/wc", 
              reporter=0x7f4cc324a680, report_baton=0x8ee760, restore_files=1, 
              depth=svn_depth_unknown, honor_depth_exclude=1, 
              depth_compatibility_trick=0, use_commit_times=0, 
              cancel_func=0x41396b <svn_cl__check_cancel>, cancel_baton=0x0, 
              notify_func=0x418bf7 <notify>, notify_baton=0x93aa28, 
              scratch_pool=0x8ec2a8) at ../src/subversion/libsvn_wc/adm_crawler.c:858
          
          and write_to_connection never returns.
          
          

          Show
          philipm Philip Martin added a comment - I can trigger the hang by checking with a client on my Debian box from a server on my Ubuntu box over my LAN. The client spins using 100% CPU, the stack is: #0 0x00007f4cc141881e in write_to_connection () from /usr/local/serf/lib/libserf-1.so.0 #1 0x00007f4cc141911b in serf__process_connection () from /usr/local/serf/lib/libserf-1.so.0 #2 0x00007f4cc1416fec in serf_event_trigger () from /usr/local/serf/lib/libserf-1.so.0 #3 0x00007f4cc141715e in serf_context_run () from /usr/local/serf/lib/libserf-1.so.0 #4 0x00007f4cc303673b in finish_report (report_baton=0x8ee760, pool=0x8ec2a8) at ../src/subversion/libsvn_ra_serf/update.c:2427 #5 0x00007f4cc4d73902 in svn_wc_crawl_revisions5 (wc_ctx=0x8e6c68, local_abspath=0x8ec3e8 "/home/pm/sw/subversion/obj/wc", reporter=0x7f4cc324a680, report_baton=0x8ee760, restore_files=1, depth=svn_depth_unknown, honor_depth_exclude=1, depth_compatibility_trick=0, use_commit_times=0, cancel_func=0x41396b <svn_cl__check_cancel>, cancel_baton=0x0, notify_func=0x418bf7 <notify>, notify_baton=0x93aa28, scratch_pool=0x8ec2a8) at ../src/subversion/libsvn_wc/adm_crawler.c:858 and write_to_connection never returns.
          Hide
          philipm Philip Martin added a comment -

          r1620 on serf trunk appears to fix the SIGPIPE problems: I see SIGPIPEs in gdb
          and the checkout continues to completion.  However I still see the occasional
          stall with the client waiting in poll().
          

          Show
          philipm Philip Martin added a comment - r1620 on serf trunk appears to fix the SIGPIPE problems: I see SIGPIPEs in gdb and the checkout continues to completion. However I still see the occasional stall with the client waiting in poll().
          Hide
          philipm Philip Martin added a comment -

          I also occasionally see this:
          
          ../src/subversion/libsvn_wc/wc_db.c:1707: (apr_err=235000)
          svn: E235000: In file '../src/subversion/libsvn_wc/wc_db.c' line 1707: assertion
          failed (SVN_IS_VALID_REVNUM(changed_rev))
          Aborted (core dumped)
          
          I believe this is a serf/ra_serf problem. The stacktrace shows
          svn_wc__db_base_add_file called from update_editor.c:close_file. All of
          fb->changed_rev/date/author are unset. It looks like the properties have not
          been received, a missing PROPFIND perhaps?
          
          If I patch the code to insert a valid revnum the update continues instead of
          aborting but crashes shortly afterwards in svn_ra_serf__set_ver_prop while
          processing a response for the file that has already been added. So the
          properties are being received but too late.
          
          Generally I have to checkout branches, or tags, rather than trunk to get a
          checkout that is big enough to trigger this occasional bug.
          

          Show
          philipm Philip Martin added a comment - I also occasionally see this: ../src/subversion/libsvn_wc/wc_db.c:1707: (apr_err=235000) svn: E235000: In file '../src/subversion/libsvn_wc/wc_db.c' line 1707: assertion failed (SVN_IS_VALID_REVNUM(changed_rev)) Aborted (core dumped) I believe this is a serf/ra_serf problem. The stacktrace shows svn_wc__db_base_add_file called from update_editor.c:close_file. All of fb->changed_rev/date/author are unset. It looks like the properties have not been received, a missing PROPFIND perhaps? If I patch the code to insert a valid revnum the update continues instead of aborting but crashes shortly afterwards in svn_ra_serf__set_ver_prop while processing a response for the file that has already been added. So the properties are being received but too late. Generally I have to checkout branches, or tags, rather than trunk to get a checkout that is big enough to trigger this occasional bug.
          Hide
          jerenkrantz Justin Erenkrantz added a comment -

          Attachment 3_3981-serf.patch has been added with description: Rough patch

          Show
          jerenkrantz Justin Erenkrantz added a comment - Attachment 3_3981-serf.patch has been added with description: Rough patch
          Hide
          jerenkrantz Justin Erenkrantz added a comment -

          Created an attachment (id=1233)
          Rough patch
          
          

          Show
          jerenkrantz Justin Erenkrantz added a comment - Created an attachment (id=1233) Rough patch
          Hide
          jerenkrantz Justin Erenkrantz added a comment -

          Philip: can you please try the patch attached?
          
          It fixes the hangs for me on debian/stable against localhost and http...I left the printf's so you can 
          determine *when* it triggers.
          
          I'm still not exactly sure *why* this is necessary - but, let's confirm that it resolves the issue for you as 
          well.
          

          Show
          jerenkrantz Justin Erenkrantz added a comment - Philip: can you please try the patch attached? It fixes the hangs for me on debian/stable against localhost and http...I left the printf's so you can determine *when* it triggers. I'm still not exactly sure *why* this is necessary - but, let's confirm that it resolves the issue for you as well.
          Hide
          jerenkrantz Justin Erenkrantz added a comment -

          Uh, I meant https not http.
          

          Show
          jerenkrantz Justin Erenkrantz added a comment - Uh, I meant https not http.
          Hide
          philipm Philip Martin added a comment -

          I'm not seeing hangs very often at present so it's hard to know whether the
          patch helps.  With the patch applied a checkout of tags asserts:
          
          svn: E235000: In file '../src/subversion/libsvn_wc/wc_db.c' line 1707: assertion
          failed (SVN_IS_VALID_REVNUM(changed_rev))
          
          That happens on debian and ubuntu.  Before the assert I see lines like:
          
          xxx!!! 0x1001458 0x1cbbd98 0x1cbbe18 0x1001358
          xxx!!! 0x1001458 0x1001358 (nil) (nil)
          xxx!!! 0xa358f8 0x5a5eb58 0x5a5ebd8 0xa357f8
          xxx!!! 0xa358f8 0xa357f8 (nil) (nil)
          xxx!!! 0x5a5ead8 0x54e6018 0x54e6098 0x5a5ebd8
          xxx!!! 0x5a5ead8 0x5a5ebd8 (nil) (nil)
          xxx!!! 0x1cbbd18 0x108fdd8 0x1049a18 0x1cbbe18
          xxx!!! 0x1cbbd18 0x1cbbe18 (nil) (nil)
          xxx!!! 0x5a5dc58 0x5829ab8 0x5829a38 0x5a5dd58
          xxx!!! 0x5a5dc58 0x5a5dd58 (nil) (nil)
          xxx!!! 0x31cec68 0xe9a798 0x33170a8 0x31ced68
          
          

          Show
          philipm Philip Martin added a comment - I'm not seeing hangs very often at present so it's hard to know whether the patch helps. With the patch applied a checkout of tags asserts: svn: E235000: In file '../src/subversion/libsvn_wc/wc_db.c' line 1707: assertion failed (SVN_IS_VALID_REVNUM(changed_rev)) That happens on debian and ubuntu. Before the assert I see lines like: xxx!!! 0x1001458 0x1cbbd98 0x1cbbe18 0x1001358 xxx!!! 0x1001458 0x1001358 (nil) (nil) xxx!!! 0xa358f8 0x5a5eb58 0x5a5ebd8 0xa357f8 xxx!!! 0xa358f8 0xa357f8 (nil) (nil) xxx!!! 0x5a5ead8 0x54e6018 0x54e6098 0x5a5ebd8 xxx!!! 0x5a5ead8 0x5a5ebd8 (nil) (nil) xxx!!! 0x1cbbd18 0x108fdd8 0x1049a18 0x1cbbe18 xxx!!! 0x1cbbd18 0x1cbbe18 (nil) (nil) xxx!!! 0x5a5dc58 0x5829ab8 0x5829a38 0x5a5dd58 xxx!!! 0x5a5dc58 0x5a5dd58 (nil) (nil) xxx!!! 0x31cec68 0xe9a798 0x33170a8 0x31ced68
          Hide
          philipm Philip Martin added a comment -

          Here are the xxx!!! lines immediately before an assert:
          
          xxx!!! 0x7ffff22aae38 0x7ffff2de4d38 0x7ffff2404db8 0x7ffff22aaf38
          xxx!!! 0x7ffff22aae38 0x7ffff22aaf38 (nil) (nil)
          xxx!!! 0x7ffff2c027b8 0x7ffff3369b38 0x7ffff2173138 0x7ffff2c026b8
          xxx!!! 0x7ffff2c027b8 0x7ffff2c026b8 (nil) (nil)
          xxx!!! 0x7ffff3442538 0x7ffff3442138 0x7ffff0ec3338 0x7ffff34426b8
          xxx!!! 0x7ffff3442538 0x7ffff34426b8 (nil) (nil)
          svn: E235000: In file '../src/subversion/libsvn_wc/wc_db.c' line 1707: assertion
          failed (SVN_IS_VALID_REVNUM(changed_rev))
          
          That's a checkout with the client on Ubuntu and apache on Debian over a LAN. 
          This typically happens when the working copy is few hundred MB (388 in this
          case) but sometimes it reaches a couple of GB before asserting.
          

          Show
          philipm Philip Martin added a comment - Here are the xxx!!! lines immediately before an assert: xxx!!! 0x7ffff22aae38 0x7ffff2de4d38 0x7ffff2404db8 0x7ffff22aaf38 xxx!!! 0x7ffff22aae38 0x7ffff22aaf38 (nil) (nil) xxx!!! 0x7ffff2c027b8 0x7ffff3369b38 0x7ffff2173138 0x7ffff2c026b8 xxx!!! 0x7ffff2c027b8 0x7ffff2c026b8 (nil) (nil) xxx!!! 0x7ffff3442538 0x7ffff3442138 0x7ffff0ec3338 0x7ffff34426b8 xxx!!! 0x7ffff3442538 0x7ffff34426b8 (nil) (nil) svn: E235000: In file '../src/subversion/libsvn_wc/wc_db.c' line 1707: assertion failed (SVN_IS_VALID_REVNUM(changed_rev)) That's a checkout with the client on Ubuntu and apache on Debian over a LAN. This typically happens when the working copy is few hundred MB (388 in this case) but sometimes it reaches a couple of GB before asserting.
          Hide
          philipm Philip Martin added a comment -

          The callback problem appears to be a separate bug so I've raised issue #4195.
          

          Show
          philipm Philip Martin added a comment - The callback problem appears to be a separate bug so I've raised issue #4195.
          Hide
          jerenkrantz Justin Erenkrantz added a comment -

          Fixed upstream in serf r1621.  (Merged into 1.1.x branch in r1622...will aim to do a new serf release 
          before Berlin wraps up.)
          

          Show
          jerenkrantz Justin Erenkrantz added a comment - Fixed upstream in serf r1621. (Merged into 1.1.x branch in r1622...will aim to do a new serf release before Berlin wraps up.)
          Hide
          gstein Greg Stein added a comment -

          *** Issue 3984 has been marked as a duplicate of this issue. ***
          

          Show
          gstein Greg Stein added a comment - *** Issue 3984 has been marked as a duplicate of this issue. ***

            People

            • Assignee:
              Unassigned
              Reporter:
              philipm Philip Martin
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development