Uploaded image for project: 'Subversion'
  1. Subversion
  2. SVN-2341

Perl bindings cat method fails when using a local file repository

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.2.x
    • Fix Version/s: nonblocking
    • Component/s: bindings_swig_perl
    • Labels:
      None
    • Environment:

      Solaris

      Description

      I've created a Perl script to test the context->cat method.  The script is as
      follows:
      
      #!/bin/env perl
      #
      # Used to test performance difference between SVN and CVS in a Perl environment
      
      use SVN::Client;
      
      use Time::HiRes qw(gettimeofday tv_interval);
      use Time::Local;
      use Cwd qw(chdir cwd);
      use IPC::Open3;
      use IO::Select;
      
      #----------------------------------------------------------------------------
      #   Main Code
      #----------------------------------------------------------------------------
      print "Running test\n";
      
      my $ctx = new SVN::Client(
      	auth => [SVN::Client::get_username_provider()]);
      	
      my $ITERATIONS = 100;
      my $READ_BLOCK_SIZE = 8192;
      
      # Log Receiver for SVN calls
      sub log_receiver {
          my $change_paths = shift;
          my $revision = shift;
          my $author = shift;
          my $date = shift;
          my $message = shift;
          my $pool = shift;
      }
      
      # Notify Handler
      sub notify_handler {
          my $changed_item_path = shift;
          my $action_type = shift;
          my $changed_item_type = shift;
          my $changed_item_mime_type = shift;
          my $changed_item_state = shift;
          my $changed_item_revision = shift;
      }
      
      
      sub test_svn_cat {
          my $srcUrl = shift;
          my $srcRevision = shift;
          my $targetPath = shift;
          
          my $err;
      
          open(OUTPUT_HANDLE, ">$targetPath/svnOutput.txt");
          $err = $ctx->cat(\*OUTPUT_HANDLE, $srcUrl, $srcRevision);
      
          if (!close(OUTPUT_HANDLE)) {
              print "Close failed: $!\n";
              return 0;
          }
          return 1;
      }
      
      sub test_svn_cat_loop {
          my $srcUrl = shift;
          my $srcRevision = shift;
          my $targetPath = shift;
          my $svnCatFn = \&test_svn_cat;
      
          print "\nTesting SVN Cat ($ITERATIONS iterations)\n";
          print "Getting Revision $srcRevision of $srcUrl ...\n";
      
              
          my $t0 = [gettimeofday];
      
          my $local_iter = 0;
          while ($local_iter < $ITERATIONS) {
          	&$svnCatFn($srcUrl, $srcRevision, $targetPath);
              $local_iter++;
          }
          
          my $t1 = [gettimeofday];
          
          $t0_t1 = tv_interval $t0, $t1;
          
          my $avg = $t0_t1 / $ITERATIONS;
          
          print "Test SVN Cat\n";
          print "------------\n";
          print " # of iters:               $ITERATIONS\n";
          print " elapsed time (seconds):   $t0_t1\n";
          print " avg:                      $avg\n";
          
      }
      
      # Uncomment this when svnserve is running.  It will work
      #test_svn_cat_loop ('svn://wallaby/trunk/conftool/docobj/build.xml', 16009,
      '/ct/acct/ehillman/svnPerformance/perl/output');
      
      # Uncomment this when the repository is on the local file system.
      # For our environments, this will succeed until the 43rd iteration
      test_svn_cat_loop
      ('file:///ct/acct/ehillman/svnRepos/trunk/conftool/docobj/build.xml', 16009,
      '/ct/acct/ehillman/svnPerformance/perl/output');
      
      print "Test completed\n";
      
      Basically, if I call the test with a repository URL starting with "svn" (ie,
      accessing the repository via svnserve), this works fine.  If I change the url to
      access the repository directly via the filesystem, it fails.  If I change the
      iteration count to less than 43, it will work.  But any more will fail.
      
      The error seems to be because there are too many open file handles at one time.
       This is a kernel setting on my Sun box that I cannot change.  As the script is
      just doing the same thing over and over (writing the same file using cat to the
      same file on the filesystem), I can't see why all these handles need to be open
      at the same time.
      
      I've looked a bit at the generated code, and it looks like it's opening threads
      to write to files, but it's never closing the files?  Could this be related? 
      I've never done anything with SWIG bindings before, so I can't really offer any
      more insight into it.
      
      I've added this as a priority 1 item, simply because I'd like to replace a tool
      which currently uses CVS with Subversion.  But it will most likely be using the
      cat method to write out files from the revision, and in all likelihood, it will
      be far more than 100 files in a go.  So this is pretty pivotal in getting this
      into our application.
      

      Original issue reported by hildo

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              subversion-importer Subversion Importer
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: