[SVN-2341] Perl bindings cat method fails when using a local file repository

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.2.x
• Fix Version/s: nonblocking
• Component/s: bindings_swig_perl

Environment

Solaris

• Assignee: Unassigned
• Reporter: Subversion Importer
• Created: 20/Jun/05 03:02
• Updated: 27/Oct/18 16:34
• Resolved: 18/Jul/05 05:09

Description

I've created a Perl script to test the context->cat method.  The script is as
follows:

#!/bin/env perl
#
# Used to test performance difference between SVN and CVS in a Perl environment

use SVN::Client;

use Time::HiRes qw(gettimeofday tv_interval);
use Time::Local;
use Cwd qw(chdir cwd);
use IPC::Open3;
use IO::Select;

#----------------------------------------------------------------------------
#   Main Code
#----------------------------------------------------------------------------
print "Running test\n";

my $ctx = new SVN::Client(
	auth => [SVN::Client::get_username_provider()]);
	
my $ITERATIONS = 100;
my $READ_BLOCK_SIZE = 8192;

# Log Receiver for SVN calls
sub log_receiver {
    my $change_paths = shift;
    my $revision = shift;
    my $author = shift;
    my $date = shift;
    my $message = shift;
    my $pool = shift;
}

# Notify Handler
sub notify_handler {
    my $changed_item_path = shift;
    my $action_type = shift;
    my $changed_item_type = shift;
    my $changed_item_mime_type = shift;
    my $changed_item_state = shift;
    my $changed_item_revision = shift;
}


sub test_svn_cat {
    my $srcUrl = shift;
    my $srcRevision = shift;
    my $targetPath = shift;
    
    my $err;

    open(OUTPUT_HANDLE, ">$targetPath/svnOutput.txt");
    $err = $ctx->cat(\*OUTPUT_HANDLE, $srcUrl, $srcRevision);

    if (!close(OUTPUT_HANDLE)) {
        print "Close failed: $!\n";
        return 0;
    }
    return 1;
}

sub test_svn_cat_loop {
    my $srcUrl = shift;
    my $srcRevision = shift;
    my $targetPath = shift;
    my $svnCatFn = \&test_svn_cat;

    print "\nTesting SVN Cat ($ITERATIONS iterations)\n";
    print "Getting Revision $srcRevision of $srcUrl ...\n";

        
    my $t0 = [gettimeofday];

    my $local_iter = 0;
    while ($local_iter < $ITERATIONS) {
    	&$svnCatFn($srcUrl, $srcRevision, $targetPath);
        $local_iter++;
    }
    
    my $t1 = [gettimeofday];
    
    $t0_t1 = tv_interval $t0, $t1;
    
    my $avg = $t0_t1 / $ITERATIONS;
    
    print "Test SVN Cat\n";
    print "------------\n";
    print " # of iters:               $ITERATIONS\n";
    print " elapsed time (seconds):   $t0_t1\n";
    print " avg:                      $avg\n";
    
}

# Uncomment this when svnserve is running.  It will work
#test_svn_cat_loop ('svn://wallaby/trunk/conftool/docobj/build.xml', 16009,
'/ct/acct/ehillman/svnPerformance/perl/output');

# Uncomment this when the repository is on the local file system.
# For our environments, this will succeed until the 43rd iteration
test_svn_cat_loop
('file:///ct/acct/ehillman/svnRepos/trunk/conftool/docobj/build.xml', 16009,
'/ct/acct/ehillman/svnPerformance/perl/output');

print "Test completed\n";

Basically, if I call the test with a repository URL starting with "svn" (ie,
accessing the repository via svnserve), this works fine.  If I change the url to
access the repository directly via the filesystem, it fails.  If I change the
iteration count to less than 43, it will work.  But any more will fail.

The error seems to be because there are too many open file handles at one time.
 This is a kernel setting on my Sun box that I cannot change.  As the script is
just doing the same thing over and over (writing the same file using cat to the
same file on the filesystem), I can't see why all these handles need to be open
at the same time.

I've looked a bit at the generated code, and it looks like it's opening threads
to write to files, but it's never closing the files?  Could this be related? 
I've never done anything with SWIG bindings before, so I can't really offer any
more insight into it.

I've added this as a priority 1 item, simply because I'd like to replace a tool
which currently uses CVS with Subversion.  But it will most likely be using the
cat method to write out files from the revision, and in all likelihood, it will
be far more than 100 files in a go.  So this is pretty pivotal in getting this
into our application.

Original issue reported by hildo