Infrastructure
  1. Infrastructure
  2. INFRA-4665

Subversion Repository as .tar.gz archive

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Fix Version/s: Initial Clearing
    • Component/s: Subversion
    • Labels:
      None

      Description

      Based on an earlier request https://issues.apache.org/jira/browse/INFRA-2606 i would like to know if it would be possible to get a tar.gz archive of the current subversion repository and not as compressed dump files... The problem i have is that after downloading all the dump files it took ages to create a svn repository from it (in relationship with INFRA-2606 it took more that 10 days to load the repository). To download a 50 GiB or 60 GiB file does not take that long ;-)

      Many thanks in advance.
      Kind regards
      Karl Heinz Marbaise

        Activity

        Hide
        #asfinfra IRC Bot added a comment -
        <danielsh> Karl, just to clarify, you're asking for a .tar.gz of the entire /repos/asf on-disk repository directory, correct?
        Show
        #asfinfra IRC Bot added a comment - <danielsh> Karl, just to clarify, you're asking for a .tar.gz of the entire /repos/asf on-disk repository directory, correct?
        Hide
        Karl Heinz Marbaise added a comment -
        Yes exactly. The entire /repos/asf on disk...Yes..
        I know that would be really a large file about 60 GiB or more but my problem for my testings is that it took too long to create the repository out of the dump file which i can do if there is not other possibility...but it would save me weeks...
        Show
        Karl Heinz Marbaise added a comment - Yes exactly. The entire /repos/asf on disk...Yes.. I know that would be really a large file about 60 GiB or more but my problem for my testings is that it took too long to create the repository out of the dump file which i can do if there is not other possibility...but it would save me weeks...
        Hide
        #asfinfra IRC Bot added a comment -
        <danielsh> we're considering rsync instead of tar, I assume that'll work for you ?
        Show
        #asfinfra IRC Bot added a comment - <danielsh> we're considering rsync instead of tar, I assume that'll work for you ?
        Hide
        Karl Heinz Marbaise added a comment -
        My idea was to do that only once not on a regular base but if it's simpler for you ....
        Show
        Karl Heinz Marbaise added a comment - My idea was to do that only once not on a regular base but if it's simpler for you ....
        Hide
        #asfinfra IRC Bot added a comment -
        <danielsh> I was thinking of rsync for the bootstrap, rather than tar.gz.
        Show
        #asfinfra IRC Bot added a comment - <danielsh> I was thinking of rsync for the bootstrap, rather than tar.gz.
        Hide
        #asfinfra IRC Bot added a comment -
        <danielsh> I think you know what are the tradeoffs of using it for ongoing syncs too: more efficient but more prone to bugs such as #4129.
        Show
        #asfinfra IRC Bot added a comment - <danielsh> I think you know what are the tradeoffs of using it for ongoing syncs too: more efficient but more prone to bugs such as #4129.
        Hide
        Karl Heinz Marbaise added a comment -
        Have you found a decision how to solve this? Or wouldn't it be solved?
        Show
        Karl Heinz Marbaise added a comment - Have you found a decision how to solve this? Or wouldn't it be solved?
        Hide
        #asfinfra IRC Bot added a comment -
        <danielsh> we'll set up rsyncd once we get the cycles for it.
        Show
        #asfinfra IRC Bot added a comment - <danielsh> we'll set up rsyncd once we get the cycles for it.
        Hide
        Tony Stevenson added a comment -
        Daniel, ping?
        Show
        Tony Stevenson added a comment - Daniel, ping?
        Hide
        #asfinfra IRC Bot added a comment -
        <danielsh> Easy to do, but low priority. Not to mention eris disks risk
        Show
        #asfinfra IRC Bot added a comment - <danielsh> Easy to do, but low priority. Not to mention eris disks risk
        Hide
        Tony Stevenson added a comment -
        Transitioning to waiting for Infra
        Show
        Tony Stevenson added a comment - Transitioning to waiting for Infra
        Hide
        Tony Stevenson added a comment -
        Karl,

        Can you confirm that this would be a one off exercise? If so I will go and create the compressed tarball now.
        Show
        Tony Stevenson added a comment - Karl, Can you confirm that this would be a one off exercise? If so I will go and create the compressed tarball now.
        Hide
        #asfinfra IRC Bot added a comment -
        <danielsh> Karl, if you need this, please reopen the ticket and reply to Tony's question. Thanks.
        Show
        #asfinfra IRC Bot added a comment - <danielsh> Karl, if you need this, please reopen the ticket and reply to Tony's question. Thanks.
        Hide
        Karl Heinz Marbaise added a comment -
        [~pctony] Hi Tony, it would nice if you could handle that just once so have an updated repository dump file for further testing. Many thanks in advance.
        Show
        Karl Heinz Marbaise added a comment - [~pctony] Hi Tony, it would nice if you could handle that just once so have an updated repository dump file for further testing. Many thanks in advance.
        Hide
        #asfinfra IRC Bot added a comment -
        <danielsh> Tony --- set up rsyncd of a harmonia zfs snapshot ? I expect khmarbaise is based in EU
        Show
        #asfinfra IRC Bot added a comment - <danielsh> Tony --- set up rsyncd of a harmonia zfs snapshot ? I expect khmarbaise is based in EU
        Hide
        #asfinfra IRC Bot added a comment -
        <danielsh> (unassigning as I get dup emails)
        Show
        #asfinfra IRC Bot added a comment - <danielsh> (unassigning as I get dup emails)
        Hide
        Karl Heinz Marbaise added a comment -
        I'm located in germany.
        Show
        Karl Heinz Marbaise added a comment - I'm located in germany.
        Hide
        #asfinfra IRC Bot added a comment -
        <danielsh> Karl maybe you can set up a writable rsyncd and send us the user/pass? Then we'll run an rsync pushing from our end
        Show
        #asfinfra IRC Bot added a comment - <danielsh> Karl maybe you can set up a writable rsyncd and send us the user/pass? Then we'll run an rsync pushing from our end
        Hide
        #asfinfra IRC Bot added a comment -
        <danielsh> done
        Show
        #asfinfra IRC Bot added a comment - <danielsh> done

          People

          • Assignee:
            Unassigned
            Reporter:
            Karl Heinz Marbaise
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development