It would be interesting to see the effect of compressing the segments within the tar files with a sufficiently effective and performant compression algorithm:
- Can we increase overall throughput by trading CPU for IO?
- Can we scale to bigger repositories (in number of nodes) by squeezing in more segments per MB and thus pushing out onset of thrashing?
- What would be a good compression algorithm/library?
- Can/should we make this optional?
- Migration and compatibility issues?