rsync alternative? (too many files)

[Tony Godshall]

On Sun, Mar 6, 2011 at 15:19, Seth David Schoen <schoen at loyalty.org> wrote:
> Tony Godshall writes:
>
>> > find . -type f -print0  -xdev | xargs -0
>> > ...running cp piped into ssh, or whatever.  'Ware slowness.
>>
>> Yeah.  I looked at doing a find ... -type d -print0 | xargs -0 mkdir
>> followed by a one that does the rsyncs without the recursion[1] so
>> that each rsync would have only one file to do, but that doesn't,
>> unless I'm missing something, preserve the hardlinks, which is pretty
>> important since I've got something like 2.5TB residing in about 1.5TB
>> after file-level deduplication that I'm trying to copy to a 2TB
>> removable volume.

[Seth]
> If you're sure that the filenames don't contain tabs, you can...

Hi Seth.

I must not have expressed myself clearly.

There are excessive unique files, not duplicate entries in a list of files.

The files have already been deduplicated in the sense that entries to
files containing the same content are hardlinks.

If I were to copy the files to new media without retaining the
hardlinks, they would take up way more space.

Which is why I was unable to take Rick's otherwise very good suggestion.

That is what I was trying to express.  Am I being clearer now?

I may have to go hack on rsync... :-(

[Don]
> Most of the GNU utilities have an option that will
> use a null byte instead of a linefeed as a delimiter. ...

Yes, Rick had that in his solution.  A very useful feature that I use
all the time.

Tony