0

I have a remote machine, where the space is very limited (~15GB free) I have an NFS share mounted on this VM, the size of the share is ~600GBs. The NFS share has lightning-fast access from this machine.

I want to bring 600GB of files in the above share, to my local system (over internet) for offline use. (Chunked as may be 2GB archives using http or off-the-shelf tools - dropbox/onedrive) My local system has a slow access to this share. I have tried rsync -ing it to a local folder after mounting as NFS, But it goes on for ever - may be since it has millions of files.

Since the remote machine has less free space, Is there a way to create an incremental archive of the NFS share files in multiple sessions (Can restart archive creation after freeing up space.)

Environment:

  1. Local system is Mac
  2. The remote VM is Ubuntu

Local machine has ~ 1TB of free space.

I looked at tar/zpaq but they don't seem to have the necessary features. Is there any faster solution than writing a custom script?

Thanks in advance!

Improve this question
10
  • Is sshfs available on Mac? I found this: macFUSE. With sshfs you can use cp or whatever. I believe the fact NFS is involved is irrelevant. I believe if the files are available after you ssh to the remote VM then you can reach them with sshfs instead.
    – Kamil Maciorowski
    Commented Mar 6, 2021 at 15:31
  • This again depends on the not so fast network. The reason why I am looking for archives are primarily because of the efficient transfers. Individual files over SSH/NFS/rsyc etc. are prohibitively slow.
    – NS Gopikrishnan
    Commented Mar 6, 2021 at 15:52
  • Is the 600GB in one or more files? If in several files, how large are they? Do they contain text?
    – harrymc
    Commented Mar 6, 2021 at 16:05
  • @harrymc mainly executables, libraries, build tools. Large no. of individual files.
    – NS Gopikrishnan
    Commented Mar 6, 2021 at 16:08
  • How will you be bringing the files over? (Network from remote to local? Sneakernet?) Does the archive have to be compressed, or does it just need to hold many files?
    – grawity_u1686
    Commented Mar 6, 2021 at 16:14

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .