I am using Compute Canada's HPC cluster and want to download files from the cluster to a local directory. I referred to Uploading and downloading files to and from a HPC for instructions and entered the following command on my Windows Command Prompt (not the HPC cluster terminal):
scp [email protected]:/scratch/file.txt /path/to/local/directory
This requires me to login every time I want to download files.
Since I am working with a large number of files from several directories, I want to avoid having to login every time, since it is very time-consuming. Also, the downloads would be peformed in series (not parallel) if I were to enter them one after the other which also takes a long time. Can anyone explain how you can download files from several directories in an HPC cluster to local directories in parallel without having to login every time?