Downloading file from FTP using cURL

Question

I'm trying to use a cURL command to download a file from an FTP server to a local drive on my computer. I've tried

curl "ftp://myftpsite" --user name:password -Q "CWD /users/myfolder/" -O "myfile.raw"

But it returns an error that says:

curl: Remote file name has no length!
curl: try 'curl --help' or 'curl --manual' for more information
curl: (6) Could not resolve host: myfile.raw; No data record of requested type

I've tried some other methods, but nothing seems to work.

Also, I'm not quite sure how to specify which folder I want the file to be downloaded to. How would I do that?

Answer 1

Try

curl -u user:password 'ftp://mysite/%2fusers/myfolder/myfile/raw' -o ~/Downloads/myfile.raw

In FTP URLs, the path is relative to the starting directory (usually your homedir). You need to specify an absolute path, and that means using %2f to specify /. This is needed because the path in ftp: URLs is treated as a list of slash-separated names, each of which is supposed to be given to a separate CWD command. The %2f is decoded after splitting. See RFC 1738 and FTP URLs.

As for the output location, just give a path to -o.

---

Security suggestions:

• Don't put your password in the URL. Storing it in ~/.netrc is not particularly secure either, but it at least is hidden from ps -ef.

• Your password is sent in clear text. If the server supports it, use curl --ssl-reqd or curl ftps://mysite/...

• Using SFTP (the SSH file transfer protocol) would be even better.

Answer 2

curl -T /users/myfolder/myfile.raw -u username:password "ftp://myftpsite/path/myfile.raw"

I use this all the time. It works like a charm.

Answer 3

Try:

curl "ftp://user:password@myftpsite/users/myfolder/myfile.raw"

(If the remote file name is 'myfile.raw')

Answer 4

It is not cURL but commandline and works super:

If you're not bound to curl, you might want to use wget in recursive mode but restricting it to one level of recursion, try the following;

wget --no-verbose --no-parent --recursive --level=1\
--no-directories --user=login --password=pass ftp://ftp.myftpsite.com/

• --no-parent : Do not ever ascend to the parent directory when retrieving recursively.
• --level=depth : Specify recursion maximum depth level depth. The default maximum depth is five layers.
• --no-directories : Do not create a hierarchy of directories when retrieving recursively.
• --delete-after : can be added if you need to delete files after downloading.
• --no-host-directories : to download right in '.' current folder, not create directory named by domain.
• --no-clobber : skip downloads that would download to existing files
• --continue : Continue getting a partially-downloaded file for more stability
• combine with cd : to define the destination directory

So this sample can look like a bit more complicated:

cd /home/destination/folder \
&& wget --no-verbose --no-parent --recursive --level=1 \
--no-directories --no-host-directories \
--no-clobber --continue \ 
--user="login" --password="pass" ftp://ftp.myftpsite.com/

Feel free to use such command in crontab to automate delivery and local folder synchronization.

crontab -e

*/5 * * * * cd /home/destination/folder && wget --no-verbose --no-parent --recursive --level=1 --no-directories --no-host-directories --no-clobber --continue --user=login --password=pass ftp://ftp.myftpsite.com/

Or add the key --delete-after for automatic delivery of files from remote to local folder.

If you have special characters in this command (e.g. password) -- put it in a separate .sh file and configure crontab to execute .sh only. Crontab is really tricky in characters escaping.

Answer 5

As yan suggests,

curl "ftp://user:password@myftpsite/users/myfolder/myfile.raw"

should work, but some FTP servers use security policies that are not standards-compliant. In those cases, the --ftp-method singlecwd or --ftp-method nocwd option may help.

Answer 6

If you're not bound to curl, you might want to use lftp , try the following;

lftp ftp://ftp.some.domain.com -u [email protected],'password' -e 'set ftp:ssl-allow no; mirror --Remove-source-files --verbose /remote/directory /home/destination/folder; bye'

Where

• -u username
• ,'password' : put your password rather in quotas
• --Remove-source-files : deete remove files after you downloaded them
• --verbose : good to see info but feel free to delete on production
• set ftp:ssl-allow no; : only in case if you do not need to check certificate (not ssl)