Questions tagged [wget]
GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive command line tool, so it may easily be called from scripts, Cron jobs, terminals without X Window System (X11) support, etc.
906
questions
1
vote
0
answers
21
views
How to skip a directory when downloading with wget?
I want to download a website with wget in Linux that have the following 'structure':
sssssssssss.com/things/summer/tables/books/
There is a directory in the directory 'summer', named 'sport'.
I want ...
- 133
0
votes
0
answers
30
views
Wget show error certificate of <domain> name was signed using an insecure algorithm, while curl works
We are trying to check domain
wget domain <like httpx://abc.com>
we are getting error "The certificate of domain(abc.com) was signed using an insecure algorithm. But SSL certificate is ...
0
votes
0
answers
42
views
Retrieve all ZIPs from specific site subdirectory
I'm trying to retrieve the *.ZIP files from this Zophar.net Music section, specifically the NES console. The files are freely downloadable per each game separately, which will be a huge time sink to ...
0
votes
0
answers
16
views
How to list a tree of webpage files given url
I can use wget --mirror url to download a webpage, but how can I only list (see) a tree directory of that page in linux terminal?
For example I have a personal website where there is a .htaccess file ...
- 43
0
votes
0
answers
46
views
How skip downloading 'robot.txt.tmp' files?
I sometimes want to only download media files from a single web page, such as gif files, which I figured out with:
wget -P c:\temp -A .gif -r -l 1 -H -nd 'https://marketplace.visualstudio.com/items?...
- 309
0
votes
1
answer
61
views
retrieve response body from wget --method=PUT
I knwo that with GNU wget (https://www.gnu.org/software/wget/manual/wget.html) I can send PUT HTTP requests with --method=PUT, but I cannot figure out how to read the response body I get back.
What I'...
- 1
0
votes
0
answers
102
views
How to find download url from a given website to be used in wget?
I try to obtain a publicly available historical document from an archive in Germany, however, the download never starts as I believe the PDF file is too big to be downloaded within a certain time, so ...
- 113
0
votes
0
answers
222
views
Inconsistent SSL connection error when using wget
I'm trying to download some .nc files from a secure online database.
It seems like for roughly every other attempted file download, I get an 'Unable to establish SSL connection' message, and then wget ...
TML
0
votes
0
answers
43
views
YARN resource manager "HTTP request sent, awaiting response..."
we have Hadoop cluster with active/stand by resource manager services the active resource manager is on master1 machine and the stand by resource manager is on master2 machine
in our cluster YARN ...
- 861
-1
votes
1
answer
73
views
how to get back the file that has been downloaded using Invoke-Webrequest without providing the output switch?
I downloaded a large file, using wget which is just alias for Invoke-WebRequest commandlet in powershell, and it asked me url i provided, it completed, and gave me the Headers of HTTP request instead ...
- 149
2
votes
1
answer
583
views
Stop wget from making index.html for Options +Indexes directories
I am using this command to mirror a directory tree on my server:
wget -m -e robots=off --no-parent --reject-regex index.html --reject-regex aaa --no-verbose --timestamp https://example.com/folders/
&...
- 187
1
vote
1
answer
142
views
Getting "Cannot write to" message after successful mirroring of remote directory with wget
I have been trying to mirror a directory on my server using wget, then afterward update the downloaded tree with only newer files. The local machine is running MacOS 12.6.3; the server is CloudLinux, ...
- 187
0
votes
0
answers
126
views
WSL wget command returns HTML rather than downloadable csv file
I am running WSL (Windows Subsystem for Linux) and using Ubuntu 22.04 to test my homegrown bash scripts as a part of my CS course.
The task is to download a CSV (comma separated values) file from the ...
0
votes
0
answers
111
views
Why won't wget mirror the .zip files from this website?
This website has links to a bunch of sub-pages like /zx-origins/sub-page, and those pages in turn have a link to a .zip file on a sub-domain. My goal is to get those .zip files in one nice command, I ...
- 113
1
vote
0
answers
351
views
Can't get past the login page with wget
I know that there are many similar questions, but they are very dated, and I have already tried pretty much everything and it still didn't work.
I'm currently trying to download a whole website using ...
- 11