9

There's this beautiful boy who periodically uploads pictures of himself to his website. I am trying to automate the process of downloading these images to my computer.

So far, I'm able to download his webpage and parse it for jpg files. I end up with a file like this.

http://stat.ameba.jp/user_images/20120129/19/maofish/f9/60/j/o0480064011762693689.jpg
http://imgstat.ameba.jp/view/d/70/stat001.ameba.jp/user_images/20120127/22/maofish/f7/3e/j/t02200293_0480064011759076335.jpg
http://imgstat.ameba.jp/view/d/70/stat001.ameba.jp/user_images/20120125/18/maofish/80/46/j/t02200293_0480064011755033425.jpg
http://imgstat.ameba.jp/view/d/70/stat001.ameba.jp/user_images/20120120/20/maofish/3c/99/j/t02200290_0480063311745603530.jpg
http://stat.ameba.jp/user_images/20100219/16/maofish/33/0b/j/t01400198_0140019810420649113.jpg
http://stat.ameba.jp/user_images/b0/09/10101851128_s.jpg
http://stat.ameba.jp/user_images/9c/26/10027225053_s.jpg

I can get any of those images by doing a wget on them, but I would like to automate this process to get everything in the list. I tried piping and redirecting to wget, but it doesn't work. How can I accomplish what I'm trying to do?

Improve this question
2
  • What have you already tried?
    – r4.
    Commented Jan 30, 2012 at 9:10
  • 1
    I have tried wget | list and wget < list.
    – tony_sid
    Commented Jan 30, 2012 at 9:12

2 Answers 2

Reset to default
17

You can use -i option of wget such as:

$ wget -i input_file.txt

You will get all files downloaded in the current directory. You can see man wget for more options.

Improve this answer
2
  • Is there a way to tell wget to ignore images that are smaller than a certain size?
    – tony_sid
    Commented Jan 30, 2012 at 9:22
  • 1
    If the target is to remove the small files (less than some threshold), you can use find to delete them automatically after download.
    – Khaled
    Commented Jan 30, 2012 at 9:26
0 
minsize="50" # grab all over 50kb

for x in $(cat list)
do

    if [ "$(echo $(GET -Ssed $x | grep Length | awk '{print $2}') /128 |bc)" -ge $minsize ]; then
        wget -q $x
    fi
done
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .