1

The Problem: I want to host lots of digital photos from a website. More than the host has disk space.

The background: I have a vps which hosts a website, the machine has tons of bandwidth but not much disk space (50gb or so). I also have a server on a slower internet line with lots of disk space.

My solution: I was thinking, would it be feasible to store the images on the machine with the slow link, mount it onto the web host (with something like sshfs, or whatever), then create symbolic links for all the images and serve up those symbolic links from apache. Then create a script which scans the apache logs looking for the most popular X number of files, and copy those files to be local to the webserver (and therefore faster).

After a while I realised that this sounded like a late night type idea, and decided to sleep on it. This morning it's still sounding a little dodgy, but there must be some sort of apache caching module which works on disk->disk caching, right ? (all the ones i know of are disk->memory, moving important files to ram rather than moving to a local disk from a slower wan link or slower local disk).

TL;DR: Is this actually a dumb idea ?

Improve this question

2 Answers 2

Reset to default
1

If you can expose the slower link machine to the internet, you could run a webserver on that machine also.

This would mean that you remove the overhead of sshfs.

You could use the apache mod_rewrite module along with mod_proxy to force any image requests to go to your server:

RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule /(.*)\.(jpg|gif|png) http://slow.server.com/$1.$2 [P]

The RewriteCond rule says only do the rewrite if the requested file doesn't exist on the server. Then the RewriteRule says if the request is an image, go get it from the slow server.

Then seperately you just need to track popular images and upload them to the server. If they are present they'll be used.

Improve this answer
2
  • hmm, that might work. The machine i was thinking of does have apache on it but it's used for custom cgi's and stuff that's very likely to be insecure. I might be able to run a second apache instance on another port though, or maybe a vpn link, though that'd have the same overhead as sshfs i'd imagine. Food for thought though, those rewrite lines are handy to know also.
    – Sirex
    Commented Jan 19, 2012 at 9:06
  • decided to give this a go, but with the sshfs for now, and redirecting to another local folder which is actually the slow server's files mounted locally. If the overhead is too much I'll try securing the apache install.
    – Sirex
    Commented Jan 19, 2012 at 9:34
0

I think what you are looking for is some sort of web-proxy that decides what should be kept inside the cache weighted on usage.

So maybe mod_proxy or mod_cache (there's even a mod_disk_cache) can be leveraged here.

I think in general, looking for a 'proxy' script will bring you somewhat more what you are looking for than searching for a 'cache' script.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .