Hot on HuffPost Tech:

See More Stories
Engadget for the iPhone: download the app now
AOL Tech

FoxySpider is a Web crawler for image and video enthusiasts

I still remember the day fondly. I was 16 and I had just got my first broadband connection. 512 kilobits of pure DOWNLOAD! Ten times the bandwidth of my dial-up modem! Unlimited traffic! To this day I'm still amazed at how quickly I filled my 20GB hard drive with porn. But that's beyond the point -- back then, you had to either painstakingly Right Click > Save As on every image and video, or rely on a bunch of really crap standalone Web crawlers.

Fast forward to today and collecting porn image galleries is a lot easier. Not only do we have terabytes of hard disk space and megabit Internet connections, but you can use peer-to-peer services like BitTorrent to download entire archives at the click of a button. But if you still want to crawl sites and make your own archives, you can't go wrong with the Firefox add-on FoxySpider.

FoxySpider is marketed as a porn crawler. Out of the box it will crawl all images and videos on the page you're currently viewing, but it has tons of options. You can change the crawl depth, make it ignore files without thumbnails and change how many files are concurrently downloaded. You have complete control over the file types that are downloaded, so if you want to crawl music files, documents or indeed any file extension, FoxySpider also has you covered.

The output galleries are configurable too -- thumbnail size, background color and so on. There's no option to automatically save the galleries you create, however!

Tags: add-on, crawler, extension, firefox, foxyspider, image, music, porn, video, web crawler, WebCrawler

Comments

5