360,000 favicons

Favicons are the (usually tiny) image files that browsers may use to represent websites in tabs, in the URL bar, or for bookmarks. This dataset contains about 360,000 favicons from popular websites.

These favicons were scraped in July 2016. I wrote a crawler that went through Alexa's top 1 million sites, and made a request for 'favicon.ico' at the site root. If I got a 200 response code, I saved the result as ${site_url}.ico. For domains that were identical but for the TLD (e.g. google.com, google.ca, google.jp...), I scraped only one favicon. My scraping/cleaning code is on GitHub here.

Of 1m sites crawled, 540k responded with a 200 code. The dataset has 360k images, which were the remains after filtering out empty files, non-image files, and corrupt/malformed image files.

all_favicons.tar.gz contains the 360k image files exactly as I received them from the site. Most are ICO files, with typical sizes being 16x16, 32x32, and 48x48. But there are a variety of unusual sizes and formats.

favicon_metadata.csv is a csv file with one row per favicon in the dataset, with image metadata (width, height, format, color depth) inferred by ImageMagick. For ico files containing multiple images, the metadata corresponds to whichever one is listed first in the ICODIR header.

`16_16.tar.gz` is a tarball containing all 16x16 favicons in the dataset, converted to PNG. It has 290k images.

`16_16_distinct.tar.gz` is identical to the above, but with 70k duplicate or near-duplicate images removed.

Alexa's top 1-million list includes 'adult' sites, so some URLs and favicons may be NSFW or offensive. (It's pretty hard to make a credible depiction of nudity in 256 pixels, but there are occasional attempts.)

If you liked this, you should also check out the Large Logo Dataset.

A copy of this dataset is also hosted on Kaggle, along with some example exploratory IPython notebooks.