Saikat Basu’s article above showcases a number of search engines to penetrate the murky world of the deep or invisible web. In it Saikat Basu notes that:
“ the size of the open web is 167 terabytes. The Invisible Web is estimated at 91,000 terabytes. Check this out – the Library of Congress, in 1997, was figured to have close to 3,000 terabytes!”
This means that the open web makes up just 0.18% of the web. This also means that search engines like Google and Bing only index a minuscule fraction of the web given that neither indexes the deep web.
So folks when a librarian tells you not everything’s on Google they really do mean it!