What is the "deep" web?

2/17/2015

Although the “Deep Web” may be hard to understand, the concept is simple. Major search engines – Google, Yahoo, Bing – index public webpages to make it easier for you to find pages on the surface internet. The deep web is not indexed by these “visible web” search engines.

This “public” part of the internet is merely the surface of what is available on the internet. Tens of trillions, a number you cannot imagine, web pages populate the internet and most people will never see those “other” pages on the “invisible web”. They are mostly made up of boring statistics, data base information, and, of course, anything illegal you can imagine. The drawing above gives you an idea of what makes up the huge portion of the internet. But the deep web is not necessarily bad: your password protected Gmail messages live there, so do many other password protected sites.

Alas, there is an underbelly of the Deep Web called the “Dark Web”. The “dark web” should never be confused with the Deep Web. You may never want to visit there, but you really should know what it is all about. Specifically, the dark web is a collection of publicly visible websites that hide their IP addresses of servers that host them. Any of these sites can be visited by any web user but you cannot find these sites using the “surface” web search engine. Almost all of the sites on the dark web use the Tor encryption tool to hide their identity. You need to use the Tor browser to take you to those places on the dark web. The browser runs on a relay system that bounces signals around to different Tor-enabled computers all over the world.

Interesting in finding out more? Check out these articles:

http://www.pcadvisor.co.uk/how-to/internet/3593569/what-is-dark-web-how-access-dark-web/

http://www.brightplanet.com/2014/03/clearing-confusion-deep-web-vs-dark-web/

http://www.pcmag.com/article2/0,2817,2475580,00.asp

1 Comment

2/18/2015 04:07:57 am

As Yogi Berra could have put it, semantics are important, especially when dealing with words.
The “World Wide Web” (aka “The Web”) and the “Internet” are two interrelated, but different things. The Web uses one primary protocol (call it a Language if you like) called the Hyper-Text Transfer Protocol, generally abbreviated as HTTP. Yes, this is the http:// one sees as the address of web pages, in fact, that is what makes a web page a web page. The Internet understands the http language and that’s how the Web works. But the Internet understands many more languages than just http; some examples are:
FTP (File Transfer Protocol) to move files around between computers
SMTP (Simple Mail Transfer Protocol) to send emails from computer to computer
VOIP (Voice Over Internet Protocol) in which telephone calls are routed from phone to phone using the wires (and fibers) that made up the Internet rather than the old POTS (plain old telephone service. Yes, POTS is a thing. <grin>)
NNTP (Network News Transfer Protocol) in which the Internet allows exchange of news messages much like the old home-grown, pre-Internet, Bulletin Board Systems used to do.
The so-called “Deep Web” is nothing more than all the stuff that isn’t the Web. In fact, it has nothing at all to do with the web, it’s just the rest of the Internet. Another good analogy is the periodical section in a modern library – the periodicals (the web) are just a part of the entire library (the Internet.) In fact, the analogy even holds further as one can wander through the book stacks and see much more than just the periodicals, but there may be areas still hidden from you, like the rare books section that requires special access. The Internet is the same way, there are areas indexed by Google (the periodicals) unindexed, but publically available books (email, newsgroups), and locked away areas that require extra passwords.
The webmaster on any Internet-connected computer (by definition, the person responsible for http connections) has the option to be, or not to be, indexed by Google and other search engines. Most sites welcome search engines as that is now the primary method of finding sites on the Web. (The number one Google search term of 2014 was “Facebook”.) The sites that prefer not to be indexed simply add a “robots.txt” file to their server that contains instructions for search engines or; more severely, add a password requirement which search engines will not have.
Tor (originally “The Onion Router”, now just Tor) is an interesting case. Originally developed under a US DoD contract from DARPA (the same place the Internet came from!) it was built to help maintain anonymity over the Internet. It was thought that this would be a good way for dissidents under repressive regimes to communicate without fear of personal disclosure. In practice this is difficult; the Internet was not built with either security or privacy in mind. (The original objective in building the Internet was reliability.) An entity with the resources of a nation-state can de-anonymize Tor users. However, petty and not-so-petty crooks have seized upon Tor so that they can conduct their business without interference from … well … anybody. Like Bittorrent, how Tor has been used has tarnished an otherwise innocent protocol.
The “Dark Web” is an over-simplified media construct to try to explain a complicated subject. There is a dark side to almost anything, see also 'Rule 34'. :-)

Your comment will be posted after it is approved.

What is the "deep" web?

Leave a Reply.

TECH TEAM

Archives

Categories

contact us