The invisible web is the hidden area of the Internet that is not searched automatically by search engines. It is information that is freely available on the Internet, but that will not be retrieved by normal search requests. The information that is hidden on the invisible web takes a little more work to find.
The visible web is information you can see or retrieve from search engines. The invisible web is sometimes referred to as the deep web. If you think of the Internet as a giant database, some of the information on the database is easier to find than others. The information held on the visible web is just a fraction of the information available on the Internet.
Internet search engine databases already hold huge amounts of information. When you type a question into a search engine, it will bring up a list of possible pages that could answer your question. It is less expensive for the search engine to bring up these answer pages than to store every possible page on the subject you are looking for. It also takes up less space in the search engine database.
Think of the millions of people who use the Internet every day. They are asking search engines questions, looking up subjects or just surfing the net. It is easier for the database to give out the answer page than to access the web pages directly.
A way of accessing material on the invisible web is simply to put the word database after your search query. If you are looking up holidays in Ireland, use the words holidays in Ireland database. This will direct the search engine to check searchable databases on your subject. A lot of databases that store information have the word database in their web pages, making them easier to find.
There are some web pages that are on the invisible web purely for technical reasons. Databases are created using spiders. These are computer programs that search around the web looking for information. However, the spiders cannot think or type. They can jump from link to link, but cannot think for themselves.
If the spider comes across a web page that needs a password access, they have no way of entering that page. There are many sites that require the user to log on and use a password to enter and view their content. Also if a web page has been created without the usual HTML, which is basic web language, then it may not always be easily read. For instance, a PDF document might only be accessed as a link and not easily found as an HTML web page.
Another link the spiders will not go to is any link that has a question mark. Spiders are informed not to go to links with a question mark, as they could be traps set to ensnare the spiders. The link could also contain badly written script and so may not be stored on the usual databases. These links will only be available when searching databases on the invisible web.
Search engines are being updated with sophisticated search techniques to access the invisible web. It is thought that in time, the invisible web will become more easily visible. Until then, it is a case of using brainpower to question the search engines and surfing links to find information.