Google used to be a great compilation of web information, but now I can't tell if the first few pages of the million plus hits I get are there because they have the content I want or because of clever search optimization techniques. What's coming in technology to make my searches more productive?

Until recently, information searching on the web was largely linear and mechanical - the computer looked for exactly what you specified in your search, adjusted a little by search optimization (which may not be based on factors you considered important). We are headed (slowly) to a "semantic web," in which the computer can execute some of the nuanced search functions only humans could provide. This will require restructuring of information and a host of other changes to fully implement, so we are left with tweaking current technology.

For now, there are two easy approaches, depending on how sophisticated you want to go. First, use a segmented search engine like Yippy (formerly Clusty), which clusters search results. For example, if you look for "management" it will group search results into management systems, software, international, business, etc. categories to give you better direction than a long linear list.

Second, you can "go deep" by using a new Deep Web add-in to your browser. This features returns tag clouds that let you see the context of your results and the relative frequency of hits, and sorts answers by blogs, news, Wikipedia, etc. A Deep Web search provides a more complete view of what exists and enhances your perspective about how to improve your search.

Tip: Don't expect the browser to do all the work. It's as much about your search techniques and choice of keywords and phrases as it is about the technology. If you don't fully understand the question to which you seek answers, the Deep Web or any other technology will not help much.

© 2010 Institute of Management Consultants USA