In the space of a single decade, internet search has changed the way we look out onto the world beyond recognition. Google has become our binoculars and our window out on the net. With that blinking cursor on our internet search box only ever a button away, and ever ready to unleash a geyser of electronic information at the touch of a button, no longer do we have to go into any encounter wholly unprepared. Sometimes we rely on it too much. A survey of one hundred American business recruiters in 2006 revealed that four-fifths of them now resort to search engines when hiring new staff. More than a third admitted that they had rejected a candidate on the basis of unverified information they had come across on the net.

While search is good, then, it is far from perfect. While writing this article, I googled a total of three people. First there was Giulia Ricci, a promising young artist who I was due to meet for coffee later on in the day; then I typed in Matthew Taylor, the former think-tanker and director of the RSA because I wanted to read his blog. The last person I looked up was Shaun Phillips, about two minutes after he rang me up to commission the piece. First place in the list of blue links Google sent my way for “Giulia Ricci” came an Italian porn star who looked nothing like the woman I was to meet later on that day. Among the hits which arrived for “Matthew Taylor”, Google was touting around information on a pop singer, a Liberal Democrat MP, a footballer and a Guardian journalist. Worse luck, the “Shaun Phillips” who works at The Times had been ousted by an American football player for the San Diego Chargers. Had I been “feeling lucky”, in Google’s whacky terminology, and plumped for the first hits which came my way on each, I would have come away with information about an Italian porn star, an American football player and a Liberal Democrat MP. This frustrating gap which exists between where we want to go on the net and what we get from Google is leading to some interesting new challenges to Google’s overwhelming superiority in the search business, and throwing up fruitful ideas about the future of search.

Ten years ago, Google arrived in the search business like a breath of fresh air. Unlike the big beasts which came before it, Google was the first company to really understand that we wanted to navigate our own way around the net rather than stare at a showy internet “portal”, and it rapidly became our trusted guide. The company’s celebrated mission-statement committed it to organising the world’s information and rendering it accessible and useful. That, however, was easier said than done. Google’s worthy ambition of digitising all the world’s books, for example, is a complex and daunting project which will likely take many decades to complete, even it if isn’t scuppered before then by the worries of everyone from authors to advertising agencies. In the meantime, what Google does so well with its search technology is to bring everyone within range of the net into a fluid and chaotic global conversation – one which is constantly being refreshed by the questions and opinions of each and everyone of them.

Think about what Google does. Like most search engines, it works by crawling the web, taking regular digital snapshots of what’s out there, indexing it and rendering it accessible to anyone in search of anything. That, however, still leaves its engineers the job of ranking the information which Google has made it its business to serve up. Though the company protects its ranking technology as assiduously as Coca Cola protects the recipe for its distinctive sugary drink, its most essential ingredient is known to be its PageRank algorithm. PageRank stems from the clever idea of Google’s founders Larry Page and Sergey Brin that a good way of calculating the import of any web page was not via any objective criteria but by counting up how many people were pointing at it from their own websites and then measuring the weight and worth of all those pointing links – weighing up, in other words, the worth of a piece of information by seeing how many other people found it worthwhile. But that wasn’t all. As Google’s search technology became more sophisticated, it enabled its users to feed back into Google’s information loop their own opinion of the information which Google sent their way. Every time we choose from the list of hits which Google serves us up in response to our search, in other words, we are helping Google rank the information of our peers, and that information is in turn used to track what the best destinations on the web. When we use Google we are tapping into a conversation of millions of people out there on the net – canvassing what other people think and why they think it before we make our decision. Some of Google’s users will trust the first source of information that Google serves them up. Most, however, will take the trouble to browse through two or three. When Google decided to measure the worth of a piece of information by looking at how many other people found it worthwhile, it sowed into its operation a kind of feedback loop which helped traffic flow around the world wide web much more quickly and smoothly. As a result, it gobbled up about four fifths of the global search business and became one of the richest companies on earth. Google is now worth roughly one hundred billion dollars.

The open, collaborative nature of the way that Google uses us to organise its material works wonders at finding interesting nuggets of information, but it is far from ideal. Sometimes, for example, the signposts it sends our way are topsy-turvy. When I interviewed the inventor of the world wide web Tim Berners-Lee for my book, he bemoaned the way in which Google’s PageRank system and others which followed in its wake are becoming increasingly skewed by search engine “optimisation” firms who buy up links from an open market in order to route traffic in the direction of their clients. There can, of course, be pleasure in the chase. Taking wrong turnings on the net can lead us to bump into information we didn’t even know we were looking for, and can encourage our natural curiosity and thirst for discovery. In April 2007 a survey concluded that two-thirds of British internet users spent time ‘wilfing’  (‘what was I looking for?’) while hopping around on the internet. A quarter of those surveyed admitted to whiling away 30 per cent or more of their time on the internet in this way, the equivalent of spending one working day every fortnight lost in an electronic reverie. In the last five years, many of us have become more impatient about getting exactly where we want to go to on the net. While Google keeps tweaking its algorithm and the mass of material which is out there on the net has grown to almost infinite proportions, the fundamental technology which powers its searches hasn’t changed a great deal. Google makes us work for our information – it turns us into map-makers, tracing relationships and assembling connections in the electronic ether to see whether it adds up to anything we can rely on. It is all too easy to get lost.

*

So what are the alternatives to Google? Recently the company has been joined by a range of different search engines which claim to outdo it either on accuracy and speed or on both. At the austere end of the search business came Wolfram Alpha, launched last month and named after the British physicist Stephen Wolfram. Wolfram Alpha promises to be an “answer engine” for researchers, returning rock-solid data in response to statistical or factual queries by scanning only authoritative databases. It throws up impressive-looking spreadsheets for results, and can tell you a great deal about the properties of potassium or the performance of a publicly listed company. Wolfram is in its infancy – thus far the databases at Wolfram’s command are limited to scientific, political, economic and historical data – but it does offer a step towards the goal of what internet technicians like Berners-Lee call a “semantic web” where search engines are capable of understanding plain English and taking more precisely to our destination. Google is also feeling the pinch among those who want immediate access to the kind of electronic chatter or “buzz” at which its constantly bubbling page-rank algorithm has traditionally excelled. Last month Google’s Larry Page admitted that his company was falling behind in the race to publish immediate and “real-time” information to Twitter, the latest online social networking craze on the net. Whereas Google’s vast library of information can take whole days to index and to update, Twitter, which allows users to express themselves or forward snippets of news in 140 characters or less, allows anyone to search through its constantly rolling mass of tweets almost instantaneously.

Then there is Microsoft’s latest, seething attempt to make up ground in the rapidly expanding search business to Google. Bing, which was launched in the United States last week and is currently being reworked for the UK, reportedly stands for ‘Bing is not Google’ and takes its onomatopoeic name from the sound of something being found very quickly. It has its work cut out. Bing is designed to replace its MSN Live Search, which currently services only 4% of UK searches compared to Google’s mammoth 84% share of the market. It boasts intuitive new technology which claims to stand a better chance of find what its users really want rather than bogging them down in links. Bing promises to be a “decision engine”, giving its users access to targeted information rather than a list of random web pages and the opportunity to see previews of web pages or videos with having to click through into them. In the hundred million dollar ad campaign which accompanied its launch, Microsoft made much of its own research which suggests that 30% of searchers abandon their searches dissatisfied with what they have returned. In one of its ads intended to mimic the experience of using Google, a woman goes into a clinic saying: “I’m having this back pain.” The receptionist says: “Back-packing? Back-to-school? Johann Sebastian Bach?”

Only time will tell whether Bing, Wolfram Alpha and Twitter can loosen Google’s iron hold on the search business. There is good reason to doubt some of their claims: the world of internet search is littered with cocky also-rans, and all Microsoft’s previous attempts to refresh its search technology have ended in tears.” In a “blind test” site set up by a Microsoft employee last weekend which showed results from all Google, Bing and Yahoo! and invited passers-by to pick the best, Bing sprinted into an early lead before falling slightly behind Google a day later. In any case, Microsoft is resigned to playing a long game. “The major search engines were developed over a decade ago, and we believe the category is still in its infancy,” says its head of UK research Paul Stoddart at the end of last month. “It’s important to challenge and evolve the search market … there is much more that people can and should expect.” Google has one of the most powerful brands in the world at its disposal, and it will be not to be easy to prise internet search away from a brand whose very name is synonymous with the activity of searching the net. One thing is clear, however – the technology is improving and the whiff of change is in the air. The problem for its competitors is that, since the company has been busy spending its money to accumulate more reliable databases than anyone else thanks to Google Books, Google Maps, Google Earth and all its other projects, the next Google is very likely to come from within Google itself.