In-Depth

Sorting Out Microsoft's Mixed-Up Enterprise Search Strategy

Redmond's hodgepodge of search solutions lacks a cohesive strategy, observers say, leaving enterprises confused and competitors primed to take advantage.

• By Paul Korzeniowski
• 07/05/2011

Choices, anyone? For corporations looking for an enterprise search solution, Microsoft offers plenty of options: Bing, SharePoint Foundation 2010 Search, Search Server Express, Search Server 2010, SharePoint Server 2010 and FAST Search Server 2010 for SharePoint. The various search alternatives vary in capabilities, sophistication and price, so there should be something for just about every enterprise.

However, the Microsoft strategy can leave customers bewildered. The various products are largely autonomous, so it may not be easy to move from one to another. In addition, there are conflicting reports about which of the search engines Microsoft considers strategic, so there's a possibility that companies may standardize on solutions that will eventually lose their luster and maybe even be phased out.

Microsoft is not the only vendor that has struggled to develop a coherent enterprise search strategy. While vendors have been able to crack the consumer search engine code fairly easily, they haven't been as successful in the enterprise search market. More than a dozen vendors -- including Attivio Inc., Autonomy Corp., Coveo Solutions Inc., Endeca Technologies Inc., Exalead, Google Inc., IBM Corp., Oracle Corp., Recommind Inc., Thunderstone Software LLC and Vivisimo Inc. -- have tried to crack this market, with varying levels of success.

To gain share, Microsoft is following some of its traditional practices, focusing on cutting pricing and forging tight connections between its search system and other Microsoft products. This approach could resonate with existing customers. However, the company appears to be a step behind competitors in understanding search market dynamics, and Microsoft eventually may need to develop a simpler, more cohesive strategy.

Searching for an Enterprise Hit
While search has become a commonly used and helpful feature among consumers, it has not been as popular in business for a few reasons. One difference is the type of data cataloged. With Internet search, information is restricted mainly to Web pages in HTML format. In enterprise searches, data is stored in a variety of formats beyond HTML -- such as word processing documents, database-management systems and image-processing systems -- so an enterprise search system has to be able to collect and catalogue all of those items.

Also, consumers are less discriminating than companies. The former usually start out with a vague idea about what they're looking for and are willing to wade through a series of links to find something like it. Business users have more precise requirements; for instance, in large corporations, employees often need to sift through millions of items just to pull out a record or two. They're also less willing to spend time ferreting through information, so their searches often seem fruitless and lead to frustration. In fact, after a few disappointments, employees will often stop using the company search system.

In addition, delivering relevant business results is difficult because each company's content is unique. Taking data from various sources, consolidating it, and then presenting the most relevant items to employees who have a wide variety of interests and needs requires time, energy and intelligence.

Lacking Security Features
Finally, consumer and enterprise search systems handle content differently. Enterprise search systems need to make distinctions when delivering sensitive data, such as an employee's annual pay or Social Security number, from non-sensitive data, such as the dates when paychecks are cut. Enterprise search systems need to be designed so they make such distinctions and map the results to company security models and user permission levels.

Consequently, enterprise search tools haven't garnered the level of acceptance seen with consumer search systems. While Google has the most name recognition in the enterprise search market, no vendor has locked down the market's top spot. Because millions of U.S. businesses (especially small and midsize businesses, or SMBs) haven't purchased an enterprise search product, there's a possibility of significant market shifts in the coming years.

To try and grab a top spot, Microsoft has pulled together a handful of solutions. SharePoint Foundation 2010 offers rudimentary search functions within SharePoint information and is available for free. Search Server Express, Search Server 2010 and SharePoint Server 2010 all work from the same codebase, with limitations imposed more by licensing practices than technical differences. Express is also free, runs on one server and scales from 300,000 to 10 million documents, depending on which back-end Microsoft SQL Server database is used.

Search Server and SharePoint Server both scale to roughly 100 million documents and work with multiple servers. Search Server adds integration of social-content searches, such as expert finder and social proximity, to the capabilities found in SharePoint Server. FAST Search Server 2010 for SharePoint stems from the Microsoft purchase of Norwegian supplier Fast Search & Transfer ASA in January 2008 for $1.2 billion. The product was designed for large, complex searches: It can juggle multiple user profiles, index data from a variety of content sources and handle extremely large data sets. The latest release ties the high-end search system product more tightly into SharePoint.

Lowering Product Pricing
To gain market share, Microsoft is following its long, well-established pattern of finding ways to commoditize products in markets first developed by other companies. Historically search systems have been large, complex, costly and difficult to implement. Consequently, their use has been largely limited to large enterprises. Following Google's lead, Redmond's approach has been to lower the cost from six or seven figures to four or five figures and push search down to enterprise departments and SMBs.

"Microsoft has been very aggressive in lowering its product pricing, especially with FAST," says Leslie Owens, senior analyst at Forrester Research Inc. The vendor's goal is to seed the market with its free tools and entice companies to eventually migrate to more full-featured (read: costlier) search solutions. The company has also tied its search systems closely to SharePoint, which has become the central repository of corporate information for many businesses.

Microsoft has also opened up its solutions to a variety of information sources. Not surprisingly, the company started with its suite of desktop applications, including Microsoft Word, Excel, PowerPoint and Access. In addition, customers want to search video, audio and image files, so Microsoft has designed connectors for file shares, EMC Documentum, IBM Lotus Notes and Filenet repositories, and Microsoft Exchange and SharePoint. In addition, the software giant has taken a leading role -- along with Amazon.com Inc., EMC Corp., IBM and SAP AG -- in developing the OpenSearch specification and including in its products the federated search connectors based on the standard.

Google Gets the Boot
The ability to connect to multiple data sources appealed to the Mississippi Department of Transportation (MDOT), which is responsible for maintaining the state's highways and transportation infrastructure. The state agency, which has six districts, 100 field offices, more than 3,000 employees and an annual budget of more than $1 billion, had been using a Google search appliance to provide its employees with access to information. "The Google system didn't have interfaces to some of the information sources that we used," notes Ken Slay, enterprise content management project manager at MDOT.

In the summer of 2009, the agency evaluated products from all of the major search vendors. "FAST was able to connect to all our information sources," explains Slay. The agency has moved 3 million of its 13 million documents into the repository and plans to have them all in place by the end of 2011. However, the benefits from making the change are already apparent: "One manager told us that he found a needed document in minutes rather than days," says Slay.

Despite that positive experience, the future of FAST is cloudy. "When the acquisition was made, Microsoft's plan seemed to be to focus its development and marketing efforts on FAST," says Lynda Moulton, a senior analyst for enterprise search, knowledge management and information technologies at Gilbane Group Inc. In fact, a Microsoft spokesperson says that the company "is committed to integrating the best-of-breed FAST search engine with the richness of SharePoint and making FAST Search our platform across all our enterprise search products."

Lacking Needed Juice
However, FAST seems to have received little support from Microsoft since the purchase. "The individuals who built FAST left the company soon after it was acquired," Moulton says. While porting the solution to SharePoint was done fairly quickly, development efforts to add functionality to the product seem to be getting lost inside the large Microsoft bureaucracy.

Instead, the vendor has put a lot of time, effort and marketing dollars into its Bing search engine. Earlier this year, the company spent more than $100 million dollars in a campaign featuring rap icon Jay-Z designed to raise that search engine's profile. It also seems that most of the company's innovation in the search area is coming from the Bing team.

"Perhaps some of the new technology developed for Bing will eventually be integrated into FAST," notes Moulton.

While Microsoft lollygags, competitors are moving their search engines in new directions. One new approach is to sell to OEMs rather than end users, so search becomes a function in applications rather than an application itself. Attivio and Thunderstone Software are moving in that direction. Suppliers are also using search to build out other horizontal and vertical applications. Autonomy has positioned its system for use in governance and marketing applications. Coveo tweaked its product so it now offers search solutions for customer service and call center, e-mail and extranet applications. Recommind is touting its system as an e-discovery tool.

Microsoft was slow to recognize the importance of enterprise search. In 2008, the company tried to make up ground by buying its way into the marketplace. Recently, it has taken a shotgun approach to delivering needed functionality. But Redmond may need to refine its focus in order to gain ground in the highly competitive and rapidly evolving enterprise search market.