When you want to develop a search engine for a specific website, you usually don’t want to re-invent the wheel. There are a lots of engines already implemented, or on the way to be finished, and there are a high percentage of those that are under open source license. So, why don’t you have a look at them and try to integrate one on your site?
The most awarded open source search engine these days is called lucene, and it offers a JAVA API. There is also Perl and C ports of the engine. But i want to integrate it in a PHP site. So i’ve been looking at the options i had to use it from PHP, and most people have used a communication protocol between JAVA and PHP. JAVA is used to, by example, develop an XMLRPC Server that uses internally the lucene API and serves the responses through a xmlrpc interface, so that PHP can connect to it, extract the search results and display them.
PHP Architect has a good article on this integration at the June 2003 issue.