7.10 Creating a Search Engine | PHP Developers Cookbook (2nd Edition)

One of the most useful features to have on a site is a search engine, so I thought that to end our discussions of both files and directories, I would roll our own search engine. The search engine has two main parts : the indexer ( octopus .php and finalize.php) and the search engine itself (voyager.php).

The indexer first searches the directory tree from the base directory (specified in the configuration file, config.txt) for all files with file endings that match those in specified in the filter option of the configuration file. We then index the meta tags of each file that is found in the search. Finally, we write those results to the browser and to a temporary file, giving the user a chance to delete the current index and re-index the site at a later time. If the user decides to go through with the index, we add the index to the SQL database ” specifically , into the following table:

 CREATE TABLE sites_table (   id INT(4) PRIMARY KEY AUTO_INCREMENT,   filename VARCHAR (70),   keywords TEXT,   description TEXT );

After the results are added to the SQL database, the indexing portion of the site is done. We now move to the search portion of the script.

Most of the querying is done in SQL. We search the filename, description, and keywords for any occurrence of the search query. If we find a match, we print it out; otherwise , we tell the user that his search yielded zero results.

Now for the source code:

config.txt

 ; This is the configuration file for the search engine ; the semi-colon is a comment and a colon sets apart the ; name value pairs dbtype: mysql      ; SQL Server type Host: localhost    ; SQL server location User: username     ; SQL username Password: password ; SQL password database_name: designmultimedia1 ; Name of database for searchengine                                  ; information filters: html,htm,php,php3,php4 ;Allowed File Endings for                                 ;indexing separated by a comma saveresults: yes ; Save the results of each successful indexing basedir: /home/designmm/www  ;Base directory where files are held

octopus.php

 <?php include_once 'File/Find.php'; include_once 'LoadConfig.php'; $config = LoadConfig ('config.txt'); # Build Regex $regex = "/$config[filters]/i"; $fs = new File_Find; $files = $fs->search ($regex, trim ($config['basedir']), 'perl'); while ($file = array_pop ($files)) {     $meta_tags = get_meta_tags ($file);     $indexed .= "$file  " . $meta_tags[keywords] . '  ';     $indexed .= str_replace ("\n", ' ', $meta_tags[description]) . "\n"; } if (strtolower ($config['saveresults']) == 'yes' or     $config['saveresults'] == '1') {     touch ('savefile'); # Create if doesn't exist     $n = (int)join('', file ('savefile'));     $fp = @fopen ('savefile','w') or die("Cannot Open savefile");     fputs ($fp, ++$n);     fclose($fp) or die("Cannot Close savefile");     $filename = 'octopus_index' . $n . '.txt';     $fp = @fopen ($filename, 'w') or die ("Cannot Open $filename");     fwrite ($fp, $indexed);     @fclose ($fp) or die("Cannot Close $filename");     confirm ($indexed); } else {     $filename = 'octopus_index.txt';     $fp = @fopen ($filename,'w') or die("Cannot Open $filename");     fwrite($fp, $indexed);     @fclose($fp) or die("Cannot Close $filename");     confirm ($indexed); } function confirm() {     global $filename, $indexed; ?> <html> <head>     <title> Files Indexed Successfully </title> <body> The Following files have been indexed:<br><br> <?php $indexed = explode ("\n", $indexed); while(list (, $file) = each ($indexed)) {     $file = str_replace ('', '<br>', $file);     print "$file\n<br><br>\n"; } ?> <br><br> <form action='finalize.php' method='get'> <input type='hidden' name='filename' value='<?php echo $filename; ?>'> <input type='radio' name='finalize' value='yes'> Finalize Index<br> <input type='radio' name='finalize' value='no'> Redo Index<br> <input type='submit' value='Make It So!'> </form> </body> </html> <?php } ?>

finalize.php

 <?php include_once 'DB.php'; include_once 'LoadConfig.php'; $config = LoadConfig ('config.txt'); if ($finalize == 'yes') {     $dbconn = array ('phptype'  => $config[dbtype],                      'username' => $config[User],                      'password' => $config[Password],                      'database' => $config[database_name],                      'hostspec' => $config[Host]);     $dbh = DB::connect($dbconn);     if (DB::isError ($dbh)) {         die (sprintf ('Error [%d]: %s',                       $dbh->getCode(), $dbh->getMessage()));     }     $fcontents = file ($filename);     while (list ($line) = each($fcontents)) {         list ($fname, $keywords, $desc) = explode ('', $line);         $fname = trim ($fname);         $keywords = trim ($keywords);         $desc = trim ($desc);         $stmt = 'INSERT INTO sites_table filename, keywords, description';         $stmt .= " VALUES ('$fname', '$keywords', '$desc')";         $dbh->query ($stmt);     }     $dbh->close();     print "Results Finalized"; } elseif ($finalize == 'no') {     unlink($filename)       or die("Cannot Delete $filename");     if ($saveresults == 'yes' or $saveresults == '1') {         $n = (int)join ('', file ('savefile'));         $fp = @fopen ('savefile','w') or die ("Cannot Open savefile");         fputs ($fp, --$n);         @fclose ($fp) or die ("Cannot Close savefile");     }     print "Results Discarded"; } else {     die ("Invalid Option for finalize"); } ?>

voyager.php

 <?php include_once 'DB.php'; include_once 'LoadConfig.php'; $config = LoadConfig ('config.txt'); // Submitted data is $query -- Users search term $dbconn = array ('phptype'  => $config[dbtype],                  'username' => $config[User],                  'password' => $config[Password],                  'database' => $config[database_name],                  'hostspec' => $config[Host]); $dbh = DB::connect($dbconn); if (DB::isError ($dbh)) {     die (sprintf ('Error [%d]: %s',         $dbh->getCode(), $dbh->getMessage())); } $stmt = 'SELECT * FROM sites_table '; $stmt .= "WHERE keywords LIKE '%$query%' OR description LIKE '%$query%'"; $stmt .= " OR filename LIKE '%$query%'"; $sth = $dbh->query ($stmt); if (DB::isError ($dbh)) {     die (sprintf ('Error [%d]: %s',          $dbh->getCode(), $dbh->getMessage())); } $count = $sth->numRows(); $header = ($count > 0) ?             "There were $count results to your query" :             'There were no results for your query, please try again'; function result_set() {     global $sth;     while ($row = $sth->fetchRow()) {     ?>     <a href='<?php echo $row[filename]; ?>'> <?php echo $row[filename]; ?> </a>     <br>     <dir><?php echo $row[description]; ?></dir><br><br>     <?php } } $dbh->close(); ?> <html> <head>     <title>         <?php echo $header; ?>     </title> </head> <body> <h1> <?php echo $header; ?> </h1> <br><br> <?php if ($count) {     result_set(); } else {     echo $header; } ?> <form action='<?php echo $PHP_SELF; ?>' method='GET'> Search Terms: <input type='text' name='query'> <input type='submit' value='Search!'> </form> </body> </html>

LoadConfig.php

 <?php function LoadConfig ($filename, $seperator=':', $comment=';') {     $regex = "/$comment.*/i";     $fp = @fopen ($filename, 'r');     if (!$fp) {         die ("Cannot open $filename");     }     while ($line = @fgets ($fp, 1024)) {         $line = preg_replace ($regex, "", $line);         list ($key, $val) = explode ($seperator, $line);         $key = trim ($key);         $val = trim ($val);         $config[$key] = $value;     }     @fclose ($fp);     return ($config); } ?>