Chapter 5. Storing mnoGoSearch data

Table of Contents
SQL storage types
Cache mode storage
mnoGoSearch performance issues
Oracle notes
IBM DB2 notes

SQL storage types

Various modes of words storage

The different modes of word storage currently supported by mnoGoSearch are: "single", "multi" and "blob". Default mode is "single". The mode can be selected using the DBMode part of DBAddr command in both indexer.conf and search.htm files.

DBAddr mysql://localhost/test/?DBMode=single
DBAddr mysql://localhost/test/?DBMode=multi
DBAddr mysql://localhost/test/?DBMode=blob

Storage mode - single

When "single" is specified, all words are stored in a single table of structure (url_id,word,weight), where url_id is the ID of the document which is referenced by rec_id field in "url" table. Word has the variable char(32) SQL type. Each appearance of the same word in a document produces a separate record in the table.

Storage mode - multi

If "multi" is selected, words are located in 256 separate tables using hash function for distribution. Structures of these tables are almost the same with "single" mode, but all word appearances are grouped into a single binary array, instead of producing multiple records. This fact makes "multi" mode much faster comparing with "single" mode.

Storage mode - blob

If "blob" is selected, words are located in a single table of structure (word, secno, intag), where intag is a binary array of coordinates. All word appearances for the current section are grouped into a single binary array. This mode is highly optimized for search. Indexing is supported since 3.2.36. You should index your data with "single", "multi" or "blob" mode and then run "indexer -Eblob" to prepare the database for searching.

Since 3.2.36 additional "indexer -Erewriteurl" parameter is available. When indexer is invoked with this parametr it rewrites URL data for DBMode=blob. It's useful for very quick rewrite of URL data after adding "Deflate=yes", without touching word information.

DBMode=blob is the fastest mode currently available in mnogosearch for both purposes: indexing and searching. DBMode=blob is know to work fine with DB2, Mimer, MSSQL, MySQL, PostgreSQL, Oracle, Sybase, Firebird/Interbase. It's currently not supported with SQLite.


# indexer.conf
DBAddr mysql://localhost/test/?DBMode=blob

# search.htm
DBAddr mysql://localhost/test/?DBMode=blob

Substring search notes

"single", "multi" and "blob" modes support substring search. An SQL query containing a LIKE predicate is executed internally in order to do substring search.