I have a film database that contains information about a film called Yes, We're Open.
When searching the database, I'm having an issue wherein a search for "yes we're open" returns another title that has the words "we're" and "open" but not "yes" in its description, even though I require all words in boolean mode (i.e. "yes we\'re open" is translated to '+yes +we\'re +open' before it's sent as a query).
I assumed this was because "yes" is in the built-in stopwords list. However, when I set ft_stopword_file = "", restart mysql, and then repair table [tablename] quick the table that i'm searching, I get NO results on a search for "yes we're open". I've included my my.cnf below. This is MySQL version 5.0.22. Any ideas?
[mysqld]
query-cache-type = 1
query-cache-size = 8M
max_allowed_packet=500M
ft_min_word_len=2
ft_stopword_file = ""
[myisamchk]
ft_min_word_len=2
set-variable=local-infile=0
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
# old_passwords=1
skip-bdb
set-variable = innodb_buffer_pool_size=2M
set-variable = innodb_additional_mem_pool_size=500K
set-variable = innodb_log_buffer_size=500K
set-variable = innodb_thread_concurrency=2
[mysql.server]
user=mysql
basedir=/var/lib
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
skip-bdb
set-variable = innodb_buffer_pool_size=2M
set-variable = innodb_additional_mem_pool_size=500K
set-variable = innodb_log_buffer_size=500K
set-variable = innodb_thread_concurrency=2
EDIT: Here are some sample queries:
#1 - With Built-in Stopword file
mysql> SHOW VARIABLES LIKE 'ft_%';
+--------------------------+----------------+
| Variable_name | Value |
+--------------------------+----------------+
| ft_boolean_syntax | + -><()~*:""&| |
| ft_max_word_len | 84 |
| ft_min_word_len | 2 |
| ft_query_expansion_limit | 20 |
| ft_stopword_file | (built-in) |
+--------------------------+----------------+
5 rows in set (0.00 sec)
mysql> SELECT title, MATCH(title,description,genre,country) AGAINST (' +yes +we\'re +open' IN BOOLEAN MODE) as title_description_genre_country_score FROM `films` WHERE MATCH(title,description,genre,country) AGAINST (' +yes +we\'re +open' IN BOOLEAN MODE) AND `hidden` <> '1' ORDER BY `title_description_genre_country_score` DESC ;
+-----------------+---------------------------------------+
| title | title_description_genre_country_score |
+-----------------+---------------------------------------+
| Yes, We?re Open | 1 |
| Present/Future | 1 |
+-----------------+---------------------------------------+
2 rows in set (0.00 sec)
.... then edit my.cnf, add ft_stopword_file="".....
#2 With no stopword file
mysql> SHOW VARIABLES LIKE 'ft_%';
+--------------------------+----------------+
| Variable_name | Value |
+--------------------------+----------------+
| ft_boolean_syntax | + -><()~*:""&| |
| ft_max_word_len | 84 |
| ft_min_word_len | 2 |
| ft_query_expansion_limit | 20 |
| ft_stopword_file | |
+--------------------------+----------------+
5 rows in set (0.00 sec)
mysql> REPAIR TABLE `films` QUICK;
+-------------------------+--------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+-------------------------+--------+----------+----------+
| db.films | repair | status | OK |
+-------------------------+--------+----------+----------+
1 row in set (0.14 sec)
mysql> SELECT title, MATCH(title,description,genre,country) AGAINST (' +yes +we\'re +open' IN BOOLEAN MODE) as title_description_genre_country_score FROM `films` WHERE MATCH(title,description,genre,country) AGAINST (' +yes +we\'re +open' IN BOOLEAN MODE) AND `hidden` <> '1' ORDER BY `title_description_genre_country_score` DESC ;
Empty set (0.00 sec)
EDIT #2: Create table:
mysql> SHOW CREATE TABLE db.films\G;
*************************** 1. row ***************************
Table: films
Create Table: CREATE TABLE `films` (
`id` varchar(8) NOT NULL default '',
`title` varchar(255) default NULL,
`hidden` tinyint(1) default '0',
`featured` tinyint(1) default NULL,
`type` varchar(255) default NULL,
`subtype` varchar(255) default NULL,
`summary` text,
`description` text,
`image_url` varchar(255) default NULL,
`trailer_url` varchar(255) default NULL,
`slug` varchar(255) default NULL,
`category` varchar(255) default NULL,
`parent` varchar(255) default NULL,
`related` varchar(255) default NULL,
`sponsor` varchar(255) default NULL,
`genre` varchar(255) default NULL,
`country` varchar(255) default NULL,
`copresenters` varchar(255) default NULL,
`original_title` varchar(255) default NULL,
`director` varchar(255) default NULL,
`executive_producer` varchar(255) default NULL,
`producer` varchar(255) default NULL,
`cinematographer` varchar(255) default NULL,
`writer` varchar(255) default NULL,
`editor` varchar(255) default NULL,
`sound` varchar(255) default NULL,
`cast` varchar(255) default NULL,
`language` varchar(255) default NULL,
`trt` varchar(255) default NULL,
`year` varchar(255) default NULL,
`subtitles` varchar(255) default NULL,
`format` varchar(255) default NULL,
`color` varchar(255) default NULL,
`premiere_status` varchar(255) default NULL,
PRIMARY KEY (`id`),
KEY `id` (`id`),
KEY `type` (`type`),
KEY `subtype` (`subtype`),
KEY `slug` (`slug`),
KEY `category` (`category`),
KEY `parent` (`parent`),
KEY `hidden` (`hidden`),
KEY `featured` (`featured`),
KEY `copresenters` (`copresenters`),
KEY `original_title` (`original_title`),
KEY `director` (`director`),
KEY `executive_producer` (`executive_producer`),
KEY `producer` (`producer`),
KEY `cinematographer` (`cinematographer`),
KEY `writer` (`writer`),
KEY `editor` (`editor`),
KEY `sound` (`sound`),
KEY `cast` (`cast`),
KEY `language` (`language`),
KEY `trt` (`trt`),
KEY `year` (`year`),
KEY `subtitles` (`subtitles`),
KEY `format` (`format`),
KEY `color` (`color`),
KEY `premiere_status` (`premiere_status`),
FULLTEXT KEY `title` (`title`),
FULLTEXT KEY `summary` (`summary`),
FULLTEXT KEY `description` (`description`),
FULLTEXT KEY `genre` (`genre`),
FULLTEXT KEY `country` (`country`),
FULLTEXT KEY `title,description` (`title`,`description`),
FULLTEXT KEY `title,description,genre,country` (`title`,`description`,`genre`,`country`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8
1 row in set (0.00 sec)
Setting stopwords.txt variable to the empty string ('') disables all stopword filtering. (Remove some words which you don't require...suggestion..)
Indexes of FULL TEXT SEARCH which was created by you needs to be destroy and create new one.
FULLTEXT indexes must be rebuilt after changing this variable or the contents of the stopword file.
Then Use REPAIR TABLE tbl_name QUICK.
InnoDB doesn't support table repairs (you can see a note when you do REPAIR TABLE tbl_name QUICK.)
The only solution that I have dound is to change the engine to MyISAM, although that might decrease READ or WRITE performance.
How to disable fulltext stopwords in MySQL:
In my.ini text file (MySQL) :
ft_stopword_file = "" or link an empty file "empty_stopwords.txt"
ft_min_word_len = 2
// set your minimum length, but be aware that shorter words (3,2) will increase the query time dramatically, especially if the fulltext indexed column fields are large.
Save the file, restart the server.
The next step should be to repair the indexes with this query:
REPAIR TABLE tbl_name QUICK.
However, this will not work if you table is using InnoDB storage engine. You will have to change it to MyISAM :
ALTER TABLE t1 ENGINE = MyISAM;
So, once again:
1. Edit my.ini file and save
2. Restart your server (this cannot be done dynamically)
3. Change the table engine (if needed) ALTER TABLE tbl_name ENGINE = MyISAM;
4. Perform repair REPAIR TABLE tbl_name QUICK.
Be aware that InnoDB and MyISAM have their speed differences. One read faster, other writes faster ( read more about that on the internet )
来源：https://stackoverflow.com/questions/9300600/no-results-after-removing-mysql-ft-stopword-file