A few weeks ago I decided to improve searcher in application I’m currently working on. My goal was to improve the quality of results. In this article I would like to show the situation I’ve had and the way I achieved my goal.

Let’s assume we have a bookstore and we want the searcher not only to search for particular titles and authors, but also to suggest books for user based on the kinds of books he checks.

So we have a bunch of tags, like “fantasy”, “s-f”, “historical”, but also “French”, “German”, “English”. Now, there a few ways we can use the tags user checks:

• join them with OR and show results with at least one checked tag
• join them with AND and show only books with all checked tags
• sort results by number of matched tags

I’ve come up with another solution. I thought that when user chooses both “French” and “English” he expects to see books tagged with (at least) one of this tags, but when he chooses “English” and “fantasy” he expects to see books tagged with both tags.

That’s why I’ve decided to divide tags into groups. So I’ve created another model, TypeTag, which I use for groupping. While searching, I wanted to join tags from the same group with OR, and join groups with AND:

(English or French) and (fantasy or historical)

Here comes the question - how to achieve such combination using Sphinx? Here is my idea:

First we need to have info about book’s tags and their groups in one string. We can store it in database, but I didn’t want to, so I’m just collecting it while running rake ‘ts:index’ task. Code uses great MySQL’s method (I’m not sure it’s available in other dbs) group_concat:

indexes ["GROUP_CONCAT(CONCAT('g',IFNULL(tags.tag_group_id,'0'),'t',tags.id) SEPARATOR ' ')"], :as => :tags_code

Now info about tags looks like that: “g10t11 g2t8” (if tag has not been marked with any group it is marked as group no. 0, but it’s highly recommended that all tags should belong to a group)

Second thing, view. It’s simple, but it requires using check_box_tag, at least I couldn’t achieve this with any higher-level helper.

= check_box_tag "search[tags][#{t.type_tag ? t.type_tag.id : '0'}][]", t.id, false

The last thing is to implement joining search[tags] params for Sphinx

sphinx_scope(:tagged) {|groups|
    sentence = []
    groups.each do |group|
      words = []
      group[1].each do |tag|
        words << "g#{group[0]}t#{tag}"
      end
      words = "( " + words.join(" | ") + " )"
      sentence << words
    end
    sentence = sentence.to_a.join(" & ")
    {:conditions => {:tags_code => sentence}}
}

Now, we just need to merge scopes.

search = Search.new(params[:search])
(...)
search.scopes.merge!({:tagged => search[:tags]}) if search[:tags]

Done! Now our searcher is much more precise!