One search box, nineteen indices, fuzzy matching switched off

The problem: staff already know what they are looking for

Almost everything written about search is written for people who do not know what they want. Someone browsing a shop needs recall, forgiveness and suggestions — cast a wide net and let them wander through it.

An admin console is the opposite. A studio’s front-desk staff know exactly what they are after. They have a member’s name on a sticky note, or a transaction id copied out of a support email, or half a product name they are about to finish typing. They want one record, they want it first, and if it is not first they will type three more characters rather than scroll.

That inverts most of the defaults. Fuzzy matching stops being generosity and becomes noise. Ranking by popularity is meaningless when the target is one specific person. Meanwhile a problem appears that consumer search does not have: the thing being looked for could be any of nineteen kinds of record — a member, an instructor, a class, a package, an order, a transaction, a gift card, a penalty, a promotion, or one of the platform’s own configuration objects like a notification template, a workflow or a landing page — and the person typing should not have to say which.

So the design question was never “how do we match more things.” It was: how do we work out what the user has already told us, and then narrow as hard as possible.

This is the sixth write-up from this platform. It is the one with no commercial hook at all: nothing here converts better or bills more. It is entirely about judgement — which defaults to reject, where to spend simplicity, and which half of a search feature actually decides whether it scales.

Act one: classify the query before searching anything

The first thing this search does is not a search. It is a look at what was typed.

if @raw.empty?
  @kind = :blank
elsif (model = match_friendly_id(@raw))
  @kind = :friendly_id
elsif @raw.match?(UUID_PATTERN)
  @kind = :uuid
elsif @raw.match?(EMAIL_PATTERN)
  @kind = :email
elsif phone_shape?(@raw)
  @kind = :phone
else
  @kind = :text
end

Six kinds, in priority order, decided in one small value object before any index is touched. The kind then drives everything downstream: which searcher runs, and which exact-match rule applies to the results.

The highest-value branch is the first one. Every record on this platform carries a human-readable id with a type prefix, and those prefixes are unambiguous:

FRIENDLY_ID_PATTERNS = {
  /\AUP\d*\z/ => UserProfile,
  /\AUT\d*\z/ => UserTransaction,
  /\AOR\d*\z/ => Order,
  /\AGI\d*\z/ => GiftCardInstance,
  # ... twelve in total
}.freeze

So UP1234 is not a text query that happens to match a member. It is a statement that the user wants one specific member. A support agent pasting an id out of a ticket gets a single-index lookup instead of a fourteen-index fan-out, and the code knows it can skip ranking altogether, because either the id matches or it does not.

Normalisation is per-kind, and it is what makes the ranking work later. A phone number keeps only its digits, so (415) 555-2929 and 4155552929 are the same query. Text is lowercased so name comparison is case-insensitive. The class carries worked examples in its own comment, which is how it should be — the behaviour is legible without running it.

scored_hits, total_count = fetch_results(query)
ranked  = ResultRanker.new(query: query).call(scored_hits)
results = build_payloads(paginate(ranked))

ID_FRIENDLY_ID_ONLY_MODELS = Set[
  ResourceOffering, PackageInstance, RecurringUserPackage,
  ResourceReservation, RetailInstance, UserPenalty
].freeze

def models_to_search
  return ALL_SEARCHABLE_MODELS.dup if @types.blank?

  @types.map { |t| TYPE_TO_MODEL[t] }.compact.uniq
end

Act two: two arguments against the obvious setting

The relevance work in this subsystem is almost entirely subtractive. Two choices, both arguing against the default, and both with the reasoning written into the code rather than into a wiki page nobody will read.

Fuzzy matching is switched off

# NOTE: misspellings are disabled. Short query tokens like "mai" or "oi"
# otherwise fuzzy-match unrelated tokens via edit-distance 1, leaking
# irrelevant records (e.g., transactions whose notes contain "AI" or "of")
# into results. Typeahead UX does not need fuzziness; users self-correct
# as they type.

That is the most useful comment in the codebase, and the argument generalises well past this product:

Fuzzy matching is a recall mechanism, and a typeahead has no recall problem. The user is mid-word and can fix their own typing.

On a three-character token, edit distance 1 matches an enormous amount of nothing. Type mai and you start pulling in every transaction whose note contains “AI” or “of”. The setting is applied on every query path rather than just the main one, and turning it off is what lets the ranking stay simple two sections from now, because the candidate set arriving from Elasticsearch is already tight.

An order cannot be found by the name of the person who placed it

The per-model field boosts are where the second argument lives, and the comment above them states it directly: person records search across identity fields, and non-person records intentionally exclude name fields.

'Product'         => ['id^30', 'friendly_id^20', 'name^15', 'product_type^12'],
'Order'           => ['id^30', 'friendly_id^20', 'order_note^10'],
'UserTransaction' => ['id^30', 'friendly_id^20', 'notes^10'],

An Order document does contain the buyer’s full name. It is indexed. Order simply does not search it. Same for transactions, which index the client’s name and never query it.

That is the difference between searching a member’s name and getting one person, and searching a member’s name and getting one person buried under four hundred of her transactions. Both are defensible index designs. Only one is usable in a box that shows twenty-five rows. The data is available and the query deliberately declines to use it, which is the kind of decision that only looks obvious after somebody has made it.

Prefix matching that survives its own migrations

# Two-phase strategy:
#   - Try :word_start (prefix matching).
#   - If that returns no hits (typically because a field doesn't have
#     an autocomplete subfield in the current index mapping), fall back
#     to :word so the query still works during reindex/migrations.
def single_model_search(model)
  hits, total = run_query(model: model, match_type: :word_start)
  hits, total = run_query(model: model, match_type: :word) if hits.empty?
  [hits, total]
end

Prefix matching depends on autocomplete subfields existing in the index mapping, and during a mapping change they may not yet. A prefix query against a missing subfield returns nothing rather than failing — which is the worst available outcome, because search looks broken while every health check says the system is fine. The fallback trades relevance for availability for exactly as long as a reindex takes, and not a moment longer.

Act three: the reranker, which is smaller than the word suggests

Worth defining before describing, because “custom reranker” invites the wrong picture. There is no machine learning here. No learned weights, no recency, no popularity, no entity-type priority, no rescoring inside the search engine. It pins records that exactly match what the user typed to the top of the list, and leaves everything below in Elasticsearch’s own order. That is the entire mechanism, and its own comment explains why that is enough:

# Why exact-match-only:
#   With ES misspellings disabled (see FullTextSearcher), the candidate
#   set is already tightly relevant -- ES TF/IDF correctly orders prefix
#   matches. The only ranking signal ES can't produce on its own is
#   "this record EXACTLY matches what the user typed", which is the
#   single boost we add here.

The two acts before this one are what earn the simplicity. Because fuzziness is off and commerce records cannot match on names, what comes back is already relevant and TF/IDF orders it sensibly. The one thing Elasticsearch cannot know is that a document is not merely a good match but the thing the user asked for — somebody typing a complete email address is not browsing.

So exact matches are promoted to infinity and the list is sorted, with the record id as a stable tie-break:

scored_results.each do |entry|
  entry[1] = Float::INFINITY if exact_match?(entry)
end

The only magic number in the ranking layer is infinity. And it applies only where “exact” means something — the id paths are explicitly skipped, because there every hit is already an exact id match.

The behaviour is unit-tested in both directions. Exact email, phone and name matches are asserted to move to the front even when Elasticsearch scored them lower, and a prefix-only match is asserted not to be promoted. That negative test is what makes the positive ones mean anything.

Being precise about what this is not: the ranking rules are unit-tested, they are not measured. There is no evaluation set, no NDCG or MRR instrumentation and no A/B test behind any of this. What is proven is that the rules behave as described. Below the pinned rows, the ordering is Elasticsearch’s, assembled from independent per-index queries — deliberately controlled at the top, approximate underneath.

Act four: the half that actually decides whether it scales

The query side of a search is what gets demoed. The indexing side is what breaks.

Searchkick’s automatic callbacks are switched off on all nineteen models. Instead, a save enqueues an id and returns — and then two independent layers of deduplication do the real work.

Step 2 is thirty seconds of Redis documentation doing an enormous amount of work:

# Only one job scheduled per SCHEDULE_INTERVAL from callbacks.
set = conn.set(lock_key, "1", nx: true, ex: SCHEDULE_INTERVAL)
SearchIndex::ReindexBatchWorker.perform_in(SCHEDULE_INTERVAL.seconds) if set

def enqueue_delete(model_name, id)
  return if model_name.blank? || id.blank?

  redis do |conn|
    conn.srem(upsert_key(model_name), id.to_s)
    conn.sadd(delete_key(model_name), id.to_s)
  end
  schedule_batch_job
end

What a result is, and why history is keyed on records

Result payloads are self-sufficient — name, image, amount, state, plus the ids and slug needed to deep-link — so selecting a result never triggers a second request to find out what it was. Associations are preloaded per entity type per page rather than per record, which is the difference between one query and twenty-five.

Then there is the detail from the hero image, which is the smallest decision on this page and possibly the most instructive. Recent searches store the entity a staff member opened, not the string they typed, with a unique index on the account, the staff member and the record.

That choice is what makes the feature useful rather than decorative. Visiting the same member twice refreshes one row instead of appending a duplicate, so the list reads as “the records you have been working on” rather than a log of half-typed words. A background job keeps the most recent hundred per person. And remembered records render through the same payload builder as live search results, so a member you looked at yesterday and a member you just found look identical, and there is one place to change what a result card shows.

Stated precisely

Five things that a search page usually leaves vague, and that we would rather put in writing.

• It is not real-time, on purpose. Indexing trails a write by roughly ten to twenty seconds: up to ten seconds of deliberate batching, plus the worker run. That delay is not latency waiting to be optimised away — it is what buys the deduplication above.
• Fourteen Elasticsearch round-trips for an unfiltered text search. Filtering by type cuts it to one per type, and a typed id cuts it to one full stop.
• Deactivated people are excluded from the index; other records are not. Member profiles get index-time filtering, including the nice touch of excluding profiles that are merge pointers to another profile. That is where it matters most and it is also the only model that does it. Extending it to the rest is one method per model, and it is on the list.
• Six of the nineteen entities are id-only in text search. Classes, reservations, package instances, retail items, recurring packages and penalties are found by id or friendly id, not by name.
• The ranking rules are unit-tested; relevance is not measured. No evaluation set, no A/B test. What is proven is that the rules behave as described.

Results

One search box over nineteen entity types, with a business_account_id filter enforced on every query path, rebuilt from a thin 2024 first cut into its current shape across three stages in 2026 without ever changing the endpoint contract.

Test coverage runs to roughly 2,225 lines across 12 files, concentrated on the search orchestration and ranking rules, the payload builder, the batch worker and its scheduler, and recent searches — plus a real-Elasticsearch integration spec covering dependent reindexing and deletion. Search latency percentiles and per-staff usage figures are the client’s to publish rather than ours to estimate.

What we would do next

Match the permission model the rest of the API already has. Every list endpoint on this platform gates on a capability — clients need read_client, sales need read_sales. Search authorises on “is a staff member” and filters by tenant, which is a coarser rule than the endpoints it searches across. The plumbing is half built already: the command passes the acting user into the payload builder, which currently ignores it. Threading permissions through so a result set reflects what the viewer is allowed to see is the first thing we would build, and the single most valuable change on this list.

Extend index-time filtering past people. Member profiles get this right and are the only model that does. Deactivated staff, archived landing pages, deactivated workflows and cancelled penalties are all still findable. The mechanism exists; it is one method per model.

Point the mention picker at this pipeline. The @-mention autocomplete in the in-product assistant runs its own SQL LIKE query. The product therefore has two different definitions of “search for a person” — one with prefix matching, field weighting and exact-match promotion, one without. Consolidating onto types=User would delete code and make the two agree.

Surface partial failure instead of swallowing it. Per-model errors are caught and turned into zero results for that model, so an Elasticsearch problem can present as “no matches” rather than “something is wrong.” Results are already assembled per model, so reporting which ones failed is a response-shape change more than a rewrite.

Give the six id-only entities searchable names. Classes and reservations are the two staff most plausibly want to find by name, and they are id-only because they were batched for query efficiency. That trade is worth revisiting now the batching exists.

Fix the disaster-recovery path. The bulk reindex task lists fifteen of the nineteen models. A rebuild from scratch would quietly leave four indices empty, and nothing would report it.

Upgrade Elasticsearch. 6.8 is end of life, and the 250-second client timeout should come down to something that fails a request rather than holding a thread for four minutes.

If this sounds like your system

The shape is common in any product with an admin console: a search box that returns too much of the wrong thing, a reindex that falls over during imports, and a “recent searches” list nobody uses because it remembers keystrokes.

The transferable decisions are these. Read the query before you run it — most applications have typed identifiers, and a prefix that identifies a record type turns a broad fan-out into a single lookup for exactly the queries where precision matters most. Turn fuzzy matching off in a typeahead, because the user is mid-word and can correct themselves, and the tighter candidate set lets the rest of your ranking stay small. Decide which documents may match on a person’s name, and accept that indexing a field and querying it are separate decisions. Put deduplication in front of your indexing, not concurrency limits behind it — a Redis set plus a SET NX EX lock is a general answer to “my after_commit hook enqueues too many jobs,” and it has nothing to do with search. And key history on records rather than strings, because people remember who they were working on, not what they typed.

Search is where the orders and transactions the revenue ledger reports on become findable by the people who answer the phone about them. If you have a search box that everyone in your company has quietly stopped trusting, or an indexing pipeline that turns every bulk import into an afternoon, tell us about it — it is a problem with a known shape.