Version 2 of 2

Introduction

Generated Aksbel book section. · Working · Aug 09, 2026 03:21 · saved by @mujirin

Introduction

Names look simple until a system must decide whether two of them refer to the same person.

A human may quickly see that “Mohamed Ali,” “Muhammad Aly,” “M. Ali,” and “Ali, Mohamed” might refer to the same person. A computer, if it uses exact equality, sees four different strings. The characters are not identical, the order changes, some letters differ, and one version hides part of the first name behind an initial. This is the beginning of the name matching problem.

In this book, name matching means the process of comparing two or more name records and deciding how likely it is that they refer to the same real-world entity. The entity is often a person, but the same ideas also apply to organizations, products, vessels, addresses, and other named things. In data management and machine learning, this broader task is closely related to record linkage, duplicate detection, and entity resolution: connecting records that belong to the same underlying entity even when the records are not written exactly the same way. The classical statistical framing of record linkage was developed by Fellegi and Sunter, whose work remains a foundation for thinking about match, non-match, and uncertain pairs in linked data systems (Fellegi & Sunter, 1969). Modern treatments of data matching still build on these ideas while adding indexing, string similarity, machine learning, and large-scale engineering methods (Christen, 2012).

This book focuses on fuzzy name matching. The word fuzzy does not mean careless or vague. It means that instead of treating a pair of names as only equal or not equal, we allow graded evidence. For example:

Name A Name B Exact equality More realistic judgment
John Smith John Smith equal almost certainly same written name
John Smith Jon Smith not equal probably similar
John Smith Smith, John not equal same components, different order
John Smith Jane Smith not equal partly similar, but likely different person
John Smith Wei Zhang not equal very dissimilar

Exact matching gives the same answer—“not equal”—to the last four rows. A practical matching system needs more nuance.

That nuance is the central theme of this book.

Why exact matching fails

A string is a sequence of characters, such as MARIA GARCIA or O'Connor. Exact string comparison asks whether two sequences of characters are identical in every position. This is useful when the data is clean and standardized. It fails when names come from real life.

Real names vary for many reasons. A person may write a full name in one form and an abbreviated name in another. Clerks may enter names differently. Databases may store accents, punctuation, or whitespace inconsistently. A name may be transliterated from Arabic, Chinese, Cyrillic, Greek, Hindi, Japanese, Korean, or another writing system into the Latin alphabet in more than one accepted way. A family name may appear before a given name in one source and after it in another. A middle name may be present, absent, shortened, or replaced by an initial.

Consider a customer database containing these records:

Record ID Name
101 Dr. Ana María López
284 Ana Maria Lopez
517 A. M. Lopez
920 Lopez, Ana M.
1113 Anna Lopes

A simple exact match would group none of them together. A good fuzzy name matching system should recognize that records 101, 284, 517, and 920 may be strong candidates for the same person, while record 1113 is more uncertain: it is similar, but the spelling differences may also indicate a different person.

This is why name matching is not merely a programming trick. It is a decision problem under uncertainty.

The core idea: similarity as evidence

A similarity score is a number that represents how alike two objects are according to some rule. In name matching, the objects are usually strings or parts of strings. A common convention is to use a score between 0 and 1:

  • 1 means maximally similar according to the chosen method.
  • 0 means minimally similar according to the chosen method.
  • Values between 0 and 1 represent partial similarity.

For example, a character-based method may give a high score to Steven and Stephen because only a small number of edits separates them. A token-based method may give a high score to John Michael Smith and Smith John Michael because the same name components appear in a different order. A phonetic method may give a high score to names that sound similar even when spelled differently.

No single similarity score captures every kind of name variation. Research comparing string distance methods for name matching has shown that different metrics behave differently depending on the error pattern, field type, and matching task (Cohen, Ravikumar, & Fienberg, 2003). That fact is important: this book will not teach one magic algorithm. It will teach a toolbox and a way to combine tools responsibly.

Where fuzzy logic enters

Fuzzy logic is a mathematical framework for reasoning with degrees of membership and degrees of truth. It was introduced by Lotfi A. Zadeh in his 1965 paper on fuzzy sets (Zadeh, 1965). In ordinary set theory, an item either belongs to a set or it does not. For example, if we define a set called “exactly equal names,” then John Smith and John Smith belong, while John Smith and Jon Smith do not.

A fuzzy set allows partial membership. Instead of saying only yes or no, we can say that a pair of names belongs to the fuzzy set “very similar names” with degree 0.92, or to the fuzzy set “weakly similar names” with degree 0.35.

This matters because name matching decisions often sound like human rules:

  • “If the last names are almost identical and the first names are compatible, treat the pair as a likely match.”
  • “If the surname is common and the first name only matches by initial, require more evidence.”
  • “If the names are similar but the date of birth conflicts, send the pair to review.”
  • “If the name score is moderate and the address score is high, classify it as a possible match.”

These rules are not purely binary. Words such as almost identical, compatible, moderate, high, and possible are graded concepts. Fuzzy logic gives us a disciplined way to convert those concepts into computable rules.

For example, suppose we compare two last names and get a raw similarity score of 0.87. A fuzzy membership function might interpret that score like this:

Fuzzy concept Membership value
low surname similarity 0.00
medium surname similarity 0.30
high surname similarity 0.85

The same raw score can participate in rules. A rule may say:

If first-name similarity is high and last-name similarity is high, then match confidence is high.

Another rule may say:

If first-name similarity is low but last-name similarity is high, then match confidence is uncertain.

This book will develop these ideas carefully. We will not assume that “fuzzy” automatically means correct. A fuzzy system still needs good normalization, suitable similarity functions, calibrated thresholds, representative test data, and monitoring after deployment.

Name matching as a machine learning problem

Although this book emphasizes fuzzy logic, it belongs naturally in the subject area of machine learning.

Machine learning is the study and practice of building systems that improve their behavior from data. In name matching, this often means learning from examples of record pairs labeled as match or non-match. A model may learn that surname similarity is highly informative in one dataset, that date of birth conflicts are strong negative evidence in another, or that initials should be treated differently depending on the application.

A simple supervised learning setup might look like this:

Feature Example value
first-name character similarity 0.91
surname character similarity 0.96
token-set similarity 1.00
phonetic surname agreement yes
date-of-birth agreement yes
address similarity 0.74
label match

Here, a feature is a measurable input used by a model. A label is the target answer supplied during training, such as match or non-match. A machine learning model can learn how features relate to labels.

However, machine learning does not remove the need for clear thinking. A model trained on biased or incomplete examples may perform poorly on names from underrepresented cultures or languages. A model may also be hard to explain. Fuzzy rule systems can be useful because they are often more interpretable: a reviewer can see which rule fired and why. Later chapters will show how fuzzy logic and machine learning can complement each other rather than compete.

The practical pipeline

A reliable name matching system is usually a pipeline: a sequence of steps where each step prepares evidence for the next one.

A simplified pipeline looks like this:

raw names
   ↓
normalization
   ↓
candidate generation / blocking
   ↓
similarity scoring
   ↓
fuzzy rules or machine learning model
   ↓
thresholding and review
   ↓
decision, audit, and monitoring

Each word in that pipeline will become familiar as the book progresses.

Normalization means converting text into a more consistent form before comparison. For example, José García, JOSE GARCIA, and Jose Garcia may be normalized by case folding, whitespace cleanup, and accent handling. Normalization must be done carefully because removing information can help in one context and harm in another.

Candidate generation, also called blocking or indexing, means reducing the number of pairs that must be compared. If a database has one million records, comparing every record with every other record would require roughly half a trillion pair comparisons. Practical entity resolution systems therefore use blocking methods to create smaller candidate sets, a standard concern in large-scale data matching work (Christen, 2012).

Similarity scoring means computing evidence: character similarity, token overlap, phonetic compatibility, nickname equivalence, or other signals.

Decisioning means turning evidence into action. A system may return:

  • match, when evidence is strong enough for automatic acceptance;
  • non-match, when evidence is weak enough for automatic rejection;
  • possible match, when the case should be reviewed by a human or by a stronger downstream process.

The third category is not a weakness. In many real applications, the gray zone is necessary. It prevents a system from pretending to be more certain than it is.

A small motivating example

Suppose we are building a deduplication system for customer records. We want to decide whether these two records describe the same person:

Field Record A Record B
first name Katherine Kate
middle name L. missing
last name O'Brien Obrien
date of birth 1988-04-12 1988-04-12
city Boston Boston

An exact name comparison fails because Katherine L. O'Brien is not identical to Kate Obrien.

A better system may reason as follows:

  1. O'Brien and Obrien are nearly identical after punctuation normalization.
  2. Katherine and Kate may be related through a nickname dictionary.
  3. The missing middle name in Record B should not be treated as a contradiction.
  4. The date of birth matches exactly.
  5. The city matches exactly.

A fuzzy rule might express the decision:

If surname similarity is high, first-name compatibility is high, and date of birth agreement is exact, then match confidence is very high.

This is not just string matching. It is evidence combination.

Now consider a different pair:

Field Record A Record B
first name Michael Michelle
last name Lee Lee
date of birth 1991-09-03 1994-02-18
city Chicago Chicago

The last name and city match, and the first names share some characters. But Lee is a common surname in many populations, and the dates of birth differ. A careful system should not overmatch simply because some fields are similar. This is one of the recurring lessons of the book: high similarity in one field can be misleading when other evidence contradicts it.

What this book will teach

The chapters follow the path from concept to production.

First, we define the name matching problem and study how names vary in the real world. This matters because algorithms are only useful when they are designed for the variation they will encounter.

Next, we build the mathematical and computational foundations: fuzzy sets, membership functions, string distances, token similarities, phonetic encodings, nickname dictionaries, and composite scoring.

Then we move from scoring to decision-making. We design fuzzy rules, choose thresholds, create gray zones, and evaluate performance using precision, recall, F1 score, false positive rate, false negative rate, ROC curves, precision-recall curves, and confusion matrices.

After that, we handle scale. Real systems cannot compare every possible pair in a large database. We will study blocking, sorted neighborhood methods, n-gram indexes, phonetic indexes, and approximate search.

Finally, we discuss machine learning, embeddings, multilingual matching, error analysis, deployment, monitoring, privacy, fairness, and end-to-end case studies.

The goal is not to memorize algorithm names. The goal is to learn how to design a reliable matching system.

What “reliable” means here

A reliable name matching system is not one that always says “match” when two names look similar. Reliability means the system behaves well for its intended purpose.

In a customer deduplication system, a false match may merge two different customers and corrupt account history. In a search system, a false match may be less harmful because the user can ignore irrelevant results. In sanctions screening or identity verification, a false positive can create serious inconvenience or harm, while a false negative can create legal, financial, or safety risk. The correct design depends on the cost of different errors.

This book will use two important terms often:

  • A false positive is a pair classified as a match when it is not truly a match.
  • A false negative is a pair classified as a non-match when it is truly a match.

For example, if Maria Santos and Maria Santos are two different people but the system merges them, that is a false positive. If Mohammed Al-Hassan and Muhammad Alhassan are the same person but the system rejects them as unrelated, that is a false negative.

Good systems measure both. They do not improve one blindly while ignoring the other.

How to think while reading

As you read, keep three questions in mind.

First: What kind of variation am I trying to handle?
A spelling correction method may help with typos but not with nicknames. A phonetic method may help with sound-alike names but not with reordered name components. A transliteration table may help in one language pair and fail in another.

Second: What evidence is strong, weak, missing, or contradictory?
A matching system should not treat every field equally in every situation. An exact date of birth may be strong evidence in some datasets. A common surname may be weak evidence. A missing middle name is not the same as a conflicting middle name.

Third: What decision will be made from the score?
A score is not the end of the process. A score may trigger automatic merge, manual review, search ranking, identity verification, fraud investigation, or no action. The downstream consequence determines how conservative the system should be.

If you keep these questions active, the technical chapters will feel less like a list of methods and more like a design language.

The central promise

By the end of this book, you should be able to look at two names and ask not only, “Are these strings similar?” but also:

  • Similar in what way?
  • Similar according to which metric?
  • Similar after which normalization?
  • Similar enough for which decision?
  • Similar enough under which risk tolerance?
  • Similar for which culture, language, script, and data source?
  • Similar with what evidence against the match?

That is the mindset of practical fuzzy name matching.

The rest of the book builds that mindset step by step.

References

Christen, P. (2012). Data Matching: Concepts and Techniques for Record Linkage, Entity Resolution, and Duplicate Detection. Springer.

Cohen, W. W., Ravikumar, P., & Fienberg, S. E. (2003). A comparison of string distance metrics for name-matching tasks. Proceedings of the IJCAI-03 Workshop on Information Integration on the Web (IIWeb-03).

Fellegi, I. P., & Sunter, A. B. (1969). A theory for record linkage. Journal of the American Statistical Association, 64(328), 1183–1210.

Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8(3), 338–353.

τ TheoryTrace