AnyTool
Your files never leave your device. All processing happens locally in your browser.

How do I find fuzzy (near) duplicate rows in a CSV?

Paste your CSV/TSV into AnyTool's Fuzzy Deduplicator, pick the column(s) to match on, and choose an algorithm — Jaro-Winkler (best for names and short text) or Levenshtein (best for longer strings). It scores every pair, clusters near-duplicates transitively (so "John Smith", "Jon Smith" and a third variant land in one group), shows each group with a keep row and the rest flagged, and exports a keep-first de-duplicated CSV. A similarity slider controls how loose a match counts. Everything runs in your browser — nothing is uploaded.

  • Jaro-Winkler & Levenshtein fuzzy matching
  • Match on one or many columns (composite key)
  • Adjustable similarity threshold + transitive clustering
  • Export a de-duplicated CSV — 100% in your browser

What is

Fuzzy deduplication

Fuzzy deduplication finds records that are nearly — not exactly — identical, catching typos, abbreviations and formatting differences that exact matching misses. It scores string similarity (e.g. Jaro-Winkler or Levenshtein) against a threshold and groups records above it. Client-side fuzzy dedup does this in the browser with no upload.

File Tools

Related terms

fuzzy matchingrecord linkageJaro-WinklerLevenshteindeduplicationentity resolution

How to Find Fuzzy Duplicates in a CSV

Cluster near-duplicate rows and export a clean list, all in your browser.

25 secWeb browser
  1. 1

    Paste data

    Paste your CSV/TSV. The columns appear as selectable chips.

  2. 2

    Tune matching

    Pick the match column(s), an algorithm, and the similarity threshold.

  3. 3

    Export

    Review the duplicate groups, then copy or download the de-duplicated CSV.

Result: A de-duplicated CSV

Frequently Asked Questions

With AnyTool, yes — matching runs in your browser and the list is never uploaded.

All parsing, similarity scoring and clustering happen client-side, so customer or contact data never leaves your device, unlike upload-based dedup services.

Jaro-Winkler is usually better for names and short identifiers; Levenshtein suits longer strings like addresses.

Jaro-Winkler weights matching prefixes, so "Jon"/"John" score high, while Levenshtein counts raw edits and is more even-handed for long values. The tool lets you switch and compare.

Yes — select several columns and they are combined into one match key.

Matching on name + city, for example, treats two rows as duplicates only when the combined value is similar, reducing false positives from a single column.

Detailed Explanation

Methodology

How the Fuzzy Deduplicator Works

AnyTool finds near-duplicate rows entirely in the browser. Data is parsed with PapaParse (quoted fields survive), then a match key is built from the column(s) you select and normalised (lower-case, collapsed whitespace, optional punctuation stripping). Every candidate pair is scored with one of two string-similarity metrics: Jaro-Winkler, which weights matching prefixes and is the standard choice for names and short identifiers (so "Jon Smith" ≈ "John Smith"), or Levenshtein edit-distance ratio, which is more even-handed for longer strings like addresses and company names. Pairs at or above your similarity threshold are linked, and a union-find pass clusters them transitively so a chain of variants becomes one group. A lossless length pre-filter skips pairs that cannot possibly meet the threshold, keeping the pairwise scan fast on the main thread. The result is a list of duplicate groups (first row kept, the rest flagged with their score) plus a keep-first de-duplicated CSV for copy or download. Nothing is uploaded.

  • Runs client-side — the list is never uploaded
  • Two metrics: Jaro-Winkler (names) and Levenshtein (long strings)
  • Single or multi-column composite match keys
  • Adjustable similarity threshold + transitive (union-find) clustering
  • Keep-first de-duplicated CSV/TSV export
  • No CDN, no backend; scans up to 5,000 rows per pass in-browser
Privacy & Security

Privacy vs Upload-based Dedup Services

Most fuzzy-matching dedup services upload your spreadsheet to a server and cap the free tier by row count. AnyTool scores and clusters everything locally, so customer and contact data never leaves the browser, there is no per-row fee, and it works offline after first load. You choose the algorithm and threshold and see exactly which rows were grouped, instead of trusting an opaque backend.

Fuzzy dedup: AnyTool vs typical online services
CapabilityAnyToolTypical services
Data handlingIn your browserUploaded to a server
AlgorithmsJaro-Winkler + LevenshteinVaries / opaque
Match keyAny column(s)Often single field
Cost / limitFree, no signupPer-row or capped
ExportDe-duplicated CSVOften paywalled

AnyTool is a privacy-first client-side deduplicator — rows are scored locally and never uploaded.