文本去重排序
按行去重、排序、反转与随机打乱。
关于「文本去重排序」
它属于「文本处理」分类：中文排版、编码转换、统计与内容检查。
- 按<b>行</b>处理文本，适合整理名单、URL 列表、关键词等。
- 去重保留<b>首次出现</b>的那一行，后续重复项被移除。
- <b>忽略大小写</b>时，<code>Apple</code> 与 <code>apple</code> 视为同一条。
- 排序使用中文本地化规则，并开启数字识别，<code>item2</code> 会排在 <code>item10</code> 前面。
- 多个选项可以叠加，执行顺序为：去空格 → 删空行 → 去重 → 排序 → 反转/打乱 → 加行号。
完全免费、无需注册；所有处理都在浏览器本地完成，文件不会上传到任何服务器。
相关关键词：文本去重排序、文本处理、去重、排序、行、在线工具、免费
如何使用「文本去重排序」
- 打开工具：打开「文本去重排序」页面，无需注册、无需安装任何软件。
- 准备输入：准备好要处理的内容或文件：文本可直接粘贴，文件可拖入页面（本工具属于「文本处理」）。
- 填入内容：把内容粘贴进输入框，或把文件拖入页面；也可以点击「示例」先填入样例数据试一下。
- 设置选项并处理：按需调整页面上的选项，然后点击处理 / 转换 / 生成按钮。
- 获取结果：处理完全在浏览器本地完成，结果可直接复制或下载，文件不会上传到任何服务器。
常见问题
- 去重保留的是哪一条？
- 保留第一次出现的那一行，之后的重复项都会被删除，因此原有的相对顺序不会被打乱。
- 排序为什么不是按 ASCII？
- 这里使用中文本地化比较规则，能正确处理中文和带数字的字符串。例如 file2 会排在 file10 之前，比纯 ASCII 排序更符合直觉。
- 可以只统计重复而不删除吗？
- 当前版本直接输出去重后的结果。你可以对比处理前后的行数差值，消息栏会显示减少了多少行。
- 随机打乱可靠吗？
- 使用标准的 Fisher-Yates 洗牌算法，每种排列出现的概率均等，适合抽奖、随机分组等场景。
Dedupe & Sort Lines (English)
Dedupe, sort, reverse and shuffle lines.
Belongs to “Text Tools”: Chinese typography, encoding conversion, statistics and content checks.
- Processes text <b>line by line</b>, ideal for cleaning up name lists, URL lists, keywords, etc.
- Deduplication keeps the <b>first occurrence</b> of each line and removes later duplicates.
- With <b>ignore case</b>, <code>Apple</code> and <code>apple</code> count as the same entry.
- Sorting uses locale-aware comparison with numeric detection, so <code>item2</code> comes before <code>item10</code>.
- Options can be combined. Execution order: trim → remove empty → dedupe → sort → reverse/shuffle → numbering.
Free, no sign-up; everything runs locally in your browser and files are never uploaded.
Keywords: Dedupe & Sort Lines, Text Tools, dedupe, sort, lines, online tool, free
How to use it
- Open the tool：Open the “Dedupe & Sort Lines” page — no sign-up and nothing to install.
- Prepare the input：Prepare the content or file to process: paste text or drop a file onto the page (this tool belongs to “Text Tools”).
- Provide the input：Paste your content into the input box or drop a file onto the page; you can also click “Sample” to try example data first.
- Adjust options and run：Adjust the options if needed, then click the process / convert / generate button.
- Get the result：Everything runs locally in your browser; copy or download the result — your files are never uploaded.
FAQ
- Which copy is kept?
- The first occurrence is kept; later duplicates are removed, so the original relative order is preserved.
- Why not ASCII sort?
- Locale-aware comparison correctly handles Chinese and numeric strings — e.g. file2 comes before file10, which is more intuitive than pure ASCII sort.
- Can it only count duplicates?
- The current version outputs the deduplicated result directly. Compare the line counts before and after — the message bar shows how many lines were removed.
- Is the shuffle reliable?
- It uses the standard Fisher-Yates algorithm, giving every permutation equal probability — suitable for lottery or random grouping.
More tools in Text Tools
- Simplified ⇄ Traditional — Convert between Simplified and Traditional Chinese variants.
- Chinese to Pinyin — Convert Chinese to pinyin with tones, initials and polyphones.
- Letter Case Converter — Upper/lower/title/camel/snake case conversion.
- Circled Numbers & Letters — Convert digits and letters into circled Unicode chars.
- Mars Text Converter — Convert between plain text and "Mars" (stylized) text.
- Word & Char Counter — Count characters, Chinese chars, words, lines and reading time.
- Text Formatter — Clean blank lines, space CJK/English, unify punctuation.
- Text Diff — Compare two texts line by line with highlights.
- RMB Amount in Words — Convert numeric amounts into Chinese capital letters.
- Banned Word Checker — Built-in e-commerce/content sensitive-word list, customizable.
- ID Card Info Extractor — Extract birthday, gender, region from an ID number and validate.
- Smart To-Do List — Local to-do list with priority and filtering.