sort — Sort & Deduplicate Lines
What is sort?
sort orders lines. With -u it also removes duplicates — the standard final step when building clean wordlists from messy logs.
Syntax
sort [OPTIONS] [FILE]
command | sort [OPTIONS]📌 Common Flags
| Flag | Description |
|---|---|
-u | Unique — remove duplicate lines (after sorting) |
-n | Numeric sort (2 before 10) |
-r | Reverse order |
-k N | Sort by field N (-k2, -k2,2 for column 2 only) |
-t CHAR | Field separator for -k |
-f | Case-insensitive |
-h | Human-readable numbers (K, M, G) |
-o FILE | Write output to file (can sort in place) |
-R | Random shuffle |
📌 Examples
# Alphabetical sort
sort file.txt
# Unique sorted (dedupe)
sort -u file.txt
sort -u users.txt > clean_users.txt
# Numeric sort
sort -n numbers.txt
# Reverse
sort -r file.txt
# Sort by 2nd column (colon-separated)
sort -t: -k3 -n /etc/passwd
# Randomize wordlist order
sort -R rockyou.txt > shuffled.txt
# Chain — extract + dedupe
grep jab.htb xmpp.txt | awk -F@ '{print $1}' | sort -u > users.txt📌 sort -u vs sort | uniq
| Method | Notes |
|---|---|
sort -u | One command — preferred |
sort | uniq | Classic; uniq only removes adjacent dupes, so sort first |
Both produce the same result when chained correctly.
📌 Quick Cheat Sheet
sort file.txt
sort -u file.txt # unique lines
sort -n file.txt # numeric
sort -u -o out.txt in.txt # sort + write file
grep pat f | awk '{print $1}' | sort -u > list.txt