Text Processing — Hub
What are these tools for?
On OSCP you constantly need to search, filter, and extract data from files, command output, and logs. These tools are the building blocks — alone they’re useful, chained with pipes (|) they’re extremely powerful.
raw file or command output → grep (filter lines) → awk (extract fields) → sort -u → clean list
OSCP use: Extract usernames from XMPP logs, find passwords in configs, parse nmap output, build wordlists from leaked data, hunt creds during privesc.
📌 The Pipe — How Chaining Works
command1 | command2 | command3 > output.txtEach command reads stdout from the previous command. Order matters — filter first, then extract, then sort.
Example (HTB-style — extract usernames from XMPP log):
grep jab.htb xmpp.txt | awk -F\> '{print $2}' | awk -F@ '{print $1}' | sort -u > users.txt| Step | What it does |
|---|---|
grep jab.htb xmpp.txt | Keep only lines containing the domain |
awk -F\> '{print $2}' | Split on >, print part after tag (email) |
awk -F@ '{print $1}' | Split on @, print username only |
sort -u | Sort alphabetically, remove duplicates |
> users.txt | Save to file |
See Pipelines & Chaining for more multi-tool recipes. XMPP user extraction from Pidgin: UseCases for ports > Port 5222 / 5223 — XMPP / Jabber.
📌 Sub-Notes (This Folder)
| Note | Purpose |
|---|---|
| grep | Search/filter lines by pattern (text, regex) — Windows: grep > 📌 Windows equivalents (findstr / Select-String) |
| find | Search filesystem for files/dirs by name, type, permissions |
| awk | Process columns/fields — extract, transform, print |
| sed | Stream editor — find/replace, delete lines, transform text |
| cut | Extract columns by delimiter or character position |
| head | First N lines/bytes of a file — preview logs, scans |
| sort | Sort lines; -u removes duplicates |
| uniq | Remove adjacent duplicate lines (use after sort) |
| xargs | Pass piped input as arguments to another command |
| tr | Translate/delete characters (e.g., newlines, case) |
| Pipelines & Chaining | Combine all tools — OSCP recipes & patterns |
📌 Quick Tool Picker
| I need to… | Tool |
|---|---|
| Find lines containing “password” | grep (Linux) · Select-String / findstr (Windows) |
Find all .conf files on disk | find |
| Extract 2nd column from CSV | awk or cut |
Replace http with https | sed |
Get usernames from /etc/passwd | cut -d: -f1 |
| Preview first 200 lines of a huge file | head -n 200 file |
| Remove duplicate lines | sort -u or sort | uniq |
| Run command on every file found | find … | xargs |
Remove \r from Windows file | tr -d '\r' |
| Compare two files line-by-line | diff file1 file2 or vimdiff file1 file2 |
| Chain multiple filters | See Pipelines & Chaining |
📌 Quick OSCP Cheat Sheet
# ─── SEARCH FILES FOR CREDS ───────────────────────────────────
grep -rni "password" /var/www/ 2>/dev/null
find / -name "*.conf" 2>/dev/null | xargs grep -l "password" 2>/dev/null
# ─── EXTRACT USERS FROM /etc/passwd ───────────────────────────
grep -v "nologin\|false" /etc/passwd | cut -d: -f1
# ─── PARSE NMAP PORTS ─────────────────────────────────────────
grep "^[0-9]" allports.txt | cut -d'/' -f1 | tr '\n' ',' | sed 's/,$//'
# ─── XMPP / LOG USER EXTRACTION ───────────────────────────────
grep domain.local log.txt | awk -F@ '{print $1}' | awk -F/ '{print $NF}' | sort -u
# ─── FIND SUID BINARIES ───────────────────────────────────────
find / -perm -4000 -type f 2>/dev/null
# ─── COMPARE FILES ────────────────────────────────────────────
diff file1 file2
vimdiff file1 file2