awk — Field / Column Processing
What is awk?
awk reads input line-by-line, splits each line into fields (columns), and lets you print, filter, or transform them. It’s pattern-matching + column extraction in one tool.
OSCP use: Extract usernames from logs, parse
/etc/passwd, pull columns from tool output, build wordlists from structured data.
Syntax
awk 'PATTERN { ACTION }' file
awk -F'SEP' '{ print $1, $2 }' file
command | awk '{ print $NF }'📌 Key Concepts
| Concept | Description |
|---|---|
$0 | Entire line |
$1, $2, $N | Field 1, 2, N (default split: whitespace) |
$NF | Last field |
-F SEP | Field separator (e.g., -F:, -F@, -F\>) |
NR | Current line number |
NF | Number of fields on current line |
BEGIN {} | Run before any input |
END {} | Run after all input |
Default: splits on spaces/tabs. Use -F for custom delimiters.
📌 Common Flags
| Flag | Description |
|---|---|
-F SEP | Field separator |
-v VAR=val | Set awk variable |
-f FILE | Read awk script from file |
📌 Basic Examples
# Print 1st and 3rd column (whitespace-separated)
awk '{ print $1, $3 }' file.txt
# Print last column
awk '{ print $NF }' file.txt
# Print line 5 only
awk 'NR==5' file.txt
# Lines where column 3 > 100
awk '$3 > 100' file.txt
# Lines matching regex pattern
awk '/error/ { print $0 }' log.txt📌 Field Separator (-F) — Most Important for OSCP
# /etc/passwd — username is field 1
awk -F: '{ print $1 }' /etc/passwd
# Email log — extract before @
awk -F@ '{ print $1 }' emails.txt
# XMPP log — extract after >
grep jab.htb xmpp.txt | awk -F\> '{ print $2 }' | awk -F@ '{ print $1 }'
# CSV
awk -F, '{ print $2 }' data.csv
# Path — last component
awk -F/ '{ print $NF }' paths.txt📌 HTB-Style User Extraction (Screenshot Example)
Input line might look like:
<message to='user@jab.htb'>...
grep jab.htb xmpp.txt | awk -F\> '{print $2}' | awk -F@ '{print $1}' | sort -u > users.txt| Step | Result |
|---|---|
grep jab.htb | Lines with domain |
awk -F\> '{print $2}' | Everything after first > |
awk -F@ '{print $1}' | Username before @ |
sort -u | Unique sorted list |
📌 Filtering + Printing
# Print usernames where UID >= 1000
awk -F: '$3 >= 1000 { print $1 }' /etc/passwd
# Print lines where field 2 equals "admin"
awk '$2 == "admin" { print $1 }' users.txt
# Count lines
awk 'END { print NR }' file.txt📌 GTFOBins / PrivEsc (careful — noisy)
# If awk is in sudo -l
sudo awk 'BEGIN {system("/bin/sh")}'
awk 'BEGIN {system("/bin/sh")}'📌 Quick Cheat Sheet
awk '{ print $1 }' file.txt # 1st column
awk '{ print $NF }' file.txt # last column
awk -F: '{ print $1 }' /etc/passwd # colon-separated
awk -F@ '{ print $1 }' emails.txt # before @
grep pat file | awk -F, '{ print $2 }' # chain with grep
awk '/pattern/ { print $1 }' file.txt # filter + print
awk 'NR==1,NR==5' file.txt # lines 1-5