Documentation
README
Data Formats
How to work with diverse and unknown data formats.
Format Detection
Always inspect before parsing:
file <filename> # MIME type detection
xxd <filename> | head -5 # hex dump (first bytes)
head -3 <filename> # text preview
python3 -c "
with open('<filename>', 'rb') as f:
h = f.read(16)
print(h, h.hex())
"
Common Formats
Binary
- Magic bytes: Most binary formats start with a signature (ELF:
\x7fELF, PNG:\x89PNG) - Endianness: Check if little-endian or big-endian (
struct.unpack('<I', ...)vs'>I') - Alignment: Fields are often aligned to 4 or 8 bytes
- Offsets: Binary headers often contain offsets to other sections
This is the opening of the README. Read the full README on GitHub.