arch: fix heap buffer overflow in rz_analysis_mask() (#6802)

rz_analysis_mask() called memset() with an unclamped oplen when an
instruction's decoded length extended past the remaining bytes in
the mask window, writing up to (oplen - nopcode) bytes past the end
of the size-byte heap allocation.

Clamp the memset length to the bytes actually available in the
buffer (avail = size - idx) before computing safe_oplen, and only
memset when nopcode fits within that clamped length.

Reproduced and verified with the h8300 arch/analysis plugin, where a
jsr instruction decodes to 4 bytes with only 2 bytes remaining in a
256-byte mask window, causing a 2-3 byte out-of-bounds heap write
(CWE-787). Confirmed via glibc _FORTIFY_SOURCE abort on direct
reproduction and via AddressSanitizer heap-buffer-overflow report.

Fixes #6795
This commit is contained in:
Sumeet Dhangare
2026-10-11 17:03:06 +00:00
committed by GitHub
parent 74ff761f4c
commit e26243f18d
2 changed files with 14 additions and 2 deletions
+4 -2
View File
@@ -903,8 +903,10 @@ RZ_API ut8 *rz_analysis_mask(RzAnalysis *analysis, ut32 size, const ut8 *data, u
if ((oplen = rz_analysis_op(analysis, op, at, data + idx, size - idx, RZ_ANALYSIS_OP_MASK_BASIC)) < 1) {
break;
}
if ((op->ptr != UT64_MAX || op->jump != UT64_MAX) && op->nopcode != 0) {
memset(ret + idx + op->nopcode, 0, oplen - op->nopcode);
ut32 avail = size - idx;
ut32 safe_oplen = (ut32)oplen > avail ? avail : (ut32)oplen;
if ((op->ptr != UT64_MAX || op->jump != UT64_MAX) && op->nopcode != 0 && op->nopcode <= safe_oplen) {
memset(ret + idx + op->nopcode, 0, safe_oplen - op->nopcode);
}
idx += oplen;
at += oplen;
+10
View File
@@ -508,3 +508,13 @@ args: 0
`---------------'
EOF
RUN
NAME=h8300: rz_analysis_mask does not overflow when jsr straddles mask window (#6795)
FILE=bins/elf/h8300/main.elf
CMDS=<<EOF
ao @ 0x534~mask
EOF
EXPECT=<<EOF
mask: ffff
EOF
RUN