qilingframework/qiling
1
0
Fork 0
mirror of https://github.com/qilingframework/qiling.git synced 2026-10-12 02:21:40 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/dev'

Browse Source
This commit is contained in:
golem9247
2025-11-04 15:06:52 +01:00
parent 29c552e05d a82abfd6aa
commit 8d52304ddd
95 changed files with 4875 additions and 3015 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/build-ci.yml
+6 -7
View File
@@ -10,10 +10,10 @@ jobs:
fail-fast: false
matrix:
#os: [windows-2019, macos-10.15, ubuntu-18.04, ubuntu-20.04]
os: [windows-latest, ubuntu-22.04]
os: [windows-latest, ubuntu-latest]
python-version: ["3.9", "3.11"]
include:
- os: ubuntu-22.04
- os: ubuntu-latest
python-version: 3.9
container: Docker
@@ -72,13 +72,12 @@ jobs:
cd ../qiling
cd ../examples/rootfs/x86_linux/kernel && unzip -P infected m0hamed_rootkit.ko.zip
cd ../../../../
pip3 install -e .[RE]
if [ ${{ matrix.os }} == 'ubuntu-18.04' ] and [ ${{ matrix.python-version }} == '3.9' ]; then
pip3 install -e .
pip3 install poetry
if [ ${{ matrix.contrainer }} != "" ]; then
docker run -it --rm -v ${GITHUB_WORKSPACE}:/qiling qilingframework/qiling:dev bash -c "cd tests && ./test_onlinux.sh"
else
pip3 install poetry
cd tests && ./test_onlinux.sh
cd tests && ./test_onlinux.sh
fi
# - name: mac run tests
README.md
+97 -101
View File
@@ -1,4 +1,4 @@
[![Documentation Status](https://readthedocs.org/projects/qilingframework/badge/?version=latest)](https://docs.qiling.io)
[![Documentation Status](https://github.com/qilingframework/qiling/wiki)](https://github.com/qilingframework/qiling/wiki)
[![Downloads](https://pepy.tech/badge/qiling)](https://pepy.tech/project/qiling)
[![Chat on Telegram](https://img.shields.io/badge/Chat%20on-Telegram-brightgreen.svg)](https://t.me/qilingframework)
@@ -8,24 +8,43 @@
<img width="150" height="150" src="https://raw.githubusercontent.com/qilingframework/qiling/master/docs/qiling2_logo_small.png">
</p>
[Qiling's use case, blog and related work](https://github.com/qilingframework/qiling/issues/134)
# Qiling Framework
Qiling is an advanced binary emulation framework, with the following features:
Qiling is an advanced binary emulation framework that allows you to emulate and sandbox code in an isolated environment across multiple platforms and architectures. Built on top of Unicorn Engine, Qiling provides a higher-level framework that understands operating system contexts, executable formats, and dynamic linking.
- Emulate multi-platforms: Windows, macOS, Linux, Android, BSD, UEFI, DOS, MBR.
- Emulate multi-architectures: 8086, X86, X86_64, ARM, ARM64, MIPS, RISC-V, PowerPC.
- Support multiple file formats: PE, Mach-O, ELF, COM, MBR.
- Support Windows Driver (.sys), Linux Kernel Module (.ko) & macOS Kernel (.kext) via [Demigod](https://groundx.io/demigod/).
- Emulates & sandbox code in an isolated environment.
- Provides a fully configurable sandbox.
- Provides in-depth memory, register, OS level and filesystem level API.
- Fine-grain instrumentation: allows hooks at various levels
(instruction/basic-block/memory-access/exception/syscall/IO/etc.)
- Provides virtual machine level API such as saving and restoring the current execution state.
- Supports cross architecture and platform debugging capabilities.
- Built-in debugger with reverse debugging capability.
- Allows dynamic hot patch on-the-fly running code, including the loaded library.
- True framework in Python, making it easy to build customized security analysis tools on top.
## Table of Contents
- [Features](#features)
- [Appearance](#Appearance)
- [Use Cases](#use-cases)
- [Quick Start](#quick-start)
- [Installation](#installation)
- [Basic Usage](#basic-usage)
- [Qiling vs. Other Emulators](#qiling-vs-other-emulators)
- [Qiling vs. Unicorn Engine](#qiling-vs-unicorn-engine)
- [Qiling vs. QEMU User Mode](#qiling-vs-qemu-user-mode)
- [Examples](#examples)
- [Qltool](#qltool)
- [Contributing](#contributing)
- [License](#license)
- [Contact](#contact)
- [Core Developers & Contributors](#core-developers--contributors)
## Features
- **Multi-platform Emulation**: Windows, macOS, Linux, Android, BSD, UEFI, DOS, MBR.
- **Multi-architecture Emulation**: 8086, X86, X86_64, ARM, ARM64, MIPS, RISC-V, PowerPC.
- **Multiple File Format Support**: PE, Mach-O, ELF, COM, MBR.
- **Kernel Module Emulation**: Supports Windows Driver (.sys), Linux Kernel Module (.ko) & macOS Kernel (.kext) via [Demigod](https://groundx.io/demigod/).
- **Isolated Sandboxing**: Emulates & sandboxes code in an isolated environment with a fully configurable sandbox.
- **In-depth API**: Provides in-depth memory, register, OS level, and filesystem level API.
- **Fine-grain Instrumentation**: Allows hooks at various levels (instruction/basic-block/memory-access/exception/syscall/IO/etc.).
- **Virtual Machine Level API**: Supports saving and restoring the current execution state.
- **Debugging Capabilities**: Supports cross-architecture and platform debugging, including a built-in debugger with reverse debugging capability.
- **Dynamic Hot Patching**: Allows dynamic hot patching of on-the-fly running code, including loaded libraries.
- **Python Framework**: A true framework in Python, making it easy to build customized security analysis tools.
## Appearance
Qiling also made its way to various international conferences.
@@ -49,79 +68,37 @@ Qiling also made its way to various international conferences.
- [Nullcon](https://nullcon.net/website/goa-2020/speakers/kaijern-lau.php)
2019:
- [DEFCON, USA](https://www.defcon.org/html/defcon-27/dc-27-demolabs.html#QiLing)
- [Hitcon](https://hitcon.org/2019/CMT/agenda)
- [Zeronights](https://zeronights.ru/report-en/qiling-io-advanced-binary-emulation-framework/)
## Use Cases
Qiling is backed by [Unicorn Engine](http://www.unicorn-engine.org).
Qiling has been presented at various international conferences, showcasing its versatility in:
Visit our [website](https://www.qiling.io) for more information.
- Binary analysis and reverse engineering.
- Malware analysis and sandboxing.
- Firmware analysis and emulation.
- Security research and vulnerability discovery.
- CTF challenges and exploit development.
---
#### License
For more details on Qiling's use cases, blog posts, and related work, please refer to [Qiling's use case, blog and related work](https://github.com/qilingframework/qiling/issues/134).
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
## Quick Start
---
### Installation
#### Qiling vs. other Emulators
Qiling requires Python 3.8 or newer. You can install it using pip:
There are many open-source emulators, but two projects closest to Qiling
are [Unicorn](http://www.unicorn-engine.org) & [QEMU user mode](https://qemu.org).
This section explains the main differences of Qiling against them.
```bash
pip install qiling
```
##### Qiling vs. Unicorn engine
For more detailed installation instructions and dependencies, please refer to the [official documentation](https://github.com/qilingframework/qiling/wiki/Installation).
Built on top of Unicorn, but Qiling & Unicorn are two different animals.
### Basic Usage
- Unicorn is just a CPU emulator, so it focuses on emulating CPU instructions,
that can understand emulator memory.
Beyond that, Unicorn is not aware of higher level concepts, such as dynamic
libraries, system calls, I/O handling or executable formats like PE, Mach-O
or ELF. As a result, Unicorn can only emulate raw machine instructions,
without Operating System (OS) context.
- Qiling is designed as a higher level framework, that leverages Unicorn to
emulate CPU instructions, but can understand OS: it has executable format
loaders (for PE, Mach-O & ELF currently), dynamic linkers (so we can
load & relocate shared libraries), syscall & IO handlers. For this reason,
Qiling can run executable binary without requiring its native OS.
##### Qiling vs. QEMU user mode
QEMU user mode does a similar thing to our emulator, that is, to emulate whole
executable binaries in a cross-architecture way.
However, Qiling offers some important differences against QEMU user mode:
- Qiling is a true analysis framework,
that allows you to build your own dynamic analysis tools on top (in Python).
Meanwhile, QEMU is just a tool, not a framework.
- Qiling can perform dynamic instrumentation, and can even hot patch code at
runtime. QEMU does neither.
- Not only working cross-architecture, Qiling is also cross-platform.
For example, you can run Linux ELF file on top of Windows.
In contrast, QEMU user mode only runs binary of the same OS, such as Linux
ELF on Linux, due to the way it forwards syscall from emulated code to
native OS.
- Qiling supports more platforms, including Windows, macOS, Linux & BSD. QEMU
user mode can only handle Linux & BSD.
---
#### Installation
Please see [setup guide](https://docs.qiling.io/en/latest/install/) file for how to install Qiling Framework.
---
#### Examples
The example below shows how to use Qiling framework in the most
straightforward way to emulate a Windows executable.
The example below shows how to use Qiling framework in the most straightforward way to emulate a Windows executable.
```python
from qiling import Qiling
@@ -135,8 +112,30 @@ if __name__ == "__main__":
ql.run()
```
- The following example shows how a Windows crackme may be patched dynamically
to make it always display the “Congratulation” dialog.
## Qiling vs. Other Emulators
There are many open-source emulators, but two projects closest to Qiling are [Unicorn](http://www.unicorn-engine.org) & [QEMU user mode](https://qemu.org). This section explains the main differences of Qiling against them.
### Qiling vs. Unicorn Engine
Built on top of Unicorn, but Qiling & Unicorn are two different animals.
- **Unicorn** is just a CPU emulator, so it focuses on emulating CPU instructions, that can understand emulator memory. Beyond that, Unicorn is not aware of higher level concepts, such as dynamic libraries, system calls, I/O handling or executable formats like PE, Mach-O or ELF. As a result, Unicorn can only emulate raw machine instructions, without Operating System (OS) context.
- **Qiling** is designed as a higher level framework, that leverages Unicorn to emulate CPU instructions, but can understand OS: it has executable format loaders (for PE, Mach-O & ELF currently), dynamic linkers (so we can load & relocate shared libraries), syscall & IO handlers. For this reason, Qiling can run executable binary without requiring its native OS.
### Qiling vs. QEMU User Mode
QEMU user mode does a similar thing to our emulator, that is, to emulate whole executable binaries in a cross-architecture way.
However, Qiling offers some important differences against QEMU user mode:
- **Qiling is a true analysis framework**, that allows you to build your own dynamic analysis tools on top (in Python). Meanwhile, QEMU is just a tool, not a framework.
- **Qiling can perform dynamic instrumentation**, and can even hot patch code at runtime. QEMU does neither.
- Not only working cross-architecture, **Qiling is also cross-platform**. For example, you can run Linux ELF file on top of Windows. In contrast, QEMU user mode only runs binary of the same OS, such as Linux ELF on Linux, due to the way it forwards syscall from emulated code to native OS.
- **Qiling supports more platforms**, including Windows, macOS, Linux & BSD. QEMU user mode can only handle Linux & BSD.
## Examples
- The following example shows how a Windows crackme may be patched dynamically to make it always display the “Congratulation” dialog.
```python
from qiling import Qiling
@@ -177,15 +176,13 @@ The below YouTube video shows how the above example works.
#### Emulating ARM router firmware on Ubuntu x64 host
Qiling Framework hot-patches and emulates an ARM router's `/usr/bin/httpd` on
an x86_64 Ubuntu host.
Qiling Framework hot-patches and emulates an ARM router's `/usr/bin/httpd` on an x86_64 Ubuntu host.
[![Qiling Tutorial: Emulating and Fuzz ARM router firmware](https://github.com/qilingframework/theme.qiling.io/blob/master/source/img/fuzzer.jpg?raw=true)](https://www.youtube.com/watch?v=e3_T3KLh2NU)
[![Qiling Tutorial: Emulating and Fuzz ARM router firmware](https://github.com/qilingframework/theme.qiling.io/blob/master/source/img/fuzzer.jpg?raw=true)](https://www.youtube.com/watch?v=e3_T3KLhNUs)
#### Qiling's IDA Pro Plugin: Instrument and Decrypt Mirai's Secret
This video demonstrates how Qiling's IDA Pro plugin can make IDA Pro run with
Qiling instrumentation engine.
This video demonstrates how Qiling's IDA Pro plugin can make IDA Pro run with Qiling instrumentation engine.
[![Qiling's IDA Pro Plugin: Instrument and Decrypt Mirai's Secret](http://img.youtube.com/vi/ZWMWTq2WTXk/0.jpg)](http://www.youtube.com/watch?v=ZWMWTq2WTXk)
@@ -195,63 +192,62 @@ Solving a simple CTF challenge with Qiling Framework and IDA Pro
[![Solving a simple CTF challenge with Qiling Framework and IDA Pro](https://i.ytimg.com/vi/SPjVAt2FkKA/0.jpg)](https://www.youtube.com/watch?v=SPjVAt2FkKA)
#### Emulating MBR
Qiling Framework emulates MBR
[![Qiling DEMO: Emulating MBR](https://github.com/qilingframework/theme.qiling.io/blob/master/source/img/mbr.png?raw=true)](https://github.com/qilingframework/theme.qiling.io/blob/master/source/img/mbr.png?raw=true)
---
#### Qltool
## Qltool
Qiling also provides a friendly tool named `qltool` to quickly emulate shellcode & executable binaries.
With qltool, easy execution can be performed:
With shellcode:
```
```bash
$ ./qltool code --os linux --arch arm --format hex -f examples/shellcodes/linarm32_tcp_reverse_shell.hex
```
With binary file:
```
```bash
$ ./qltool run -f examples/rootfs/x8664_linux/bin/x8664_hello --rootfs examples/rootfs/x8664_linux/
```
With binary and GDB debugger enabled:
```
```bash
$ ./qltool run -f examples/rootfs/x8664_linux/bin/x8664_hello --gdb 127.0.0.1:9999 --rootfs examples/rootfs/x8664_linux
```
With code coverage collection (UEFI only for now):
```
```bash
$ ./qltool run -f examples/rootfs/x8664_efi/bin/TcgPlatformSetupPolicy --rootfs examples/rootfs/x8664_efi --coverage-format drcov --coverage-file TcgPlatformSetupPolicy.cov
```
With JSON output (Windows, mainly):
```
```bash
$ ./qltool run -f examples/rootfs/x86_windows/bin/x86_hello.exe --rootfs examples/rootfs/x86_windows/ --console False --json
```
---
## Contributing
#### Contact
We welcome contributions from the community! If you're interested in contributing to Qiling Framework, please check out our [GitHub repository](https://github.com/qilingframework/qiling) and look for open issues or submit a pull request.
Get the latest info from our website https://www.qiling.io
## License
Contact us at email info@qiling.io,
via Twitter [@qiling_io](https://twitter.com/qiling_io).
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
---
## Contact
#### Core developers, Key Contributors and etc.
Get the latest info from our website [https://www.qiling.io](https://www.qiling.io)
Please refer to [CREDITS.md](https://github.com/qilingframework/qiling/blob/dev/CREDITS.md).
Contact us at email [[email protected]](mailto:[email protected]), or via Twitter [@qiling_io](https://twitter.com/qiling_io).
## Core Developers & Contributors
Please refer to [CREDITS.md](https://github.com/qilingframework/qiling/blob/dev/CREDITS.md).
examples/blob_raw.ql
+4
View File
@@ -0,0 +1,4 @@
[CODE]
load_address = 0x10000000
entry_point = 0x10000008
ram_size = 0xa00000
examples/hello_arm_blob_raw.py
+101
View File
@@ -0,0 +1,101 @@
##############################################################################
# This example is meant to demonstrate the modifications necessary
# to enable code coverage when emulating small code snippets or bare-metal
# code.
##############################################################################
from qiling import Qiling
from qiling.const import QL_ARCH, QL_OS, QL_VERBOSE
from qiling.extensions.coverage import utils as cov_utils
from qiling.loader.loader import Image
import os
BASE_ADDRESS = 0x10000000
CHECKSUM_FUNC_ADDR = BASE_ADDRESS + 0x8
END_ADDRESS = 0x100000ba
DATA_ADDR = 0xa0000000 # Arbitrary address for data
STACK_ADDR = 0xb0000000 # Arbitrary address for stack
# Python implementation of the checksum function being emulated
# This checksum function is intended to have different code paths based on the input
# which is useful for observing code coverage
def checksum_function(input_data_buffer: bytes):
expected_checksum_python = 0
input_data_len = len(input_data_buffer)
if input_data_len >= 1 and input_data_buffer[0] == 0xDE: # MAGIC_VALUE_1
for i in range(min(input_data_len, 4)):
expected_checksum_python += input_data_buffer[i]
expected_checksum_python += 0x10
elif input_data_len >= 2 and input_data_buffer[1] == 0xAD: # MAGIC_VALUE_2
for i in range(input_data_len):
expected_checksum_python ^= input_data_buffer[i]
expected_checksum_python += 0x20
else:
for i in range(input_data_len):
expected_checksum_python += input_data_buffer[i]
expected_checksum_python &= 0xFF # Ensure it's a single byte
return expected_checksum_python
def unmapped_handler(ql: Qiling, type: int, addr: int, size: int, value: int) -> None:
print(f"Unmapped Memory R/W, trying to access {size:d} bytes at {addr:#010x} from {ql.arch.regs.pc:#010x}")
def emulate_checksum_function(input_data_buffer: bytes) -> None:
print(f"\n--- Testing with input: {input_data_buffer.hex()} ---")
test_file = "rootfs/blob/example_raw.bin"
with open(test_file, "rb") as f:
raw_code: bytes = f.read()
ql: Qiling = Qiling(
code=raw_code,
archtype=QL_ARCH.ARM,
ostype=QL_OS.BLOB,
profile="blob_raw.ql",
verbose=QL_VERBOSE.DEBUG,
thumb=True
)
''' monkeypatch - Correcting the loader image name, used for coverage collection
removing all images with name 'blob_code' that were created by the blob loader.
This is necessary because some code coverage visualization tools require the
module name to match that of the input file '''
ql.loader.images = [img for img in ql.loader.images if img.path != 'blob_code']
ql.loader.images.append(Image(ql.loader.load_address, ql.loader.load_address + ql.os.code_ram_size, os.path.basename(test_file)))
input_data_len: int = len(input_data_buffer)
# Map memory for the data and stack
ql.mem.map(STACK_ADDR, 0x2000)
ql.mem.map(DATA_ADDR, ql.mem.align_up(input_data_len + 0x100)) # Map enough space for data
# Write input data
ql.mem.write(DATA_ADDR, input_data_buffer)
# Set up the stack pointer
ql.arch.regs.sp = STACK_ADDR + 0x2000 - 4
# Set up argument registers
ql.arch.regs.r0 = DATA_ADDR
ql.arch.regs.r1 = input_data_len
# Set the program counter to the function's entry point
ql.arch.regs.pc = CHECKSUM_FUNC_ADDR
# Set the return address (LR) to a dummy address.
ql.arch.regs.lr = 0xbebebebe
ql.hook_mem_unmapped(unmapped_handler)
#ql.debugger="gdb:127.0.0.1:9999"
# Start emulation
print(f"Starting emulation at PC: {hex(ql.arch.regs.pc)}")
try:
with cov_utils.collect_coverage(ql, 'drcov', 'output.cov'):
ql.run(begin=CHECKSUM_FUNC_ADDR, end=END_ADDRESS)
except Exception as e:
print(f"Emulation error: {e}")
print(f"Emulated checksum: {hex(ql.arch.regs.r0)}")
if __name__ == "__main__":
data = b"\x01\x02\x03\x04\x05" # Example input data
emulate_checksum_function(data)
examples/hello_arm_uboot.py
+59 -45
View File
@@ -8,68 +8,82 @@ sys.path.append("..")
from qiling.core import Qiling
from qiling.const import QL_ARCH, QL_OS, QL_VERBOSE
from qiling.os.const import STRING
from qiling.os.const import STRING, SIZE_T, POINTER
def get_kaimendaji_password():
def my_getenv(ql: Qiling):
env = {
"ID" : b"000000000000000",
"ethaddr" : b"11:22:33:44:55:66"
}
def my_getenv(ql: Qiling):
env = {
"ID" : b"000000000000000",
"ethaddr" : b"11:22:33:44:55:66"
}
params = ql.os.resolve_fcall_params({'key': STRING})
value = env.get(params["key"], b"")
params = ql.os.resolve_fcall_params({'key': STRING})
value = env.get(params["key"], b"")
value_addr = ql.os.heap.alloc(len(value))
ql.mem.write(value_addr, value)
value_addr = ql.os.heap.alloc(len(value))
ql.mem.write(value_addr, value)
ql.arch.regs.r0 = value_addr
ql.arch.regs.arch_pc = ql.arch.regs.lr
ql.arch.regs.r0 = value_addr
ql.arch.regs.arch_pc = ql.arch.regs.lr
def get_password(ql: Qiling):
password_raw = ql.mem.read(ql.arch.regs.r0, ql.arch.regs.r2)
password = ''
for item in password_raw:
if 0 <= item <= 9:
password += chr(item + 48)
else:
password += chr(item + 87)
def get_password(ql: Qiling):
# we land on a memcmp call, where the real password is being compared to
# the one provided by the user. we can follow the arguments to read the
# real password
print("The password is: %s" % password)
params = ql.os.resolve_fcall_params({
'ptr1': POINTER, # points to real password
'ptr2': POINTER, # points to user provided password
'size': SIZE_T # comparison length
})
def partial_run_init(ql: Qiling):
# argv prepare
ql.arch.regs.arch_sp -= 0x30
arg0_ptr = ql.arch.regs.arch_sp
ql.mem.write(arg0_ptr, b"kaimendaji")
ptr1 = params['ptr1']
size = params['size']
ql.arch.regs.arch_sp -= 0x10
arg1_ptr = ql.arch.regs.arch_sp
ql.mem.write(arg1_ptr, b"000000") # arbitrary password
password_raw = ql.mem.read(ptr1, size)
ql.arch.regs.arch_sp -= 0x20
argv_ptr = ql.arch.regs.arch_sp
ql.mem.write_ptr(argv_ptr, arg0_ptr)
ql.mem.write_ptr(argv_ptr + ql.arch.pointersize, arg1_ptr)
def __hex_digit(ch: int) -> str:
off = ord('0') if ch in range(10) else ord('a') - 10
ql.arch.regs.r2 = 2
ql.arch.regs.r3 = argv_ptr
return chr(ch + off)
with open("../examples/rootfs/blob/u-boot.bin.img", "rb") as f:
uboot_code = f.read()
# should be: "013f1f"
password = "".join(__hex_digit(ch) for ch in password_raw)
ql = Qiling(code=uboot_code[0x40:], archtype=QL_ARCH.ARM, ostype=QL_OS.BLOB, profile="uboot_bin.ql", verbose=QL_VERBOSE.OFF)
print(f'The password is: {password}')
image_base_addr = ql.loader.load_address
ql.hook_address(my_getenv, image_base_addr + 0x13AC0)
ql.hook_address(get_password, image_base_addr + 0x48634)
partial_run_init(ql)
def partial_run_init(ql: Qiling):
# argv prepare
ql.arch.regs.arch_sp -= 0x30
arg0_ptr = ql.arch.regs.arch_sp
ql.mem.write(arg0_ptr, b"kaimendaji")
ql.run(image_base_addr + 0x486B4, image_base_addr + 0x48718)
ql.arch.regs.arch_sp -= 0x10
arg1_ptr = ql.arch.regs.arch_sp
ql.mem.write(arg1_ptr, b"000000") # arbitrary password
ql.arch.regs.arch_sp -= 0x20
argv_ptr = ql.arch.regs.arch_sp
ql.mem.write_ptr(argv_ptr, arg0_ptr)
ql.mem.write_ptr(argv_ptr + ql.arch.pointersize, arg1_ptr)
ql.arch.regs.r2 = 2
ql.arch.regs.r3 = argv_ptr
if __name__ == "__main__":
get_kaimendaji_password()
with open("../examples/rootfs/blob/u-boot.bin.img", "rb") as f:
uboot_code = f.read()
ql = Qiling(code=uboot_code[0x40:], archtype=QL_ARCH.ARM, ostype=QL_OS.BLOB, profile="uboot_bin.ql", verbose=QL_VERBOSE.DEBUG)
imgbase = ql.loader.images[0].base
ql.hook_address(my_getenv, imgbase + 0x13AC0)
ql.hook_address(get_password, imgbase + 0x48634)
partial_run_init(ql)
ql.run(imgbase + 0x486B4, imgbase + 0x48718)
examples/rootfs
+1 -1
Submodule examples/rootfs updated: 6d4d654fdc...120fb6d377
examples/scripts/dllscollector.bat
+6
View File
@@ -94,6 +94,9 @@ CALL :collect_dll32 wininet.dll
CALL :collect_dll32 winmm.dll
CALL :collect_dll32 ws2_32.dll
CALL :collect_dll32 wsock32.dll
CALL :collect_dll32 msvcp140.dll
CALL :collect_dll32 msvcp140_1.dll
CALL :collect_dll32 msvcp140_2.dll
CALL :collect_dll32 downlevel\api-ms-win-core-fibers-l1-1-1.dll
CALL :collect_dll32 downlevel\api-ms-win-core-localization-l1-2-1.dll
@@ -131,6 +134,9 @@ CALL :collect_dll64 win32u.dll
CALL :collect_dll64 winhttp.dll
CALL :collect_dll64 wininet.dll
CALL :collect_dll64 ws2_32.dll
CALL :collect_dll64 msvcp140.dll
CALL :collect_dll64 msvcp140_1.dll
CALL :collect_dll64 msvcp140_2.dll
CALL :collect_dll64 downlevel\api-ms-win-crt-heap-l1-1-0.dll
CALL :collect_dll64 downlevel\api-ms-win-crt-locale-l1-1-0.dll
examples/src/blob/Makefile
+52
View File
@@ -0,0 +1,52 @@
# Makefile for Bare-Metal ARM Checksum Calculator
# --- Toolchain Definitions ---
TOOLCHAIN_PREFIX = arm-none-eabi
# Compiler, Linker, and Objcopy executables
CC = $(TOOLCHAIN_PREFIX)-gcc
LD = $(TOOLCHAIN_PREFIX)-gcc
OBJCOPY = $(TOOLCHAIN_PREFIX)-objcopy
# --- Source and Output Files ---
SRCS = example_raw.c
OBJS = $(SRCS:.c=.o) # Convert .c to .o
ELF = example_raw.elf
BIN = example_raw.bin
# --- Linker Script ---
LDSCRIPT = linker.ld
# --- Compiler Flags ---
CFLAGS = -c -O0 -mcpu=cortex-a7 -mthumb -ffreestanding -nostdlib
# --- Linker Flags ---
LDFLAGS = -T $(LDSCRIPT) -nostdlib
# --- Objcopy Flags ---
OBJCOPYFLAGS = -O binary
# --- Default Target ---
.PHONY: all clean
all: $(BIN)
# Rule to build the raw binary (.bin) from the ELF file
$(BIN): $(ELF)
$(OBJCOPY) $(OBJCOPYFLAGS) $< $@
@echo "Successfully created $(BIN)"
# Rule to link the object file into an ELF executable
$(ELF): $(OBJS) $(LDSCRIPT)
$(LD) $(LDFLAGS) $(OBJS) -o $@
@echo "Successfully linked $(ELF)"
# Rule to compile the C source file into an object file
%.o: %.c
$(CC) $(CFLAGS) $< -o $@
@echo "Successfully compiled $<"
# --- Clean Rule ---
clean:
rm -f $(OBJS) $(ELF) $(BIN)
@echo "Cleaned build artifacts."
examples/src/blob/example_raw.c
+56
View File
@@ -0,0 +1,56 @@
// example checksum algorithm to demonstrate raw binary code coverage in qiling
// example_raw.c
// Define some magic values
#define MAGIC_VALUE_1 0xDE
#define MAGIC_VALUE_2 0xAD
// This function calculates a checksum with branches based on input data
// It takes a pointer to data and its length
// Returns the checksum (unsigned char to fit in a byte)
unsigned char calculate_checksum(const unsigned char *data, unsigned int length) {
unsigned char checksum = 0;
// Branch 1: Check for MAGIC_VALUE_1 at the start
if (length >= 1 && data[0] == MAGIC_VALUE_1) {
// If first byte is MAGIC_VALUE_1, do a simple sum of first 4 bytes
// (or up to length if less than 4)
for (unsigned int i = 0; i < length && i < 4; i++) {
checksum += data[i];
}
// Add a fixed offset to make this path distinct
checksum += 0x10;
}
// Branch 2: Check for MAGIC_VALUE_2 at the second byte
else if (length >= 2 && data[1] == MAGIC_VALUE_2) {
// If second byte is MAGIC_VALUE_2, do a XOR sum of all bytes
for (unsigned int i = 0; i < length; i++) {
checksum ^= data[i];
}
// Add a fixed offset to make this path distinct
checksum += 0x20;
}
// Default Branch: Standard byte sum checksum
else {
for (unsigned int i = 0; i < length; i++) {
checksum += data[i];
}
}
return checksum;
}
// Minimal entry point for bare-metal.
// This function will not be called directly during Qiling emulation,
// but it's needed for the linker to have an entry point.
__attribute__((section(".text.startup")))
void _start() {
// In a real bare-metal application, this would initialize hardware,
// set up stacks, etc. For this example, it's just a placeholder.
// We'll call calculate_checksum directly from our Qiling script.
while (1) {
// Do nothing, or perhaps put the CPU to sleep
asm volatile ("wfi"); // Wait For Interrupt (ARM instruction)
}
}
examples/src/blob/linker.ld
+39
View File
@@ -0,0 +1,39 @@
/* linker.ld */
ENTRY(_start) /* Define the entry point of our program */
/* Define memory regions - simple RAM region for this example */
MEMORY
{
ram (rwx) : ORIGIN = 0x10000000, LENGTH = 64K /* 64KB of RAM for our program */
}
SECTIONS
{
/* Define the start of our program in memory.
*/
. = 0x10000000;
.text : {
KEEP(*(.text.startup)) /* Keep the _start function */
*(.text) /* All other code */
*(.text.*)
*(.rodata) /* Read-only data */
*(.rodata.*)
. = ALIGN(4);
} > ram /* Place .text section in the 'ram' region */
.data : {
. = ALIGN(4);
*(.data) /* Initialized data */
*(.data.*)
. = ALIGN(4);
} > ram
.bss : {
. = ALIGN(4);
*(.bss)
*(.bss.*)
. = ALIGN(4);
} > ram
}
examples/src/windows/except/CppHelloWorld.cpp
+11
View File
@@ -0,0 +1,11 @@
// This is the default Hello World program generated by Visual Studio 2022.
#include <iostream>
int main()
{
std::cout << "Hello World!\n";
return 0;
}
examples/src/windows/except/README
+3
View File
@@ -0,0 +1,3 @@
In this folder: Sources for programs intended to help test C++ features and software exceptions.
Compile with MSVC (Visual Studio 2022)
examples/src/windows/except/TestCppEx.cpp
+95
View File
@@ -0,0 +1,95 @@
#include <iostream>
#include <cstdlib>
/*
* Test simple try..catch.
*/
void test1()
{
std::cout << "y";
try {
std::cout << "y";
throw (unsigned int)0x12345678;
std::cout << "n";
}
catch(unsigned int n) {
n;
std::cout << "y";
}
std::cout << "y";
}
/*
* Test simple try..catch with throw.
*/
void test2()
{
std::cout << "y";
try {
std::cout << "y";
throw (unsigned int)0x12345679;
std::cout << "n";
}
catch (unsigned int n) {
n;
if (n == 0x12345679) {
std::cout << "y";
}
else {
std::cout << "n";
}
}
std::cout << "y";
}
/*
* Test nested try..catch with throw.
*/
void test3()
{
std::cout << "y";
try {
std::cout << "y";
try {
std::cout << "y";
throw (unsigned int)0x1234567A;
std::cout << "n";
}
catch (unsigned int n) {
n;
if (n == 0x1234567A) {
std::cout << "y";
}
else {
std::cout << "n";
}
}
std::cout << "y";
}
catch (unsigned int n) {
n;
std::cout << "n";
}
std::cout << "y";
}
int main()
{
/*
* For this program, all subtests successful will print:
* - 14 'y'
* - 0 'n'
*/
test1();
test2();
test3();
}
examples/src/windows/except/TestCppExUnhandled.cpp
+46
View File
@@ -0,0 +1,46 @@
#include <windows.h>
#include <cstdio>
LONG WINAPI CustomExceptionFilter(EXCEPTION_POINTERS* ExceptionInfo) {
printf("Inside exception filter (GOOD)\n");
DWORD exceptionCode = (DWORD)ExceptionInfo->ExceptionRecord->ExceptionCode;
printf("Exception Code: 0x%X\n", exceptionCode);
if (exceptionCode == 0xE06D7363) { // code for C++ exception
printf("Exception code DOES match, GOOD\n");
}
else {
printf("Exception code DOES NOT match, BAD\n");
}
printf("Exception Address: 0x%llx\n", (ULONGLONG)ExceptionInfo->ExceptionRecord->ExceptionAddress);
printf("After printing exception: (GOOD)\n");
return EXCEPTION_EXECUTE_HANDLER;
}
int main() {
/*
* For this program, all subtests successful will print:
* - 3 'GOOD'
* - 0 'BAD'
*
* It is expected that the program terminates abnormally
* with status code 0xE06D7363 (C++ exception)
*/
// Set the custom top-level exception filter
SetUnhandledExceptionFilter(CustomExceptionFilter);
// Throw an unhandled exception.
// It should be caught by our filter.
throw (unsigned int)5;
// We should never reach this point, because the exception
// dispatcher should terminate the program after our unhandled
// exception filter is called.
printf("After exception filter (BAD)\n");
return 0;
}
examples/src/windows/except/TestCppExUnhandled2.cpp
+21
View File
@@ -0,0 +1,21 @@
#include <iostream>
#include <cstdio>
int main()
{
/*
* For this program, all subtests successful will print:
* - 1 'GOOD'
* - 0 'BAD'
*
* It is expected that the program terminates abnormally
* with status code 0xC0000409 (stack buffer overrun/security
* check failure)
*/
printf("Before throw (GOOD)\n");
throw (unsigned int)5;
printf("After throw (BAD)\n");
}
examples/src/windows/except/TestCppTypes.cpp
+93
View File
@@ -0,0 +1,93 @@
#include <iostream>
struct TestStruct {
float q;
};
class TestClass {
public:
int x, y;
virtual ~TestClass() {
std::cout << "TestClass destructor, GOOD" << std::endl;
};
void yyy() {
std::cout << "REALLY GOOD" << std::endl;
}
};
class Something {
public:
char z;
virtual ~Something() {
std::cout << "Something destructor, GOOD" << std::endl;
};
virtual void zzz() {
std::cout << "BAD" << std::endl;
};
};
class TestClass2 : public TestClass, public Something {
public:
int z;
virtual ~TestClass2() {
std::cout << "TestClass2 destructor, GOOD" << std::endl;
};
virtual void zzz() {
std::cout << "GOOD" << std::endl;
};
};
int main()
{
/*
* For this program, all subtests successful will print:
* - 12 'GOOD'
* - 0 'BAD'
*/
int x = 5;
TestClass p;
TestStruct s;
std::cout << typeid(x).name() << std::endl;
if (strcmp(typeid(x).name(), "int") == 0) {
std::cout << "typeid(x) is int, GOOD" << std::endl;
}
else {
std::cout << "typeid(x) is NOT int, BAD" << std::endl;
}
std::cout << typeid(p).name() << std::endl;
if (strcmp(typeid(p).name(), "class TestClass") == 0) {
std::cout << "typeid(p) is \"class TestClass\", GOOD" << std::endl;
}
else {
std::cout << "typeid(p) is NOT \"class TestClass\", BAD" << std::endl;
}
std::cout << typeid(s).name() << std::endl;
if (strcmp(typeid(s).name(), "struct TestStruct") == 0) {
std::cout << "typeid(s) is \"struct TestStruct\", GOOD" << std::endl;
}
else {
std::cout << "typeid(s) is NOT \"struct TestStruct\", BAD" << std::endl;
}
std::cout << "Reached virtual methods and dynamic_cast test. GOOD" << std::endl;
TestClass2* kz = new TestClass2;
Something* ks = static_cast<Something*>(kz);
ks->zzz();
TestClass* pk = dynamic_cast<TestClass*>(ks);
pk->yyy();
std::cout << "Reached virtual destructor test. GOOD" << std::endl;
delete pk;
std::cout << "Finished all tests. GOOD" << std::endl;
}
examples/src/windows/except/TestSoftSEH.cpp
+45
View File
@@ -0,0 +1,45 @@
#include <windows.h>
#include <cstdio>
void test1() {
__try {
printf("Inside __try block. (GOOD)\n");
RaiseException(
0xE0000001,
0,
0,
nullptr
);
printf("After RaiseException. (BAD)\n");
}
__except (EXCEPTION_EXECUTE_HANDLER) {
printf("In __except block. (GOOD)\n");
unsigned long excepCode = GetExceptionCode();
printf("Exception code=0x%x\n", excepCode);
if (excepCode == 0xE0000001) {
printf("Exception code IS same, GOOD\n");
}
else {
printf("Exception code DOES NOT MATCH, BAD\n");
}
}
printf("After __except block. (GOOD)\n");
}
int main() {
/*
* For this program, all subtests successful will print:
* - 4 'GOOD'
* - 0 'BAD'
*/
test1();
return 0;
}
examples/uboot_bin.ql
+2
View File
@@ -1,6 +1,8 @@
[CODE]
ram_size = 0xa00000
load_address = 0x80800000
entry_point = 0x80800000
heap_address = 0xa0000000
heap_size = 0x300000
pyproject.toml
+1 -1
View File
@@ -28,7 +28,7 @@ keywords = [
[tool.poetry.dependencies]
python = "^3.8"
capstone = "^4"
capstone = "^5"
unicorn = "2.1.3"
pefile = ">=2022.5.30"
python-registry = "^1.3.1"
qiling/arch/arm64.py
+2 -1
View File
@@ -45,7 +45,8 @@ class QlArchARM64(QlArch):
**arm64_const.reg_map_q,
**arm64_const.reg_map_s,
**arm64_const.reg_map_w,
**arm64_const.reg_map_v
**arm64_const.reg_map_v,
**arm64_const.reg_map_fp
)
pc_reg = 'pc'
qiling/arch/arm64_const.py
+6
View File
@@ -68,6 +68,7 @@ reg_map = {
"pc": UC_ARM64_REG_PC,
"lr": UC_ARM64_REG_LR,
"cpacr_el1": UC_ARM64_REG_CPACR_EL1,
"pstate": UC_ARM64_REG_PSTATE,
}
reg_map_b = {
@@ -313,3 +314,8 @@ reg_map_v = {
"v30": UC_ARM64_REG_V30,
"v31": UC_ARM64_REG_V31
}
reg_map_fp = {
"fpcr": UC_ARM64_REG_FPCR,
"fpsr": UC_ARM64_REG_FPSR
}
qiling/arch/x86_utils.py
+42 -6
View File
@@ -5,6 +5,7 @@ from typing import Optional
from qiling import Qiling
from qiling.arch.x86 import QlArchIntel
from qiling.arch.x86_const import *
from qiling.const import QL_ARCH
from qiling.exception import QlGDTError, QlMemoryMappedError
from qiling.os.memory import QlMemoryManager
@@ -60,6 +61,8 @@ class GDTManager:
# setup GDT by writing to GDTR
ql.arch.regs.write(UC_X86_REG_GDTR, (0, base, limit, 0x0))
self.is_long_mode = ql.arch.type is QL_ARCH.X8664
self.array = GDTArray(ql.mem, base, num_entries)
@staticmethod
@@ -93,7 +96,18 @@ class GDTManager:
return (idx << 3) | QL_X86_SEGSEL_TI_GDT | rpl
def register_gdt_segment(self, index: int, seg_base: int, seg_limit: int, access: int) -> int:
flags = QL_X86_F_OPSIZE_32
is_code = access & QL_X86_A_CODE
if is_code and self.is_long_mode:
# If this is a code segment and 64-bit long mode is enabled,
# then set the long segment descriptor bit.
# This prevents some strange CPU errors encountered with
# intra-privilege level IRET instructions used for
# context switching on 64-bit Windows.
flags = QL_X86_F_LONG
else:
# Otherwise, OPSIZE_32 should be set.
flags = QL_X86_F_OPSIZE_32
# is this a huge segment?
if seg_limit > (1 << 16):
@@ -138,16 +152,21 @@ class SegmentManager:
class SegmentManager86(SegmentManager):
def setup_cs_ds_ss_es(self, base: int, size: int) -> None:
# While debugging the linux kernel segment, the cs segment was found on the third segment of gdt.
# TODO: 64-bit code segment access bits were adjusted, removing the conforming bit.
# Perhaps make the same change for x86?
access = QL_X86_A_PRESENT | QL_X86_A_PRIV_3 | QL_X86_A_DESC_CODE | QL_X86_A_CODE | QL_X86_A_CODE_C | QL_X86_A_CODE_R
# While debugging the linux kernel segment, the cs segment was found on the third segment of gdt.
selector = self.gdtm.register_gdt_segment(3, base, size - 1, access)
self.arch.regs.cs = selector
# TODO : The section permission here should be QL_X86_A_PRIV_3, but I do n’t know why it can only be set to QL_X86_A_PRIV_0.
# TODO: 64-bit data segment access bits were adjusted, removing the direction bit.
# After this change, there were no problems changing the privilege level to ring 3.
# Perhaps make the same change for x86?
access = QL_X86_A_PRESENT | QL_X86_A_PRIV_0 | QL_X86_A_DESC_DATA | QL_X86_A_DATA | QL_X86_A_DATA_E | QL_X86_A_DATA_W
# While debugging the Linux kernel segment, I found that the three segments DS, SS, and ES all point to the same location in the GDT table.
# This position is the fifth segment table of GDT.
access = QL_X86_A_PRESENT | QL_X86_A_PRIV_0 | QL_X86_A_DESC_DATA | QL_X86_A_DATA | QL_X86_A_DATA_E | QL_X86_A_DATA_W
selector = self.gdtm.register_gdt_segment(5, base, size - 1, access)
self.arch.regs.ds = selector
@@ -169,15 +188,32 @@ class SegmentManager86(SegmentManager):
class SegmentManager64(SegmentManager):
def setup_cs_ds_ss_es(self, base: int, size: int) -> None:
# Code segment access bits:
# * QL_X86_A_PRESENT : Present
# * QL_X86_A_PRIV_3 : Ring 3 (user-mode)
# * QL_X86_A_DESC_CODE : Segment describes a code segment
# * QL_X86_A_CODE : Executable bit set
# * QL_X86_A_CODE_R : Readable
# Not set:
# * QL_X86_A_CODE_C : Conforming bit
# -> unset means code in this segment can only be executed from the ring set in DPL.
access = QL_X86_A_PRESENT | QL_X86_A_PRIV_3 | QL_X86_A_DESC_CODE | QL_X86_A_CODE | QL_X86_A_CODE_R
# While debugging the linux kernel segment, the cs segment was found on the sixth segment of gdt.
access = QL_X86_A_PRESENT | QL_X86_A_PRIV_3 | QL_X86_A_DESC_CODE | QL_X86_A_CODE | QL_X86_A_CODE_C | QL_X86_A_CODE_R
selector = self.gdtm.register_gdt_segment(6, base, size - 1, access)
self.arch.regs.cs = selector
# TODO : The section permission here should be QL_X86_A_PRIV_3, but I do n’t know why it can only be set to QL_X86_A_PRIV_0.
# Data segment access bits:
# * QL_X86_A_PRESENT : Present
# * QL_X86_A_PRIV_3 : Ring 3 (user-mode)
# * QL_X86_A_DESC_DATA : Segment describes a data segment
# * QL_X86_A_DATA : Executable bit NOT set
# * QL_X86_A_DATA_W : Writable
# Not set:
# * QL_X86_A_DATA_E : Direction bit
# -> unset means the data segment grows upward, rather than downward.
access = QL_X86_A_PRESENT | QL_X86_A_PRIV_3 | QL_X86_A_DESC_DATA | QL_X86_A_DATA | QL_X86_A_DATA_W
# When I debug the Linux kernel, I find that only the SS is set to the fifth segment table, and the rest are not set.
access = QL_X86_A_PRESENT | QL_X86_A_PRIV_0 | QL_X86_A_DESC_DATA | QL_X86_A_DATA | QL_X86_A_DATA_E | QL_X86_A_DATA_W
selector = self.gdtm.register_gdt_segment(5, base, size - 1, access)
# self.arch.regs.ds = selector
qiling/debugger/__init__.py
-2
View File
@@ -1,3 +1 @@
from .debugger import QlDebugger
# from .disassember import QlDisassember
# from .utils import QlReadELF
qiling/debugger/disassember.py
-55
View File
@@ -1,55 +0,0 @@
#!/usr/bin/env python3
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from elftools.elf.elffile import ELFFile
from qiling import Qiling
from qiling.const import *
from capstone import *
class QlDisassember():
def __init__(self, ql:Qiling):
self.ql = ql
def disasm_all_lines(self):
disasm_result = []
if self.ql.os.type == QL_OS.LINUX:
disasm_result = self.disasm_elf()
return disasm_result
def disasm_elf(self, seg_name='.text'):
def disasm(ql, address, size):
md = ql.arch.disassembler
md.detail = True
return md.disasm(ql.mem.read(address, size), address)
disasm_result = []
if self.ql.arch.type == QL_ARCH.X86:
BASE = int(self.ql.profile.get("OS32", "load_address"), 16)
seg_start = 0x0
seg_end = 0x0
f = open(self.ql.path, 'rb')
elffile = ELFFile(f)
elf_header = elffile.header
reladyn = elffile.get_section_by_name(seg_name)
# No PIE
if elf_header['e_type'] == 'ET_EXEC':
seg_start = reladyn.header.sh_addr
seg_end = seg_start + reladyn.data_size
# PIE
elif elf_header['e_type'] == 'ET_DYN':
seg_start = BASE + reladyn.header.sh_addr
seg_end = seg_start + reladyn.data_size
for insn in disasm(ql, seg_start, seg_end-seg_start):
disasm_result.append(insn)
return disasm_result
qiling/debugger/gdb/xml/arm/arm-m-profile.xml → qiling/debugger/gdb/xml/cortex_m/arm-m-profile.xml
+7 -1
View File
@@ -25,4 +25,10 @@
<reg name="pc" bitsize="32" type="code_ptr"/>
<reg name="xpsr" bitsize="32" regnum="25"/>
</feature>
<reg name="msp" bitsize="32"/>
<reg name="psp" bitsize="32"/>
<reg name="primask" bitsize="32"/>
<reg name="basepri" bitsize="32"/>
<reg name="faultmask" bitsize="32"/>
<reg name="control" bitsize="32"/>
</feature>
qiling/debugger/gdb/xml/cortex_m/target.xml
+12
View File
@@ -0,0 +1,12 @@
<?xml version="1.0"?>
<!-- Copyright (C) 2009-2016 Free Software Foundation, Inc.
*!Copying and distribution of this file, with or without modification,
*!are permitted in any medium without royalty provided the copyright
*!notice and this notice are preserved. -->
<!DOCTYPE target SYSTEM "gdb-target.dtd">
<target xmlns:xi="http://www.w3.org/2001/XInclude">
<architecture>armv7-m</architecture>
<xi:include href="arm-m-profile.xml"/>
</target>
qiling/debugger/gdb/xmlregs.py
+12 -4
View File
@@ -13,13 +13,21 @@ from qiling.arch.arm_const import (
reg_map_q as arm_regs_q,
reg_map_s as arm_regs_s
)
from qiling.arch.cortex_m_const import (
reg_map as cortex_m_regs
)
from qiling.arch.arm64_const import (
reg_map as arm64_regs,
reg_map_v as arm64_regs_v
reg_map_v as arm64_regs_v,
reg_map_fp as arm64_reg_map_fp
)
from qiling.arch.mips_const import (
reg_map as mips_regs_gpr
)
from qiling.arch.x86_const import (
reg_map_32 as x86_regs_32,
reg_map_64 as x86_regs_64,
@@ -136,10 +144,10 @@ class QlGdbFeatures:
QL_ARCH.X86: dict(**x86_regs_32, **x86_regs_misc, **x86_regs_cr, **x86_regs_st, **x86_regs_xmm),
QL_ARCH.X8664: dict(**x86_regs_64, **x86_regs_misc, **x86_regs_cr, **x86_regs_st, **x86_regs_xmm, **x86_regs_ymm),
QL_ARCH.ARM: dict(**arm_regs, **arm_regs_vfp, **arm_regs_q, **arm_regs_s),
QL_ARCH.CORTEX_M: arm_regs,
QL_ARCH.ARM64: dict(**arm64_regs, **arm64_regs_v),
QL_ARCH.CORTEX_M: dict(**cortex_m_regs),
QL_ARCH.ARM64: dict(**arm64_regs, **arm64_regs_v, **arm64_reg_map_fp),
QL_ARCH.MIPS: dict(**mips_regs_gpr),
QL_ARCH.PPC: dict(**ppc_regs),
QL_ARCH.PPC: dict(**ppc_regs)
}[archtype]
regsinfo = sorted(QlGdbFeatures.__walk_xml_regs(xmltree))
qiling/debugger/qdb/arch/__init__.py
+2 -3
View File
@@ -3,7 +3,6 @@
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from .arch_x86 import ArchX86
from .arch_mips import ArchMIPS
from .arch_arm import ArchARM, ArchCORTEX_M
from .arch_x8664 import ArchX8664
from .arch_intel import ArchIntel, ArchX86, ArchX64
from .arch_mips import ArchMIPS
qiling/debugger/qdb/arch/arch.py
+66 -17
View File
@@ -3,32 +3,81 @@
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from typing import Collection, Dict, Mapping, Optional, TypeVar
from qiling.const import QL_ARCH
from unicorn import UC_ERR_READ_UNMAPPED
import unicorn
T = TypeVar('T')
class Arch:
"""
base class for arch
"""Arch base class.
"""
def __init__(self):
pass
def __init__(self, regs: Collection[str], swaps: Mapping[str, str], asize: int, isize: int) -> None:
"""Initialize architecture instance.
Args:
regs : collection of registers names to include in context
asize : native address size in bytes
isize : instruction size in bytes
swaps : readable register names alternatives, may be empty
"""
self._regs = regs
self._swaps = swaps
self._asize = asize
self._isize = isize
@property
def arch_insn_size(self):
return 4
def regs(self) -> Collection[str]:
"""Collection of registers names.
"""
return self._regs
@property
def archbit(self):
return 4
def isize(self) -> int:
"""Native instruction size.
"""
def read_insn(self, address: int):
try:
result = self.read_mem(address, self.arch_insn_size)
except unicorn.unicorn.UcError as err:
result = None
return self._isize
return result
@property
def asize(self) -> int:
"""Native pointer size.
"""
return self._asize
def swap_regs(self, mapping: Mapping[str, T]) -> Dict[str, T]:
"""Swap default register names with their aliases.
Args:
mapping: regsiters names mapped to their values
Returns: a new dictionary where all swappable names were swapped with their aliases
"""
return {self._swaps.get(k, k): v for k, v in mapping.items()}
def unalias(self, name: str) -> str:
"""Get original register name for the specified alias.
Args:
name: aliaes register name
Returns: original name of aliased register, or same name if not an alias
"""
# perform a reversed lookup in swaps to find the original name for given alias
return next((org for org, alt in self._swaps.items() if name == alt), name)
def read_insn(self, address: int) -> Optional[bytearray]:
"""Read a single instruction from given address.
Args:
address: memory address to read from
Returns: instruction bytes, or None if memory could not be read
"""
return self.try_read_mem(address, self.isize)
qiling/debugger/qdb/arch/arch_arm.py
+112 -72
View File
@@ -3,105 +3,145 @@
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from typing import Mapping
from typing import ClassVar, Dict, Optional
from .arch import Arch
class ArchARM(Arch):
def __init__(self):
super().__init__()
self._regs = (
"r0", "r1", "r2", "r3",
"r4", "r5", "r6", "r7",
"r8", "r9", "r10", "r11",
"r12", "sp", "lr", "pc",
)
_flags_reg: ClassVar[str] = 'cpsr'
@property
def regs(self):
return self._regs
def __init__(self) -> None:
regs = (
'r0', 'r1', 'r2', 'r3',
'r4', 'r5', 'r6', 'r7',
'r8', 'r9', 'r10', 'r11',
'r12', 'sp', 'lr', 'pc'
)
@regs.setter
def regs(self, regs):
self._regs += regs
aliases = {
'r9' : 'sb',
'r10': 'sl',
'r12': 'ip',
'r11': 'fp'
}
@property
def regs_need_swapped(self):
return {
"sl": "r10",
"ip": "r12",
"fp": "r11",
}
asize = 4
isize = 4
super().__init__(regs, aliases, asize, isize)
@staticmethod
def get_flags(bits: int) -> Mapping[str, bool]:
"""
get flags for ARM
"""
def get_mode(bits: int) -> int:
"""
get operating mode for ARM
"""
return {
0b10000: "User",
0b10001: "FIQ",
0b10010: "IRQ",
0b10011: "Supervisor",
0b10110: "Monitor",
0b10111: "Abort",
0b11010: "Hypervisor",
0b11011: "Undefined",
0b11111: "System",
}.get(bits & 0x00001f)
def get_flags(bits: int) -> Dict[str, bool]:
return {
"mode": get_mode(bits),
"thumb": bits & 0x00000020 != 0,
"fiq": bits & 0x00000040 != 0,
"irq": bits & 0x00000080 != 0,
"neg": bits & 0x80000000 != 0,
"zero": bits & 0x40000000 != 0,
"carry": bits & 0x20000000 != 0,
"overflow": bits & 0x10000000 != 0,
}
'thumb': bits & (0b1 << 5) != 0,
'fiq': bits & (0b1 << 6) != 0,
'irq': bits & (0b1 << 7) != 0,
'overflow': bits & (0b1 << 28) != 0,
'carry': bits & (0b1 << 29) != 0,
'zero': bits & (0b1 << 30) != 0,
'neg': bits & (0b1 << 31) != 0
}
@staticmethod
def get_mode(bits: int) -> str:
modes = {
0b10000: 'User',
0b10001: 'FIQ',
0b10010: 'IRQ',
0b10011: 'Supervisor',
0b10110: 'Monitor',
0b10111: 'Abort',
0b11010: 'Hypervisor',
0b11011: 'Undefined',
0b11111: 'System'
}
return modes.get(bits & 0b11111, '?')
@property
def thumb_mode(self) -> bool:
"""
helper function for checking thumb mode
def is_thumb(self) -> bool:
"""Query whether the processor is currently in thumb mode.
"""
return self.ql.arch.is_thumb
@property
def isize(self) -> int:
return 2 if self.is_thumb else self._isize
def read_insn(self, address: int) -> bytes:
"""
read instruction depending on current operating mode
@staticmethod
def __is_wide_insn(data: bytes) -> bool:
"""Determine whether a sequence of bytes respresents a wide thumb instruction.
"""
def thumb_read(address: int) -> bytes:
assert len(data) in (2, 4), f'unexpected instruction length: {len(data)}'
first_two = self.ql.mem.read_ptr(address, 2)
result = self.ql.pack16(first_two)
# determine whether this is a wide instruction by inspecting the 5 most
# significant bits in the first half-word
return (data[1] >> 3) & 0b11111 in (0b11101, 0b11110, 0b11111)
# to judge it's thumb mode or not
if any([
first_two & 0xf000 == 0xf000,
first_two & 0xf800 == 0xf800,
first_two & 0xe800 == 0xe800,
]):
def __read_thumb_insn_fail(self, address: int) -> Optional[bytearray]:
"""A failsafe method for reading thumb instructions. This method is needed for
rare cases in which a narrow instruction is on a page boundary where the next
page is unavailable.
"""
latter_two = self.ql.mem.read_ptr(address+2, 2)
result += self.ql.pack16(latter_two)
lo_half = self.try_read_mem(address, 2)
return result
if lo_half is None:
return None
return super().read_insn(address) if not self.thumb_mode else thumb_read(address)
data = lo_half
if ArchARM.__is_wide_insn(data):
hi_half = self.try_read_mem(address + 2, 2)
# fail if higher half-word was required but could not be read
if hi_half is None:
return None
data.extend(hi_half)
return data
def __read_thumb_insn(self, address: int) -> Optional[bytearray]:
"""Read one instruction in thumb mode.
Thumb instructions may be either 2 or 4 bytes long, depending on encoding of
the first word. However, reading two chunks of two bytes each is slower. For
most cases reading all four bytes in advance will be safe and quicker.
"""
data = self.try_read_mem(address, 4)
if data is None:
# there is a slight chance we could not read 4 bytes because only 2
# are available. try the failsafe method to find out
return self.__read_thumb_insn_fail(address)
if ArchARM.__is_wide_insn(data):
return data
return data[:2]
def read_insn(self, address: int) -> Optional[bytearray]:
"""Read one instruction worth of bytes.
"""
if self.is_thumb:
return self.__read_thumb_insn(address)
return super().read_insn(address)
class ArchCORTEX_M(ArchARM):
_flags_reg: ClassVar[str] = 'xpsr'
def __init__(self):
super().__init__()
self.regs += ("xpsr", "control", "primask", "basepri", "faultmask")
self._regs += (
'xpsr', 'control', 'primask',
'basepri', 'faultmask'
)
qiling/debugger/qdb/arch/arch_intel.py
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/env python3
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from typing import Collection, Dict
from .arch import Arch
class ArchIntel(Arch):
"""Arch base class for Intel architecture.
"""
def __init__(self, regs: Collection[str], asize: int) -> None:
super().__init__(regs, {}, asize, 15)
@staticmethod
def get_flags(bits: int) -> Dict[str, bool]:
return {
'CF' : bits & (0b1 << 0) != 0, # carry
'PF' : bits & (0b1 << 2) != 0, # parity
'AF' : bits & (0b1 << 4) != 0, # adjust
'ZF' : bits & (0b1 << 6) != 0, # zero
'SF' : bits & (0b1 << 7) != 0, # sign
'IF' : bits & (0b1 << 9) != 0, # interrupt enable
'DF' : bits & (0b1 << 10) != 0, # direction
'OF' : bits & (0b1 << 11) != 0 # overflow
}
@staticmethod
def get_iopl(bits: int) -> int:
return bits & (0b11 << 12)
class ArchX86(ArchIntel):
def __init__(self) -> None:
regs = (
'eax', 'ebx', 'ecx', 'edx',
'ebp', 'esp', 'esi', 'edi',
'eip', 'eflags' ,'ss', 'cs',
'ds', 'es', 'fs', 'gs'
)
super().__init__(regs, 4)
class ArchX64(ArchIntel):
def __init__(self) -> None:
regs = (
'rax', 'rbx', 'rcx', 'rdx',
'rbp', 'rsp', 'rsi', 'rdi',
'r8', 'r9', 'r10', 'r11',
'r12', 'r13', 'r14', 'r15',
'rip', 'eflags', 'ss', 'cs',
'ds', 'es', 'fs', 'gs'
)
super().__init__(regs, 8)
qiling/debugger/qdb/arch/arch_mips.py
+19 -21
View File
@@ -3,29 +3,27 @@
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from .arch import Arch
class ArchMIPS(Arch):
def __init__(self):
super().__init__()
def __init__(self) -> None:
regs = (
'gp', 'at', 'v0', 'v1',
'a0', 'a1', 'a2', 'a3',
't0', 't1', 't2', 't3',
't4', 't5', 't6', 't7',
't8', 't9', 'sp', 's8',
's0', 's1', 's2', 's3',
's4', 's5', 's6', 's7',
'ra', 'k0', 'k1', 'pc'
)
@property
def regs(self):
return (
"gp", "at", "v0", "v1",
"a0", "a1", "a2", "a3",
"t0", "t1", "t2", "t3",
"t4", "t5", "t6", "t7",
"t8", "t9", "sp", "s8",
"s0", "s1", "s2", "s3",
"s4", "s5", "s6", "s7",
"ra", "k0", "k1", "pc",
)
aliases = {
's8': 'fp'
}
@property
def regs_need_swapped(self):
return {
"fp": "s8",
}
asize = 4
isize = 4
super().__init__(regs, aliases, asize, isize)
qiling/debugger/qdb/arch/arch_x86.py
-47
View File
@@ -1,47 +0,0 @@
#!/usr/bin/env python3
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from typing import Mapping
from .arch import Arch
class ArchX86(Arch):
def __init__(self):
super().__init__()
@property
def arch_insn_size(self):
return 15
@property
def regs(self):
return (
"eax", "ebx", "ecx", "edx",
"esp", "ebp", "esi", "edi",
"eip", "ss", "cs", "ds", "es",
"fs", "gs", "eflags",
)
def read_insn(self, address: int) -> bytes:
# due to the variadic lengh of x86 instructions ( 1~15 )
# always assume the maxium size for disassembler to tell
# what is it exactly.
return self.read_mem(address, self.arch_insn_size)
@staticmethod
def get_flags(bits: int) -> Mapping[str, bool]:
"""
get flags from ql.reg.eflags
"""
return {
"CF" : bits & 0x0001 != 0, # CF, carry flag
"PF" : bits & 0x0004 != 0, # PF, parity flag
"AF" : bits & 0x0010 != 0, # AF, adjust flag
"ZF" : bits & 0x0040 != 0, # ZF, zero flag
"SF" : bits & 0x0080 != 0, # SF, sign flag
"OF" : bits & 0x0800 != 0, # OF, overflow flag
}
qiling/debugger/qdb/arch/arch_x8664.py
-66
View File
@@ -1,66 +0,0 @@
#!/usr/bin/env python3
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from typing import Mapping
from .arch import Arch
class ArchX8664(Arch):
'''
This is currently mostly just a copy of x86 - other than the size of archbits. Some of this may be wrong.
'''
def __init__(self):
super().__init__()
@property
def arch_insn_size(self):
'''
Architecture maximum instruction size. x86_64 instructions are a maximum size of 15 bytes.
@returns bytes
'''
return 15
@property
def regs(self):
return (
"rax", "rbx", "rcx", "rdx",
"rsp", "rbp", "rsi", "rdi",
"rip", "r8", "r9", "r10",
"r11", "r12", "r13", "r14",
"r15", "ss", "cs", "ds", "es",
"fs", "gs", "eflags"
)
@property
def archbit(self):
'''
Architecture maximum register size. x86 is a maximum of 4 bytes.
@returns bytes
'''
return 8
def read_insn(self, address: int) -> bytes:
# Due to the variadicc length of x86 instructions
# always assume the maximum size for disassembler to tell
# what it is.
return self.read_mem(address, self.arch_insn_size)
@staticmethod
def get_flags(bits: int) -> Mapping[str, bool]:
return {
"CF" : bits & 0x0001 != 0, # CF, carry flag
"PF" : bits & 0x0004 != 0, # PF, parity flag
"AF" : bits & 0x0010 != 0, # AF, adjust flag
"ZF" : bits & 0x0040 != 0, # ZF, zero flag
"SF" : bits & 0x0080 != 0, # SF, sign flag
"OF" : bits & 0x0800 != 0, # OF, overflow flag
}
qiling/debugger/qdb/branch_predictor/__init__.py
+9 -3
View File
@@ -4,7 +4,13 @@
#
from .branch_predictor import BranchPredictor
from .branch_predictor_x86 import BranchPredictorX86
from .branch_predictor_mips import BranchPredictorMIPS
from .branch_predictor_arm import BranchPredictorARM, BranchPredictorCORTEX_M
from .branch_predictor_x8664 import BranchPredictorX8664
from .branch_predictor_intel import BranchPredictorX86, BranchPredictorX64
from .branch_predictor_mips import BranchPredictorMIPS
__all__ = [
'BranchPredictor',
'BranchPredictorARM', 'BranchPredictorCORTEX_M',
'BranchPredictorX86', 'BranchPredictorX64',
'BranchPredictorMIPS'
]
qiling/debugger/qdb/branch_predictor/branch_predictor.py
+63 -18
View File
@@ -4,37 +4,82 @@
#
from abc import abstractmethod
from typing import ClassVar, NamedTuple, Optional
from capstone import CS_GRP_JUMP, CS_GRP_CALL, CS_GRP_RET, CS_GRP_BRANCH_RELATIVE
from ..context import Context
from ..misc import InvalidInsn
class Prophecy:
"""
container for storing result of the predictor
@going: indicate the certian branch will be taken or not
@where: where will it go if going is true
class Prophecy(NamedTuple):
"""Simple container for storing prediction results.
"""
def __init__(self):
self.going = False
self.where = None
going: bool
"""Indicate whether the certian branch is taken or not.
"""
where: Optional[int]
"""Branch target in case it is taken.
Target may be `None` if it should have been read from memory, but that memory location
could not be reached.
"""
def __iter__(self):
return iter((self.going, self.where))
class BranchPredictor(Context):
"""
Base class for predictor
"""Branch predictor base class.
"""
def read_reg(self, reg_name):
"""
read specific register value
stop: ClassVar[str]
"""Instruction mnemonic that can be used to determine program's end.
"""
def has_ended(self) -> bool:
"""Determine whether the program has ended by inspecting the currnet instruction.
"""
return self.ql.arch.regs.read(reg_name)
insn = self.disasm_lite(self.cur_addr)
if not insn:
return False
# (address, size, mnemonic, op_str)
return insn[2] == self.stop
def is_branch(self) -> bool:
"""Determine whether the current instruction is a branching instruction.
This does not provide indication whether the branch is going to be taken or not.
"""
insn = self.disasm(self.cur_addr, True)
# invalid instruction; definitely not a branch
if isinstance(insn, InvalidInsn):
return False
branching = (
CS_GRP_JUMP,
CS_GRP_CALL,
CS_GRP_RET,
CS_GRP_BRANCH_RELATIVE
)
return any(grp in branching for grp in insn.groups)
def is_fcall(self) -> bool:
"""Determine whether the current instruction is a function call.
"""
insn = self.disasm(self.cur_addr, True)
# invalid instruction; definitely not a function call
if isinstance(insn, InvalidInsn):
return False
return insn.group(CS_GRP_CALL)
@abstractmethod
def predict(self) -> Prophecy:
"""
Try to predict certian branch will be taken or not based on current context
"""Predict whether a certian branch will be taken or not based on current context.
"""
qiling/debugger/qdb/branch_predictor/branch_predictor_arm.py
+198 -189
View File
@@ -3,255 +3,264 @@
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from typing import Callable, Dict, List, Optional, Tuple
from capstone import CS_OP_IMM, CS_OP_MEM, CS_OP_REG
from capstone.arm import ArmOp, ArmOpMem
from capstone.arm_const import (
ARM_CC_EQ, ARM_CC_NE, ARM_CC_HS, ARM_CC_LO,
ARM_CC_MI, ARM_CC_PL, ARM_CC_VS, ARM_CC_VC,
ARM_CC_HI, ARM_CC_LS, ARM_CC_GE, ARM_CC_LT,
ARM_CC_GT, ARM_CC_LE, ARM_CC_AL
)
from .branch_predictor import *
from ..arch import ArchARM
from ..misc import read_int
from unicorn.arm_const import UC_ARM_REG_PC
from .branch_predictor import BranchPredictor, Prophecy
from ..arch import ArchARM, ArchCORTEX_M
from ..misc import InvalidInsn
class BranchPredictorARM(BranchPredictor, ArchARM):
"""
predictor for ARM
"""Branch Predictor for ARM.
"""
def __init__(self, ql):
super().__init__(ql)
ArchARM.__init__(self)
stop = 'udf'
self.INST_SIZE = 4
self.THUMB_INST_SIZE = 2
self.CODE_END = "udf"
def read_reg(self, reg_name):
reg_name = reg_name.replace("ip", "r12").replace("fp", "r11")
return getattr(self.ql.arch.regs, reg_name)
def regdst_eq_pc(self, op_str):
return op_str.partition(", ")[0] == "pc"
@staticmethod
def get_cpsr(bits: int) -> (bool, bool, bool, bool):
"""
get flags from ql.reg.cpsr
def get_cond_flags(self) -> Tuple[bool, bool, bool, bool]:
"""Get condition status flags from CPSR / xPSR.
"""
cpsr = self.read_reg(self._flags_reg)
return (
bits & 0x10000000 != 0, # V, overflow flag
bits & 0x20000000 != 0, # C, carry flag
bits & 0x40000000 != 0, # Z, zero flag
bits & 0x80000000 != 0, # N, sign flag
)
(cpsr & (0b1 << 28)) != 0, # V, overflow flag
(cpsr & (0b1 << 29)) != 0, # C, carry flag
(cpsr & (0b1 << 30)) != 0, # Z, zero flag
(cpsr & (0b1 << 31)) != 0 # N, sign flag
)
def predict(self, pref_addr=None):
prophecy = Prophecy()
cur_addr = self.cur_addr if pref_addr is None else pref_addr
line = self.disasm(cur_addr)
def predict(self) -> Prophecy:
insn = self.disasm(self.cur_addr, True)
if line.mnemonic == self.CODE_END: # indicates program exited
prophecy.where = True
return prophecy
going = False
where = 0
jump_table = {
# unconditional branch
"b" : (lambda *_: True),
"bl" : (lambda *_: True),
"bx" : (lambda *_: True),
"blx" : (lambda *_: True),
"b.w" : (lambda *_: True),
# invalid instruction; nothing to predict
if isinstance(insn, InvalidInsn):
return Prophecy(going, where)
# branch on equal, Z == 1
"beq" : (lambda V, C, Z, N: Z == 1),
"bxeq" : (lambda V, C, Z, N: Z == 1),
"beq.w": (lambda V, C, Z, N: Z == 1),
# iname is the instruction's basename stripped from all optional suffixes.
# this greatly simplifies the case handling
iname: str = insn.insn_name() or ''
operands: List[ArmOp] = insn.operands
# branch on not equal, Z == 0
"bne" : (lambda V, C, Z, N: Z == 0),
"bxne" : (lambda V, C, Z, N: Z == 0),
"bne.w": (lambda V, C, Z, N: Z == 0),
# branch instructions
branches = ('b', 'bl', 'bx', 'blx')
# branch on signed greater than, Z == 0 and N == V
"bgt" : (lambda V, C, Z, N: (Z == 0 and N == V)),
"bgt.w": (lambda V, C, Z, N: (Z == 0 and N == V)),
# reg-based conditional branches
conditional_reg: Dict[str, Callable[[int], bool]] = {
'cbz' : lambda r: r == 0,
'cbnz': lambda r: r != 0
}
# branch on signed less than, N != V
"blt" : (lambda V, C, Z, N: N != V),
def __read_reg(reg: int) -> Optional[int]:
"""[internal] Read register value where register is provided as a Unicorn constant.
"""
# branch on signed greater than or equal, N == V
"bge" : (lambda V, C, Z, N: N == V),
# name will be None in case of an invalid register. this is expected in some cases
# and should not raise an exception, but rather silently dropped
name = insn.reg_name(reg)
# branch on signed less than or queal
"ble" : (lambda V, C, Z, N: Z == 1 or N != V),
# pc reg value needs adjustment
adj = (2 * self.isize) if reg == UC_ARM_REG_PC else 0
# branch on unsigned higher or same (or carry set), C == 1
"bhs" : (lambda V, C, Z, N: C == 1),
"bcs" : (lambda V, C, Z, N: C == 1),
return name and self.read_reg(self.unalias(name)) + adj
# branch on unsigned lower (or carry clear), C == 0
"bcc" : (lambda V, C, Z, N: C == 0),
"blo" : (lambda V, C, Z, N: C == 0),
"bxlo" : (lambda V, C, Z, N: C == 0),
"blo.w": (lambda V, C, Z, N: C == 0),
def __read_mem(mem: ArmOpMem, size: int = 0, *, signed: bool = False) -> Optional[int]:
"""[internal] Attempt to read memory contents. By default memory accesses are in
native size and values are unsigned.
"""
# branch on negative or minus, N == 1
"bmi" : (lambda V, C, Z, N: N == 1),
base = __read_reg(mem.base) or 0
index = __read_reg(mem.index) or 0
scale = mem.scale
disp = mem.disp
# branch on positive or plus, N == 0
"bpl" : (lambda V, C, Z, N: N == 0),
return self.try_read_pointer(base + index * scale + disp, size, signed=signed)
# branch on signed overflow
"bvs" : (lambda V, C, Z, N: V == 1),
def __parse_op(op: ArmOp, *args, **kwargs) -> Optional[int]:
"""[internal] Parse an operand and return its value. Register references will be
substitued with the corresponding register value, while memory dereferences will
be substitued by the effective address they refer to.
"""
# branch on no signed overflow
"bvc" : (lambda V, C, Z, N: V == 0),
if op.type == CS_OP_REG:
value = __read_reg(op.reg)
# branch on unsigned higher
"bhi" : (lambda V, C, Z, N: (Z == 0 and C == 1)),
"bxhi" : (lambda V, C, Z, N: (Z == 0 and C == 1)),
"bhi.w": (lambda V, C, Z, N: (Z == 0 and C == 1)),
elif op.type == CS_OP_IMM:
value = op.imm
# branch on unsigned lower
"bls" : (lambda V, C, Z, N: (C == 0 or Z == 1)),
"bls.w": (lambda V, C, Z, N: (C == 0 or Z == 1)),
}
elif op.type == CS_OP_MEM:
value = __read_mem(op.mem, *args, **kwargs)
cb_table = {
# branch on equal to zero
"cbz" : (lambda r: r == 0),
# branch on not equal to zero
"cbnz": (lambda r: r != 0),
}
if line.mnemonic in jump_table:
prophecy.going = jump_table.get(line.mnemonic)(*self.get_cpsr(self.ql.arch.regs.cpsr))
elif line.mnemonic in cb_table:
prophecy.going = cb_table.get(line.mnemonic)(self.read_reg(line.op_str.split(", ")[0]))
if prophecy.going:
if "#" in line.op_str:
prophecy.where = read_int(line.op_str.split("#")[-1])
else:
prophecy.where = self.read_reg(line.op_str)
# we are not expecting any other operand type, including floating point (CS_OP_FP)
raise RuntimeError(f'unexpected operand type: {op.type}')
if self.regdst_eq_pc(line.op_str):
next_addr = cur_addr + line.size
n2_addr = next_addr + len(self.read_insn(next_addr))
prophecy.where += len(self.read_insn(n2_addr)) + len(self.read_insn(next_addr))
# LSR
if op.shift.type == 1:
value *= (1 >> op.shift.value)
elif line.mnemonic.startswith("it"):
# handle IT block here
# LSL
elif op.shift.type == 2:
value *= (1 << op.shift.value)
cond_met = {
"eq": lambda V, C, Z, N: (Z == 1),
"ne": lambda V, C, Z, N: (Z == 0),
"ge": lambda V, C, Z, N: (N == V),
"hs": lambda V, C, Z, N: (C == 1),
"lo": lambda V, C, Z, N: (C == 0),
"mi": lambda V, C, Z, N: (N == 1),
"pl": lambda V, C, Z, N: (N == 0),
"ls": lambda V, C, Z, N: (C == 0 or Z == 1),
"le": lambda V, C, Z, N: (Z == 1 or N != V),
"hi": lambda V, C, Z, N: (Z == 0 and C == 1),
}.get(line.op_str)(*self.get_cpsr(self.ql.arch.regs.cpsr))
# ROR ?
it_block_range = [each_char for each_char in line.mnemonic[1:]]
return value
next_addr = cur_addr + self.THUMB_INST_SIZE
for each in it_block_range:
_insn = self.read_insn(next_addr)
n2_addr = self.predict(ql, next_addr)
def __is_taken(cc: int) -> Tuple[bool, Tuple[bool, ...]]:
pred = predicate[cc]
flags = self.get_cond_flags()
if (cond_met and each == "t") or (not cond_met and each == "e"):
if n2_addr != (next_addr+len(_insn)): # branch detected
break
return pred(*flags), flags
next_addr += len(_insn)
# conditions predicate selector
predicate: Dict[int, Callable[..., bool]] = {
ARM_CC_EQ: lambda V, C, Z, N: Z,
ARM_CC_NE: lambda V, C, Z, N: not Z,
ARM_CC_HS: lambda V, C, Z, N: C,
ARM_CC_LO: lambda V, C, Z, N: not C,
ARM_CC_MI: lambda V, C, Z, N: N,
ARM_CC_PL: lambda V, C, Z, N: not N,
ARM_CC_VS: lambda V, C, Z, N: V,
ARM_CC_VC: lambda V, C, Z, N: not V,
ARM_CC_HI: lambda V, C, Z, N: (not Z) and C,
ARM_CC_LS: lambda V, C, Z, N: (not C) or Z,
ARM_CC_GE: lambda V, C, Z, N: (N == V),
ARM_CC_LT: lambda V, C, Z, N: (N != V),
ARM_CC_GT: lambda V, C, Z, N: not Z and (N == V),
ARM_CC_LE: lambda V, C, Z, N: Z or (N != V),
ARM_CC_AL: lambda V, C, Z, N: True
}
prophecy.where = next_addr
# implementation of simple binary arithmetic and bitwise operations
binop: Dict[str, Callable[[int, int, int], int]] = {
'add': lambda a, b, _: a + b,
'adc': lambda a, b, c: a + b + c,
'sub': lambda a, b, _: a - b,
'rsb': lambda a, b, _: b - a,
'sbc': lambda a, b, c: a - b - (1 - c),
'rsc': lambda a, b, c: b - a - (1 - c),
'mul': lambda a, b, _: a * b,
'and': lambda a, b, _: a & b,
'orr': lambda a, b, _: a | b,
'eor': lambda a, b, _: a ^ b
}
elif line.mnemonic in ("ldr",):
# is this a branch?
if iname in branches:
going, _ = __is_taken(insn.cc)
if self.regdst_eq_pc(line.op_str):
_, _, rn_offset = line.op_str.partition(", ")
r, _, imm = rn_offset.strip("[]!").partition(", #")
if going:
where = __parse_op(operands[0])
if "]" in rn_offset.split(", ")[1]: # pre-indexed immediate
prophecy.where = self.unpack32(self.read_mem(read_int(imm) + self.read_reg(r), self.INST_SIZE))
return Prophecy(going, where)
else: # post-indexed immediate
# FIXME: weired behavior, immediate here does not apply
prophecy.where = self.unpack32(self.read_mem(self.read_reg(r), self.INST_SIZE))
if iname in conditional_reg:
is_taken = conditional_reg[iname]
reg = __parse_op(operands[0])
assert reg is not None, 'unrecognized reg'
elif line.mnemonic in ("addls", "addne", "add") and self.regdst_eq_pc(line.op_str):
V, C, Z, N = self.get_cpsr(self.ql.arch.regs.cpsr)
r0, r1, r2, *imm = line.op_str.split(", ")
going = is_taken(reg)
# program counter is awalys 8 bytes ahead when it comes with pc, need to add extra 8 bytes
extra = 8 if 'pc' in (r0, r1, r2) else 0
if going:
where = __parse_op(operands[1])
if imm:
expr = imm[0].split()
# TODO: should support more bit shifting and rotating operation
if expr[0] == "lsl": # logical shift left
n = read_int(expr[-1].strip("#")) * 2
return Prophecy(going, where)
if line.mnemonic == "addls" and (C == 0 or Z == 1):
prophecy.where = extra + self.read_reg(r1) + self.read_reg(r2) * n
# instruction is not a branch; check whether pc is affected by this instruction.
#
# insn.regs_write doesn't work well, so we use insn.regs_access instead
if UC_ARM_REG_PC in insn.regs_access()[1]:
elif line.mnemonic == "add" or (line.mnemonic == "addne" and Z == 0):
prophecy.where = extra + self.read_reg(r1) + (self.read_reg(r2) * n if imm else self.read_reg(r2))
if iname == 'mov':
going = True
where = __parse_op(operands[1])
elif line.mnemonic in ("tbh", "tbb"):
elif iname.startswith('ldr'):
suffix: str = insn.mnemonic[3:]
cur_addr += self.INST_SIZE
r0, r1, *imm = line.op_str.strip("[]").split(", ")
# map possible ldr suffixes to kwargs required for the memory access.
#
# to improve readability we also address the case where ldr has no suffix
# and no special kwargs are required. all strings start with '', so it
# serves as a safe default case
msize: Dict[str, Dict] = {
'b' : {'size': 1, 'signed': False},
'h' : {'size': 2, 'signed': False},
'sb': {'size': 1, 'signed': True},
'sh': {'size': 2, 'signed': True},
'' : {}
}
if imm:
expr = imm[0].split()
if expr[0] == "lsl": # logical shift left
n = read_int(expr[-1].strip("#")) * 2
# ldr has different variations that affect the memory access size and
# whether the value should be signed or not.
suffix = next(s for s in msize if suffix.startswith(s))
if line.mnemonic == "tbh":
going, _ = __is_taken(insn.cc)
r1 = self.read_reg(r1) * n
if going:
where = __parse_op(operands[1], **msize[suffix])
elif line.mnemonic == "tbb":
elif iname in binop:
going, flags = __is_taken(insn.cc)
r1 = self.read_reg(r1)
if going:
operator = binop[iname]
op1 = __parse_op(operands[1])
op2 = __parse_op(operands[2])
carry = int(flags[1])
to_add = int.from_bytes(self.read_mem(cur_addr+r1, 2 if line.mnemonic == "tbh" else 1), byteorder="little") * n
prophecy.where = cur_addr + to_add
where = (op1 and op2) and operator(op1, op2, carry)
elif line.mnemonic.startswith("pop") and "pc" in line.op_str:
elif iname == 'pop':
going, _ = __is_taken(insn.cc)
prophecy.where = self.ql.stack_read(line.op_str.strip("{}").split(", ").index("pc") * self.INST_SIZE)
if not { # step to next instruction if cond does not meet
"pop" : lambda *_: True,
"pop.w": lambda *_: True,
"popeq": lambda V, C, Z, N: (Z == 1),
"popne": lambda V, C, Z, N: (Z == 0),
"pophi": lambda V, C, Z, N: (C == 1),
"popge": lambda V, C, Z, N: (N == V),
"poplt": lambda V, C, Z, N: (N != V),
}.get(line.mnemonic)(*self.get_cpsr(self.ql.arch.regs.cpsr)):
if going:
# find pc position within pop regs list
idx = next(i for i, op in enumerate(operands) if (op.type == CS_OP_REG) and (op.reg == UC_ARM_REG_PC))
prophecy.where = cur_addr + self.INST_SIZE
# read the corresponding stack entry
where = self.ql.stack_read(idx * self.asize)
elif line.mnemonic == "sub" and self.regdst_eq_pc(line.op_str):
_, r, imm = line.op_str.split(", ")
prophecy.where = self.read_reg(r) - read_int(imm.strip("#"))
else:
# left here for users to provide feedback when encountered
raise RuntimeWarning(f'instruction affects pc but was not considered: {insn.mnemonic}')
elif line.mnemonic == "mov" and self.regdst_eq_pc(line.op_str):
_, r = line.op_str.split(", ")
prophecy.where = self.read_reg(r)
# for some reason capstone does not consider pc to be affected by 'tbb' and 'tbh'
# so we need to test for them specifically
if prophecy.where is not None:
prophecy.where &= ~0b1
# table branch byte
elif iname == 'tbb':
offset = __read_mem(operands[0].mem, 1)
pc = __read_reg(UC_ARM_REG_PC)
return prophecy
going = True
where = (offset and pc) and (pc + offset * 2)
class BranchPredictorCORTEX_M(BranchPredictorARM):
def __init__(self, ql):
super().__init__(ql)
# table branch half-word
elif iname == 'tbh':
offset = __read_mem(operands[0].mem, 2)
pc = __read_reg(UC_ARM_REG_PC)
going = True
where = (offset and pc) and (pc + offset * 2)
return Prophecy(going, where)
class BranchPredictorCORTEX_M(BranchPredictorARM, ArchCORTEX_M):
"""Branch Predictor for ARM Cortex-M.
"""
qiling/debugger/qdb/branch_predictor/branch_predictor_intel.py
+181
View File
@@ -0,0 +1,181 @@
#!/usr/bin/env python3
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from typing import Callable, Dict, List, Optional, Tuple
from capstone.x86 import X86Op
from capstone.x86_const import X86_OP_REG, X86_OP_IMM, X86_OP_MEM, X86_INS_LEA
from .branch_predictor import Prophecy, BranchPredictor
from ..arch import ArchX86, ArchX64
from ..misc import InvalidInsn
class BranchPredictorIntel(BranchPredictor):
"""Branch Predictor base class for Intel architecture.
"""
stop = 'hlt'
def get_eflags(self) -> Tuple[int, int, int, int, int]:
eflags = self.read_reg('eflags')
return (
(eflags & (0b1 << 0)) != 0, # carry
(eflags & (0b1 << 2)) != 0, # parity
(eflags & (0b1 << 6)) != 0, # zero
(eflags & (0b1 << 7)) != 0, # sign
(eflags & (0b1 << 11)) != 0 # overflow
)
def predict(self) -> Prophecy:
insn = self.disasm(self.cur_addr, True)
going = False
where = 0
# invalid instruction; nothing to predict
if isinstance(insn, InvalidInsn):
return Prophecy(going, where)
mnem: str = insn.mnemonic
operands: List[X86Op] = insn.operands
# unconditional branches
unconditional = ('call', 'jmp')
# flags-based conditional branches
conditional: Dict[str, Callable[..., bool]] = {
'jb' : lambda C, P, Z, S, O: C,
'jc' : lambda C, P, Z, S, O: C,
'jnae': lambda C, P, Z, S, O: C,
'jnb' : lambda C, P, Z, S, O: not C,
'jnc' : lambda C, P, Z, S, O: not C,
'jae' : lambda C, P, Z, S, O: not C,
'jp' : lambda C, P, Z, S, O: P,
'jpe' : lambda C, P, Z, S, O: P,
'jnp' : lambda C, P, Z, S, O: not P,
'jpo' : lambda C, P, Z, S, O: not P,
'je' : lambda C, P, Z, S, O: Z,
'jz' : lambda C, P, Z, S, O: Z,
'jne' : lambda C, P, Z, S, O: not Z,
'jnz' : lambda C, P, Z, S, O: not Z,
'js' : lambda C, P, Z, S, O: S,
'jns' : lambda C, P, Z, S, O: not S,
'jo' : lambda C, P, Z, S, O: O,
'jno' : lambda C, P, Z, S, O: not O,
'jbe' : lambda C, P, Z, S, O: C or Z,
'jna' : lambda C, P, Z, S, O: C or Z,
'ja' : lambda C, P, Z, S, O: (not C) and (not Z),
'jnbe': lambda C, P, Z, S, O: (not C) and (not Z),
'jl' : lambda C, P, Z, S, O: S != O,
'jnge': lambda C, P, Z, S, O: S != O,
'jge' : lambda C, P, Z, S, O: S == O,
'jnl' : lambda C, P, Z, S, O: S == O,
'jle' : lambda C, P, Z, S, O: Z or (S != O),
'jng' : lambda C, P, Z, S, O: Z or (S != O),
'jg' : lambda C, P, Z, S, O: (not Z) or (not S),
'jnle': lambda C, P, Z, S, O: (not Z) or (not S)
}
# reg-based conditional branches
conditional_reg = {
"jcxz" : 'cx',
"jecxz" : 'ecx',
"jrcxz" : 'rcx'
}
def __read_reg(reg: int) -> Optional[int]:
"""Read register value where register is provided as a Unicorn constant.
"""
# name will be None in case of an illegal or unknown register
name = insn.reg_name(reg)
return name and self.read_reg(name)
def __parse_op(op: X86Op) -> Optional[int]:
"""Parse an operand and return its value. Memory dereferences will be
substitued by the effective address they refer to.
"""
if op.type == X86_OP_REG:
value = __read_reg(op.reg)
elif op.type == X86_OP_IMM:
value = op.imm
elif op.type == X86_OP_MEM:
mem = op.mem
base = __read_reg(mem.base) or 0
index = __read_reg(mem.index) or 0
scale = mem.scale
disp = mem.disp
seg = __read_reg(mem.segment) or 0
ea = seg * 0x10 + (base + index * scale + disp)
# lea does not really dereference memory
value = ea if insn.id == X86_INS_LEA else self.try_read_pointer(ea)
else:
raise RuntimeError(f'unexpected operand type: {op.type}')
return value
# is this an unconditional branch?
if mnem in unconditional:
going = True
where = __parse_op(operands[0])
# is this a return from a function call?
elif mnem == 'ret':
going = True
where = self.ql.arch.stack_read(0)
# is this a flags-based branch?
elif mnem in conditional:
predict = conditional[mnem]
eflags = self.get_eflags()
going = predict(*eflags)
if going:
where = __parse_op(operands[0])
elif mnem in conditional_reg:
reg = conditional_reg[mnem]
predict = lambda c: c == 0
going = predict(self.read_reg(reg))
if going:
where = __parse_op(operands[0])
return Prophecy(going, where)
class BranchPredictorX86(BranchPredictorIntel, ArchX86):
"""Branch Predictor for x86.
"""
class BranchPredictorX64(BranchPredictorIntel, ArchX64):
"""Branch Predictor for x86-64.
"""
qiling/debugger/qdb/branch_predictor/branch_predictor_mips.py
+77 -70
View File
@@ -3,88 +3,95 @@
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from typing import Optional
from capstone.mips import MipsOp, MIPS_OP_REG, MIPS_OP_IMM
from .branch_predictor import *
from .branch_predictor import BranchPredictor, Prophecy
from ..arch import ArchMIPS
from ..misc import InvalidInsn
class BranchPredictorMIPS(BranchPredictor, ArchMIPS):
"""
predictor for MIPS
"""Branch Predictor for MIPS 32.
"""
def __init__(self, ql):
super().__init__(ql)
ArchMIPS.__init__(self)
self.CODE_END = "break"
self.INST_SIZE = 4
@staticmethod
def signed_val(val: int) -> int:
"""
signed value convertion
"""
def is_negative(i: int) -> int:
"""
check wether negative value or not
"""
return i & (1 << 31)
return (val-1 << 32) if is_negative(val) else val
def read_reg(self, reg_name):
reg_name = reg_name.strip("$").replace("fp", "s8")
return self.signed_val(getattr(self.ql.arch.regs, reg_name))
stop = 'break'
def predict(self):
prophecy = Prophecy()
line = self.disasm(self.cur_addr)
insn = self.disasm(self.cur_addr, True)
if line.mnemonic == self.CODE_END: # indicates program extied
prophecy.where = True
return prophecy
going = False
where = 0
prophecy.where = self.cur_addr + self.INST_SIZE
if line.mnemonic.startswith('j') or line.mnemonic.startswith('b'):
# invalid instruction; nothing to predict
if isinstance(insn, InvalidInsn):
return Prophecy(going, where)
# make sure at least delay slot executed
prophecy.where += self.INST_SIZE
unconditional = ('j', 'jr', 'jal', 'jalr', 'b', 'bl', 'bal')
# get registers or memory address from op_str
targets = [
self.read_reg(each)
if '$' in each else read_int(each)
for each in line.op_str.split(", ")
]
conditional = {
'beq' : lambda r0, r1: r0 == r1, # branch on equal
'bne' : lambda r0, r1: r0 != r1, # branch on not equal
'blt' : lambda r0, r1: r0 < r1, # branch on r0 less than r1
'bgt' : lambda r0, r1: r0 > r1, # branch on r0 greater than r1
'ble' : lambda r0, r1: r0 <= r1, # branch on r0 less than or equal to r1
'bge' : lambda r0, r1: r0 >= r1, # branch on r0 greater than or equal to r1
prophecy.going = {
"j" : (lambda _: True), # unconditional jump
"jr" : (lambda _: True), # unconditional jump
"jal" : (lambda _: True), # unconditional jump
"jalr" : (lambda _: True), # unconditional jump
"b" : (lambda _: True), # unconditional branch
"bl" : (lambda _: True), # unconditional branch
"bal" : (lambda _: True), # unconditional branch
"beq" : (lambda r0, r1, _: r0 == r1), # branch on equal
"bne" : (lambda r0, r1, _: r0 != r1), # branch on not equal
"blt" : (lambda r0, r1, _: r0 < r1), # branch on r0 less than r1
"bgt" : (lambda r0, r1, _: r0 > r1), # branch on r0 greater than r1
"ble" : (lambda r0, r1, _: r0 <= r1), # brach on r0 less than or equal to r1
"bge" : (lambda r0, r1, _: r0 >= r1), # branch on r0 greater than or equal to r1
"beqz" : (lambda r, _: r == 0), # branch on equal to zero
"bnez" : (lambda r, _: r != 0), # branch on not equal to zero
"bgtz" : (lambda r, _: r > 0), # branch on greater than zero
"bltz" : (lambda r, _: r < 0), # branch on less than zero
"bltzal" : (lambda r, _: r < 0), # branch on less than zero and link
"blez" : (lambda r, _: r <= 0), # branch on less than or equal to zero
"bgez" : (lambda r, _: r >= 0), # branch on greater than or equal to zero
"bgezal" : (lambda r, _: r >= 0), # branch on greater than or equal to zero and link
}.get(line.mnemonic)(*targets)
'beqz' : lambda r: r == 0, # branch on equal to zero
'bnez' : lambda r: r != 0, # branch on not equal to zero
'bgtz' : lambda r: r > 0, # branch on greater than zero
'bltz' : lambda r: r < 0, # branch on less than zero
'bltzal': lambda r: r < 0, # branch on less than zero and link
'blez' : lambda r: r <= 0, # branch on less than or equal to zero
'bgez' : lambda r: r >= 0, # branch on greater than or equal to zero
'bgezal': lambda r: r >= 0 # branch on greater than or equal to zero and link
}
if prophecy.going:
# target address is always the rightmost one
prophecy.where = targets[-1]
def __as_signed(val: int) -> int:
"""Get the signed integer representation of a given value.
"""
return prophecy
msb = 0b1 << 31
return (val & ~msb) - (val & msb)
def __read_reg(reg: int) -> Optional[int]:
"""Read register value where register is provided as a Unicorn constant.
"""
# name will be None in case of an illegal or unknown register
name = insn.reg_name(reg)
return name and __as_signed(self.read_reg(self.unalias(name)))
def __parse_op(op: MipsOp) -> Optional[int]:
"""Parse an operand and return its value.
"""
if op.type == MIPS_OP_REG:
value = __read_reg(op.reg)
elif op.type == MIPS_OP_IMM:
value = op.imm
else:
raise RuntimeError(f'unexpected operand type: {op.type}')
return value
# get operands. target address is always the rightmost one
if insn.operands:
*operands, target = insn.operands
if insn.mnemonic in unconditional:
going = True
elif insn.mnemonic in conditional:
predict = conditional[insn.mnemonic]
going = predict(*(__parse_op(op) for op in operands))
if going:
where = __parse_op(target)
return Prophecy(going, where)
qiling/debugger/qdb/branch_predictor/branch_predictor_x86.py
-128
View File
@@ -1,128 +0,0 @@
#!/usr/bin/env python3
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
import re
from .branch_predictor import *
from ..arch import ArchX86
from ..misc import check_and_eval
class BranchPredictorX86(BranchPredictor, ArchX86):
"""
predictor for X86
"""
class ParseError(Exception):
"""
indicate parser error
"""
pass
def __init__(self, ql):
super().__init__(ql)
ArchX86.__init__(self)
def predict(self):
prophecy = Prophecy()
line = self.disasm(self.cur_addr)
jump_table = {
# conditional jump
"jo" : (lambda C, P, A, Z, S, O: O == 1),
"jno" : (lambda C, P, A, Z, S, O: O == 0),
"js" : (lambda C, P, A, Z, S, O: S == 1),
"jns" : (lambda C, P, A, Z, S, O: S == 0),
"je" : (lambda C, P, A, Z, S, O: Z == 1),
"jz" : (lambda C, P, A, Z, S, O: Z == 1),
"jne" : (lambda C, P, A, Z, S, O: Z == 0),
"jnz" : (lambda C, P, A, Z, S, O: Z == 0),
"jb" : (lambda C, P, A, Z, S, O: C == 1),
"jc" : (lambda C, P, A, Z, S, O: C == 1),
"jnae" : (lambda C, P, A, Z, S, O: C == 1),
"jnb" : (lambda C, P, A, Z, S, O: C == 0),
"jnc" : (lambda C, P, A, Z, S, O: C == 0),
"jae" : (lambda C, P, A, Z, S, O: C == 0),
"jbe" : (lambda C, P, A, Z, S, O: C == 1 or Z == 1),
"jna" : (lambda C, P, A, Z, S, O: C == 1 or Z == 1),
"ja" : (lambda C, P, A, Z, S, O: C == 0 and Z == 0),
"jnbe" : (lambda C, P, A, Z, S, O: C == 0 and Z == 0),
"jl" : (lambda C, P, A, Z, S, O: S != O),
"jnge" : (lambda C, P, A, Z, S, O: S != O),
"jge" : (lambda C, P, A, Z, S, O: S == O),
"jnl" : (lambda C, P, A, Z, S, O: S == O),
"jle" : (lambda C, P, A, Z, S, O: Z == 1 or S != O),
"jng" : (lambda C, P, A, Z, S, O: Z == 1 or S != O),
"jg" : (lambda C, P, A, Z, S, O: Z == 0 or S == O),
"jnle" : (lambda C, P, A, Z, S, O: Z == 0 or S == O),
"jp" : (lambda C, P, A, Z, S, O: P == 1),
"jpe" : (lambda C, P, A, Z, S, O: P == 1),
"jnp" : (lambda C, P, A, Z, S, O: P == 0),
"jpo" : (lambda C, P, A, Z, S, O: P == 0),
# unconditional jump
"call" : (lambda *_: True),
"jmp" : (lambda *_: True),
}
jump_reg_table = {
"jcxz" : (lambda cx: cx == 0),
"jecxz" : (lambda ecx: ecx == 0),
"jrcxz" : (lambda rcx: rcx == 0),
}
if line.mnemonic in jump_table:
eflags = self.get_flags(self.ql.arch.regs.eflags).values()
prophecy.going = jump_table.get(line.mnemonic)(*eflags)
elif line.mnemonic in jump_reg_table:
prophecy.going = jump_reg_table.get(line.mnemonic)(self.ql.arch.regs.ecx)
if prophecy.going:
takeaway_list = ["ptr", "dword", "[", "]"]
if len(line.op_str.split()) > 1:
new_line = line.op_str.replace(":", "+")
for each in takeaway_list:
new_line = new_line.replace(each, " ")
new_line = " ".join(new_line.split())
for each_reg in filter(lambda r: len(r) == 3, self.ql.arch.regs.register_mapping.keys()):
if each_reg in new_line:
new_line = re.sub(each_reg, hex(self.read_reg(each_reg)), new_line)
for each_reg in filter(lambda r: len(r) == 2, self.ql.arch.regs.register_mapping.keys()):
if each_reg in new_line:
new_line = re.sub(each_reg, hex(self.read_reg(each_reg)), new_line)
prophecy.where = check_and_eval(new_line)
elif line.op_str in self.ql.arch.regs.register_mapping:
prophecy.where = self.ql.arch.regs.read(line.op_str)
else:
prophecy.where = read_int(line.op_str)
else:
prophecy.where = self.cur_addr + line.size
return prophecy
qiling/debugger/qdb/branch_predictor/branch_predictor_x8664.py
-127
View File
@@ -1,127 +0,0 @@
#!/usr/bin/env python3
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
import re
from .branch_predictor import *
from ..arch import ArchX8664
from ..misc import check_and_eval
class BranchPredictorX8664(BranchPredictor, ArchX8664):
"""
predictor for X86
"""
class ParseError(Exception):
"""
indicate parser error
"""
pass
def __init__(self, ql):
super().__init__(ql)
ArchX8664.__init__(self)
def predict(self):
prophecy = Prophecy()
line = self.disasm(self.cur_addr)
jump_table = {
# conditional jump
"jo" : (lambda C, P, A, Z, S, O: O == 1),
"jno" : (lambda C, P, A, Z, S, O: O == 0),
"js" : (lambda C, P, A, Z, S, O: S == 1),
"jns" : (lambda C, P, A, Z, S, O: S == 0),
"je" : (lambda C, P, A, Z, S, O: Z == 1),
"jz" : (lambda C, P, A, Z, S, O: Z == 1),
"jne" : (lambda C, P, A, Z, S, O: Z == 0),
"jnz" : (lambda C, P, A, Z, S, O: Z == 0),
"jb" : (lambda C, P, A, Z, S, O: C == 1),
"jc" : (lambda C, P, A, Z, S, O: C == 1),
"jnae" : (lambda C, P, A, Z, S, O: C == 1),
"jnb" : (lambda C, P, A, Z, S, O: C == 0),
"jnc" : (lambda C, P, A, Z, S, O: C == 0),
"jae" : (lambda C, P, A, Z, S, O: C == 0),
"jbe" : (lambda C, P, A, Z, S, O: C == 1 or Z == 1),
"jna" : (lambda C, P, A, Z, S, O: C == 1 or Z == 1),
"ja" : (lambda C, P, A, Z, S, O: C == 0 and Z == 0),
"jnbe" : (lambda C, P, A, Z, S, O: C == 0 and Z == 0),
"jl" : (lambda C, P, A, Z, S, O: S != O),
"jnge" : (lambda C, P, A, Z, S, O: S != O),
"jge" : (lambda C, P, A, Z, S, O: S == O),
"jnl" : (lambda C, P, A, Z, S, O: S == O),
"jle" : (lambda C, P, A, Z, S, O: Z == 1 or S != O),
"jng" : (lambda C, P, A, Z, S, O: Z == 1 or S != O),
"jg" : (lambda C, P, A, Z, S, O: Z == 0 or S == O),
"jnle" : (lambda C, P, A, Z, S, O: Z == 0 or S == O),
"jp" : (lambda C, P, A, Z, S, O: P == 1),
"jpe" : (lambda C, P, A, Z, S, O: P == 1),
"jnp" : (lambda C, P, A, Z, S, O: P == 0),
"jpo" : (lambda C, P, A, Z, S, O: P == 0),
# unconditional jump
"call" : (lambda *_: True),
"jmp" : (lambda *_: True),
}
jump_reg_table = {
"jcxz" : (lambda cx: cx == 0),
"jecxz" : (lambda ecx: ecx == 0),
"jrcxz" : (lambda rcx: rcx == 0),
}
if line.mnemonic in jump_table:
eflags = self.get_flags(self.ql.arch.regs.eflags).values()
prophecy.going = jump_table.get(line.mnemonic)(*eflags)
elif line.mnemonic in jump_reg_table:
prophecy.going = jump_reg_table.get(line.mnemonic)(self.ql.arch.regs.ecx)
if prophecy.going:
takeaway_list = ["ptr", "dword", "qword", "[", "]"]
if len(line.op_str.split()) > 1:
new_line = line.op_str.replace(":", "+")
for each in takeaway_list:
new_line = new_line.replace(each, " ")
new_line = " ".join(new_line.split())
for each_reg in filter(lambda r: len(r) == 3, self.ql.arch.regs.register_mapping.keys()):
if each_reg in new_line:
new_line = re.sub(each_reg, hex(self.read_reg(each_reg)), new_line)
for each_reg in filter(lambda r: len(r) == 2, self.ql.arch.regs.register_mapping.keys()):
if each_reg in new_line:
new_line = re.sub(each_reg, hex(self.read_reg(each_reg)), new_line)
prophecy.where = check_and_eval(new_line)
elif line.op_str in self.ql.arch.regs.register_mapping:
prophecy.where = self.ql.arch.regs.read(line.op_str)
else:
prophecy.where = read_int(line.op_str)
else:
prophecy.where = self.cur_addr + line.size
return prophecy
qiling/debugger/qdb/const.py
+19 -17
View File
@@ -1,23 +1,25 @@
from enum import IntEnum
class color:
"""
class for colorful prints
"""
CYAN = '\033[96m'
PURPLE = '\033[95m'
BLUE = '\033[94m'
YELLOW = '\033[93m'
GREEN = '\033[92m'
RED = '\033[91m'
DARKGRAY = '\033[90m'
WHITE = '\033[48m'
DARKCYAN = '\033[36m'
BLACK = '\033[35m'
UNDERLINE = '\033[4m'
BOLD = '\033[1m'
END = '\033[0m'
RESET = '\x1b[39m'
"""
class for colorful prints
"""
DARKGRAY = '\033[90m'
RED = '\033[91m'
GREEN = '\033[92m'
YELLOW = '\033[93m'
BLUE = '\033[94m'
PURPLE = '\033[95m'
CYAN = '\033[96m'
WHITE = '\033[48m'
BLACK = '\033[35m'
DARKCYAN = '\033[36m'
UNDERLINE = '\033[4m'
BOLD = '\033[1m'
END = '\033[0m'
RESET = '\033[39m'
class QDB_MSG(IntEnum):
ERROR = 10
qiling/debugger/qdb/context.py
+104 -59
View File
@@ -3,102 +3,147 @@
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from typing import Optional
from __future__ import annotations
from unicorn import UC_ERR_READ_UNMAPPED
import unicorn
from typing import TYPE_CHECKING, Optional, Tuple, Union
from unicorn import UcError
from capstone import CsInsn
from .misc import InvalidInsn
if TYPE_CHECKING:
from qiling import Qiling
from .misc import InsnLike
from .misc import read_int, InvalidInsn
class Context:
"""
base class for accessing context of running qiling instance
"""Emulation context accessor.
"""
def __init__(self, ql):
def __init__(self, ql: Qiling):
# make sure mixin classes are properly initialized
super().__init__()
self.ql = ql
self.pointersize = self.ql.arch.pointersize
self.unpack = ql.unpack
self.unpack16 = ql.unpack16
self.unpack32 = ql.unpack32
self.unpack64 = ql.unpack64
@property
def cur_addr(self):
"""
program counter of qiling instance
def cur_addr(self) -> int:
"""Read current program counter register.
"""
return self.ql.arch.regs.arch_pc
def read_mem(self, address: int, size: int):
@property
def cur_sp(self) -> int:
"""Read current stack pointer register.
"""
read data from memory of qiling instance
return self.ql.arch.regs.arch_sp
def read_reg(self, reg: Union[str, int]) -> int:
"""Get register value.
"""
return self.ql.arch.regs.read(reg)
def write_reg(self, reg: Union[str, int], value: int) -> None:
"""Set register value.
"""
self.ql.arch.regs.write(reg, value)
def disasm(self, address: int, detail: bool = False) -> InsnLike:
"""Helper function for disassembling.
"""
insn_bytes = self.read_insn(address) or b''
insn = None
if insn_bytes:
md = self.ql.arch.disassembler
md.detail = detail
insn = next(md.disasm(insn_bytes, address, 1), None)
return insn or InvalidInsn(insn_bytes, address)
def disasm_lite(self, address: int) -> Tuple[int, int, str, str]:
"""Helper function for light disassembling, when details are not required.
Returns:
A tuple of: instruction address, size, mnemonic and operands
"""
insn_bytes = self.read_insn(address) or b''
insn = None
if insn_bytes:
md = self.ql.arch.disassembler
insn = next(md.disasm_lite(insn_bytes, address, 1), None)
return insn or tuple()
def read_mem(self, address: int, size: int) -> bytearray:
"""Read data of a certain size from specified memory location.
"""
return self.ql.mem.read(address, size)
def disasm(self, address: int, detail: bool = False) -> Optional[CsInsn]:
"""
helper function for disassembling
def try_read_mem(self, address: int, size: int) -> Optional[bytearray]:
"""Attempt to read data from memory.
"""
md = self.ql.arch.disassembler
md.detail = detail
if (bytes_read := self.read_insn(address)):
return next(md.disasm(bytes_read, address), InvalidInsn(bytes_read, address))
return InvalidInsn(bytes_read, address)
def try_read(self, address: int, size: int) -> Optional[bytes]:
"""
try to read data from ql.mem
"""
result = None
err_msg = ""
try:
result = self.read_mem(address, size)
data = self.read_mem(address, size)
except UcError:
data = None
except unicorn.unicorn.UcError as err:
if err.errno == UC_ERR_READ_UNMAPPED: # Invalid memory read (UC_ERR_READ_UNMAPPED)
err_msg = f"Can not access memory at address 0x{address:08x}"
return data
except:
pass
return (result, err_msg)
def try_read_pointer(self, address: int) -> Optional[bytes]:
"""
try to read pointer size of data from ql.mem
def read_pointer(self, address: int, size: int = 0, *, signed: bool = False) -> int:
"""Attempt to read a native-size integer from memory.
"""
return self.try_read(address, self.archbit)
return self.ql.mem.read_ptr(address, size, signed=signed)
def try_read_pointer(self, address: int, size: int = 0, *, signed: bool = False) -> Optional[int]:
"""Attempt to read a native-size integer from memory.
"""
try:
value = self.read_pointer(address, size, signed=signed)
except UcError:
value = None
return value
def read_string(self, address: int) -> Optional[str]:
"""
read string from memory of qiling instance
"""Read string from memory.
"""
return self.ql.mem.string(address)
def try_read_string(self, address: int) -> Optional[str]:
"""
try to read string from memory of qiling instance
"""Attempt to read a string from memory.
"""
s = None
try:
s = self.read_string(address)
except:
pass
except UcError:
s = None
@staticmethod
def read_int(s: str) -> int:
return read_int(s)
return s
if __name__ == "__main__":
pass
def get_deref(self, ptr: int) -> Union[int, str, None]:
"""Get content referenced by a pointer.
If dereferenced data is printable, a string will be returned. Otherwise
an integer value is retgurned. If the specified address is not reachable
None is returned.
"""
val = self.try_read_string(ptr)
return val if val and val.isprintable() else self.try_read_pointer(ptr)
qiling/debugger/qdb/helper.py
+252
View File
@@ -0,0 +1,252 @@
#!/usr/bin/env python3
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from __future__ import annotations
import re
from typing import TYPE_CHECKING, List, Tuple
from qiling.const import QL_ARCH
from .context import Context
from .arch import ArchCORTEX_M, ArchARM, ArchMIPS, ArchX86, ArchX64
if TYPE_CHECKING:
from re import Match
from qiling import Qiling
from .misc import InsnLike
def setup_command_helper(ql: Qiling):
atypes = {
QL_ARCH.X86: ArchX86,
QL_ARCH.X8664: ArchX64,
QL_ARCH.MIPS: ArchMIPS,
QL_ARCH.ARM: ArchARM,
QL_ARCH.CORTEX_M: ArchCORTEX_M
}
ret = type('CommandHelper', (CommandHelper, atypes[ql.arch.type]), {})
return ret(ql)
# pre-compile the safe arithmetics and bitwise pattern
__arith_pattern = re.compile(r'^(0[xX][0-9a-fA-F]+|0[0-7]+|\d+|[\+\-\*/\(\)|&^~\s])+$')
def safe_arith(expr: str) -> int:
"""Safely evaluate an arithmetic expression. The expression may include only
digits, arithmetic and bitwise operators, parantheses, whitespaces, hexadecimal
and octal values.
Args:
expr: arithmetic expression to evaluate
Returns: integer result
Raises:
ValueError: if disallowed tokens are included in `expr`
SyntaxError: in case the arithmetic expression does not make sense
"""
if not __arith_pattern.fullmatch(expr):
raise ValueError
# adjust gdb-style octal values to python: 0644 -> 0o644
re.sub(r'0([0-7]+)', r'0o\1', expr)
# safely evaluate the expression
return eval(expr, {}, {})
class CommandHelper(Context):
"""
memory manager for handing memory access
"""
def __init__(self, ql: Qiling):
super().__init__(ql)
# default values for the examine ('x') command
self.x_defaults = {
'n': '1', # number of units to read
'f': 'x', # output format
'u': 'w' # unit type
}
def sub_reg_values(self, expr: str) -> str:
def __sub_reg(m: Match[str]) -> str:
reg = m.group(1).lower()
return f'{self.read_reg(self.unalias(reg)):#x}'
# replace reg names with their actual values
return re.sub(r'\$(\w+)', __sub_reg, expr)
def resolve_expr(self, expr: str) -> int:
"""Resolve an arithmetic expression that might include register names.
Registers names will be substituted with their current value before
proceeding to evaluate the expression.
Args:
expr: an expression to evaluate
Returns:
final evaluation result
Raises:
KeyError: if `expr` contains an unrecognized register name
ValueError: if `expr` contains disallowed tokens
SyntaxError: if `expr` contains a broken arithmetic syntax
"""
try:
# look for registers names and replace them with their actual values
expr = self.sub_reg_values(expr)
# expr contains an unrecognized register name
except KeyError as ex:
raise KeyError(f'unrecognized register name: {ex.args[0]}') from ex
try:
# expr should contain only values and aithmetic tokens by now; attempt to evaluate it
res = safe_arith(expr)
# expr contains a disallowed token
except ValueError as ex:
raise ValueError('only integers, hexadecimals, octals, arithmetic and bitwise operators are allowed') from ex
# arithmetic syntax is broken
except SyntaxError as ex:
raise SyntaxError('error evaluating arithmetic expression') from ex
return res
def handle_set(self, line: str) -> Tuple[str, int]:
"""
set register value of current context
"""
# set $a = b
m = re.match(r'\s*\$(?P<reg>\w+)\s*=\s*(?P<expr>.+)', line)
if m is None:
raise SyntaxError('illegal command syntax')
if not m['reg']:
raise KeyError('error parsing input: invalid lhand expression')
if not m['expr']:
raise SyntaxError('error parsing input: invalid rhand expression')
reg = self.unalias(m['reg'])
expr = self.resolve_expr(m['expr'])
self.write_reg(reg, expr)
return (reg, expr)
def handle_i(self, addr: int, count: int) -> List[InsnLike]:
result = []
for _ in range(count):
insn = self.disasm(addr)
addr += insn.size
result.append(insn)
return result
def handle_examine(self, line: str) -> None:
# examples:
# x/xw address
# x/4xw $esp
# x/4xg $rsp
# x/i $eip - 0x10
# x $sp
# x $sp + 0xc
m = re.match(r'(?:/(?P<n>\d+)?(?P<f>[oxdutfacis])?(?P<u>[bhwg])?)?\s*(?P<target>.+)?', line)
# there should be always a match, at least for target, but let's be on the safe side
if m is None:
raise ValueError('unexpected examine command syntax')
n = m['n'] or self.x_defaults['n']
f = m['f'] or self.x_defaults['f']
u = m['u'] or self.x_defaults['u']
target = m['target']
# if target was specified, determine its value. otherwise use the current address
target = self.resolve_expr(target) if target else self.cur_addr
n = int(n)
if f == r'i':
for insn in self.handle_i(target, n):
print(f"{insn.address:#010x}: {insn.mnemonic:10s} {insn.op_str}")
# handle read c-style string
elif f == r's':
s = self.try_read_string(target)
if s is None:
raise ValueError(f'error reading c-style string at {target:#010x}')
print(f"{target:#010x}: {s}")
else:
def __to_size(u: str) -> int:
"""Convert a gdb unit name to its corresponding size in bytes.
"""
sizes = {
'b': 1, # byte
'h': 2, # halfword
'w': 4, # word
'g': 8 # giant
}
# assume u is in sizes
return sizes[u]
def __to_py_spec(f: str, size: int) -> Tuple[str, str, str]:
"""Convert a gdb format specifier to its corresponding python format,
prefix and padding specifiers.
"""
specs = {
'o': ('o', '0', ''), # octal
'x': ('x', '0x', f'0{size * 2}'), # hex
'd': ('d', '', ''), # decimal
'u': ('u', '', ''), # unsigned decimal
't': ('b', '', f'0{size * 8}'), # binary
'f': ('f', '', ''), # float
'a': ('x', '0x', f'0{size * 2}'), # address
'c': ('c', '', ''), # char
}
# assume f is in specs
return specs[f]
size = __to_size(u)
pyfmt, prefix, pad = __to_py_spec(f, size)
values = [self.try_read_pointer(target + (i * size), size) for i in range(n)]
ipr = 4 # number of items to display per row
for i in range(0, len(values), ipr):
vset = values[i:i + ipr]
print(f'{target + i * size:#10x}:', end='\t')
for v in vset:
print('?' if v is None else f'{prefix}{v:{pad}{pyfmt}}', end='\t')
print()
qiling/debugger/qdb/memory.py
-204
View File
@@ -1,204 +0,0 @@
#!/usr/bin/env python3
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from qiling.const import QL_ARCH
from .context import Context
from .arch import ArchCORTEX_M, ArchARM, ArchMIPS, ArchX86, ArchX8664
from .misc import check_and_eval
import re, math
def setup_memory_Manager(ql):
arch_type = {
QL_ARCH.X86: ArchX86,
QL_ARCH.X8664: ArchX8664,
QL_ARCH.MIPS: ArchMIPS,
QL_ARCH.ARM: ArchARM,
QL_ARCH.CORTEX_M: ArchCORTEX_M,
}.get(ql.arch.type)
ret = type(
"MemoryManager",
(MemoryManager, arch_type),
{}
)
return ret(ql)
class MemoryManager(Context):
"""
memory manager for handing memory access
"""
def __init__(self, ql):
super().__init__(ql)
@property
def get_default_fmt(self):
return ('x', 4, 1)
@property
def get_format_letter(self):
return {
"o", # octal
"x", # hex
"d", # decimal
"u", # unsigned decimal
"t", # binary
"f", # float
"a", # address
"i", # instruction
"c", # char
"s", # string
"z", # hex, zero padded on the left
}
@property
def get_size_letter(self):
return {
"b": 1, # 1-byte, byte
"h": 2, # 2-byte, halfword
"w": 4, # 4-byte, word
"g": 8, # 8-byte, giant
}
def extract_count(self, t):
return "".join([s for s in t if s.isdigit()])
def get_fmt(self, text):
f, s, c = self.get_default_fmt
if self.extract_count(text):
c = int(self.extract_count(text))
for char in text.strip(str(c)):
if char in self.get_size_letter.keys():
s = self.get_size_letter.get(char)
elif char in self.get_format_letter:
f = char
return (f, s, c)
def fmt_unpack(self, bs: bytes, sz: int) -> int:
return {
1: lambda x: x[0],
2: self.unpack16,
4: self.unpack32,
8: self.unpack64,
}.get(sz)(bs)
def handle_i(self, addr, ct=1):
result = []
for offset in range(addr, addr+ct*4, 4):
if (line := self.disasm(offset)):
result.append(line)
return result
def parse(self, line: str):
# test case
# x/wx address
# x/i address
# x $sp
# x $sp +0xc
# x $sp+0xc
# x $sp + 0xc
if line.startswith("/"): # followed by format letter and size letter
fmt, *rest = line.strip("/").split()
fmt = self.get_fmt(fmt)
else:
args = line.split()
rest = [args[0]] if len(args) == 1 else args
fmt = self.get_default_fmt
if len(rest) == 0:
return
line = []
if (regs_dict := getattr(self, "regs_need_swapped", None)):
for each in rest:
for reg in regs_dict:
if each in regs_dict:
line.append(regs_dict[each])
else:
line.append(each)
else:
line = rest
# for simple calculation with register and address
line = " ".join(line)
# substitue register name with real value
for each_reg in filter(lambda r: len(r) == 3, self.ql.arch.regs.register_mapping):
reg = f"${each_reg}"
if reg in line:
line = re.sub(f"\\{reg}", hex(self.ql.arch.regs.read(each_reg)), line)
for each_reg in filter(lambda r: len(r) == 2, self.ql.arch.regs.register_mapping):
reg = f"${each_reg}"
if reg in line:
line = re.sub(f"\\{reg}", hex(self.ql.arch.regs.read(each_reg)), line)
ft, sz, ct = fmt
try:
addr = check_and_eval(line)
except:
return "something went wrong ..."
if ft == "i":
output = self.handle_i(addr, ct)
for each in output:
print(f"0x{each.address:x}: {each.mnemonic}\t{each.op_str}")
elif ft == "s":
# handle read c-style string
try:
print(f"0x{addr:08x}: {self.ql.os.utils.read_cstring(addr)}")
except:
return f"error reading c-style string at 0x{addr:08x}"
else:
lines = 1 if ct <= 4 else math.ceil(ct / 4)
# parse command
prefix = "0x" if ft in ("x", "a") else ""
pad = '0' + str(sz*2) if ft in ('x', 'a', 't') else ''
ft = ft.lower() if ft in ("x", "o", "b", "d") else ft.lower().replace("t", "b").replace("a", "x")
mem_read = []
for offset in range(ct):
# append data if read successfully, otherwise return error message
if (data := self.try_read(addr+(offset*sz), sz))[0] is not None:
mem_read.append(data[0])
else:
return data[1]
for line in range(lines):
offset = line * sz * 4
print(f"0x{addr+offset:x}:\t", end="")
idx = line * self.ql.arch.pointersize
for each in mem_read[idx:idx+self.ql.arch.pointersize]:
data = self.fmt_unpack(each, sz)
print(f"{prefix}{data:{pad}{ft}}\t", end="")
print()
return True
qiling/debugger/qdb/misc.py
+35 -59
View File
@@ -3,92 +3,68 @@
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from typing import AnyStr, Callable, Optional
from typing import Optional, Union
from dataclasses import dataclass
from capstone import CsInsn
import ast
def check_and_eval(line: str):
"""
This function will valid all type of nodes and evaluate it if nothing went wrong
"""
class AST_checker(ast.NodeVisitor):
def generic_visit(self, node):
if type(node) in (ast.Module, ast.Expr, ast.BinOp, ast.Constant, ast.Add, ast.Mult, ast.Sub):
ast.NodeVisitor.generic_visit(self, node)
else:
raise ParseError("malform or invalid ast node")
checker = AST_checker()
ast_tree = ast.parse(line)
checker.visit(ast_tree)
return eval(line)
@dataclass
class InvalidInsn:
"""
class for displaying invalid instruction
"""
bytes: bytes
address: bytes
mnemonic: str = 'invalid'
address: int
mnemonic: str = '(invalid)'
op_str: str = ''
def __post_init__(self):
self.size = len(self.bytes)
self.size = len(self.bytes) if self.bytes else 1
class Breakpoint:
"""Dummy class for breakpoints.
"""
dummy class for breakpoint
"""
def __init__(self, addr: int):
# monotonically increasing index counter
_counter = 0
def __init__(self, addr: int, temp: bool = False):
"""Initialize a breakpoint object.
Args:
addr: address to break upon arrival
temp: whether this is a temporary breakpoint. temporary breakpoints
get removed after they get hit for the first time
"""
self.index = Breakpoint._counter
Breakpoint._counter += 1
self.addr = addr
self.hitted = False
self.temp = temp
self.enabled = True
class TempBreakpoint(Breakpoint):
def read_int(s: str, /) -> int:
"""Turn a numerical string into its integer value.
"""
dummy class for temporay breakpoint
"""
def __init__(self, addr: int):
super().__init__(addr)
def read_int(s: str) -> int:
"""
parse unsigned integer from string
"""
return int(s, 0)
def try_read_int(s: AnyStr) -> Optional[int]:
"""
try to read string as integer is possible
def try_read_int(s: str, /) -> Optional[int]:
"""Attempt to convert string to an integer value.
"""
try:
ret = read_int(s)
except:
ret = None
val = read_int(s)
except (ValueError, TypeError):
val = None
return ret
return val
def parse_int(func: Callable) -> Callable:
"""
function dectorator for parsing argument as integer
"""
def wrap(qdb, s: str = "") -> int:
assert type(s) is str
ret = try_read_int(s)
return func(qdb, ret)
return wrap
if __name__ == "__main__":
pass
InsnLike = Union[CsInsn, InvalidInsn]
qiling/debugger/qdb/qdb.py
+428 -350
View File
File diff suppressed because it is too large Load Diff
qiling/debugger/qdb/render/__init__.py
+1 -2
View File
@@ -4,7 +4,6 @@
#
from .render import ContextRender
from .render_x86 import ContextRenderX86
from .render_intel import ContextRenderX86, ContextRenderX64
from .render_mips import ContextRenderMIPS
from .render_arm import ContextRenderARM, ContextRenderCORTEX_M
from .render_x8664 import ContextRenderX8664
qiling/debugger/qdb/render/render.py
+164 -143
View File
@@ -3,168 +3,184 @@
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
"""Context Render for rendering UI
"""
from capstone import CsInsn
from typing import Mapping
import os, copy
from __future__ import annotations
import os
from typing import TYPE_CHECKING, Callable, Collection, Dict, Iterator, List, Mapping, Optional, Sequence, Tuple, Union
from ..context import Context
from ..const import color
if TYPE_CHECKING:
from qiling.core import Qiling
from ..branch_predictor.branch_predictor import BranchPredictor, Prophecy
from ..misc import InsnLike
"""
Context Render for rendering UI
COLORS = (
color.DARKCYAN,
color.BLUE,
color.RED,
color.YELLOW,
color.GREEN,
color.PURPLE,
color.CYAN,
color.WHITE
)
"""
RARROW = '\u2192'
RULER = '\u2500'
CURSOR = '\u25ba' # current instruction cursor
GOING_DN = '\u2ba6' # branching downward to a higher address
GOING_UP = '\u2ba4' # branching upward to a lower address
COLORS = (color.DARKCYAN, color.BLUE, color.RED, color.YELLOW, color.GREEN, color.PURPLE, color.CYAN, color.WHITE)
class Render:
"""
base class for rendering related functions
"""Base class for graphical rendering functionality.
Render objects are agnostic to current emulation state.
"""
def divider_printer(field_name, ruler="─"):
def __init__(self):
# make sure mixin classes are properly initialized
super().__init__()
self.regs_a_row = 4 # number of regs to display per row
self.stack_num = 8 # number of stack entries to display in context
self.disasm_num = 4 # number of instructions to display in context before and after current pc
@staticmethod
def divider_printer(header: str, footer: bool = False):
"""
decorator function for printing divider and field name
"""
def decorator(context_dumper):
def decorator(wrapped: Callable):
def wrapper(*args, **kwargs):
try:
width, _ = os.get_terminal_size()
except OSError:
width = 130
bar = (width - len(field_name)) // 2 - 1
print(ruler * bar, field_name, ruler * bar)
context_dumper(*args, **kwargs)
if "DISASM" in field_name:
print(ruler * width)
print(header.center(width, RULER))
wrapped(*args, **kwargs)
if footer:
print(RULER * width)
return wrapper
return decorator
def __init__(self):
self.regs_a_row = 4
self.stack_num = 10
self.disasm_num = 0x10
self.color = color
def reg_diff(self, cur_regs, saved_reg_dump):
def reg_diff(self, curr: Mapping[str, int], prev: Mapping[str, int]) -> List[str]:
"""
helper function for highlighting register changed during execution
"""
if saved_reg_dump:
reg_dump = copy.deepcopy(saved_reg_dump)
if getattr(self, "regs_need_swapped", None):
reg_dump = self.swap_reg_name(reg_dump)
return [k for k in curr if curr[k] != prev[k]] if prev else []
return [k for k in cur_regs if cur_regs[k] != reg_dump[k]]
def render_regs_dump(self, regs, diff_reg=None):
"""
helper function for redering registers dump
def render_regs_dump(self, regs: Mapping[str, int], diff_reg: Collection[str]) -> None:
"""Helper function for rendering registers dump.
"""
lines = ""
for idx, r in enumerate(regs, 1):
line = "{}{}: 0x{{:08x}} {}\t".format(COLORS[(idx-1) // self.regs_a_row], r, color.END)
# find the length of the longest reg name to have all regs aligned in columns
longest = max(len(name) for name in regs)
if diff_reg and r in diff_reg:
line = f"{color.UNDERLINE}{color.BOLD}{line}"
def __render_regs_line() -> Iterator[str]:
elements = []
if idx % self.regs_a_row == 0 and idx != 32:
line += "\n"
for idx, (name, value) in enumerate(regs.items()):
line_color = f'{COLORS[idx // self.regs_a_row]}'
lines += line
if name in diff_reg:
line_color = f'{color.UNDERLINE}{color.BOLD}{line_color}'
print(lines.format(*regs.values()))
elements.append(f'{line_color}{name:{longest}s}: {value:#010x}{color.END}')
def render_stack_dump(self, arch_sp: int) -> None:
"""
helper function for redering stack dump
if (idx + 1) % self.regs_a_row == 0:
yield '\t'.join(elements)
elements.clear()
for line in __render_regs_line():
print(line)
def render_flags(self, flags: Mapping[str, int], before: str = ''):
def __set(f: str) -> str:
return f'{color.BLUE}{f.upper()}{color.END}'
def __cleared(f: str) -> str:
return f'{color.GREEN}{f.lower()}{color.END}'
s_before = f"[{before}] " if before else ""
s_flags = " ".join(__set(f) if val else __cleared(f) for f, val in flags.items())
print(f'{s_before}[flags: {s_flags}]')
def render_stack_dump(self, sp: int, dump: Sequence[Tuple[int, int, Union[int, str, None]]]) -> None:
"""Helper function for rendering stack dump.
"""
# Loops over stack range (last 10 addresses)
for idx in range(self.stack_num):
addr = arch_sp + idx * self.pointersize
# number of hexadecimal nibbles to display per value
nibbles = self.pointersize * 2
'''
@NOTE: Implemented new class arch_x8664 in order to bugfix issue with only dereferencing 32-bit pointers
on 64-bit emulation passes.
'''
if (val := self.try_read_pointer(addr)[0]): # defined to be try_read_pointer(addr)[0] - dereferneces pointer
for address, value, deref in dump:
offset = address - sp
# @TODO: Bug here where the values on the stack are being displayed in 32-bit format
print(f"SP + 0x{idx*self.pointersize:02x}│ [0x{addr:08x}] —▸ 0x{self.unpack(val):08x}", end="")
value_str = '(unreachable)' if value is None else f'{value:#0{nibbles + 2}x}'
# try to dereference wether it's a pointer
if (buf := self.try_read_pointer(addr))[0] is not None:
if isinstance(deref, int):
deref_str = f'{deref:#0{nibbles + 2}x}'
if (addr := self.unpack(buf[0])):
elif isinstance(deref, str):
deref_str = f'"{deref}"'
# try to dereference again
if (buf := self.try_read_pointer(addr))[0] is not None:
s = self.try_read_string(addr)
else:
deref_str = ''
if s and s.isprintable():
print(f" ◂— {self.read_string(addr)}", end="")
else:
print(f" ◂— 0x{self.unpack(buf[0]):08x}", end="")
print()
print(f'SP + {offset:#04x} │ {address:#010x} : {value_str}{f" {RARROW} {deref_str}" if deref_str else ""}')
def render_assembly(self, lines) -> None:
"""
helper function for rendering assembly
def render_assembly(self, listing: Sequence[InsnLike], pc: int, prediction: Prophecy) -> None:
"""Helper function for rendering assembly.
"""
# assembly before current location
if (backward := lines.get("backward", None)):
for line in backward:
self.print_asm(line)
def __render_asm_line(insn: InsnLike) -> str:
"""Helper function for rendering assembly instructions, indicates where we are and
the branch prediction provided by branch predictor
"""
# assembly for current location
if (cur_insn := lines.get("current", None)):
prophecy = self.predictor.predict()
self.print_asm(cur_insn, to_jump=prophecy.going)
trace_line = f"{insn.address:#010x} │ {insn.bytes.hex():18s} {insn.mnemonic:12} {insn.op_str:35s}"
# assembly after current location
if (forward := lines.get("forward", None)):
for line in forward:
self.print_asm(line)
cursor = '' # current instruction cursor
brmark = '' # branching mark
def swap_reg_name(self, cur_regs: Mapping[str, int], extra_dict=None) -> Mapping[str, int]:
"""
swap register name with more readable register name
"""
if insn.address == pc:
cursor = CURSOR
target_items = extra_dict.items() if extra_dict else self.regs_need_swapped.items()
if prediction.going:
# branch target might be None in case it should have been
# read from memory but that memory could not be reached
bmark = '?' if prediction.where is None else (GOING_DN if prediction.where > pc else GOING_UP)
for old_reg, new_reg in target_items:
cur_regs.update({old_reg: cur_regs.pop(new_reg)})
# apply some colors
brmark = f'{color.RED}{bmark}{color.RESET}'
return cur_regs
# <DEBUG>
where = '?' if prediction.where is None else f'{prediction.where:#010x}'
def print_asm(self, insn: CsInsn, to_jump: bool = False) -> None:
"""
helper function for printing assembly instructions, indicates where we are and the branch prediction
provided by BranchPredictor
"""
print(f'prediction: {f"taken, {where}" if prediction.going else "not taken"}')
# </DEBUG>
opcode = "".join(f"{b:02x}" for b in insn.bytes)
return f"{brmark:1s} {cursor:1s} {color.DARKGRAY}{trace_line}{color.RESET}"
trace_line = f"0x{insn.address:08x} │ {opcode:15s} {insn.mnemonic:10} {insn.op_str:35s}"
cursor = "►" if self.cur_addr == insn.address else " "
jump_sign = f"{color.RED}✓{color.END}" if to_jump else " "
print(f"{jump_sign} {cursor} {color.DARKGRAY}{trace_line}{color.END}")
for insn in listing:
print(__render_asm_line(insn))
class ContextRender(Context, Render):
@@ -172,17 +188,17 @@ class ContextRender(Context, Render):
base class for context render
"""
def __init__(self, ql, predictor):
def __init__(self, ql: Qiling, predictor: BranchPredictor):
super().__init__(ql)
Render.__init__(self)
self.predictor = predictor
self.prev_regs: Dict[str, int] = {}
def dump_regs(self) -> Mapping[str, int]:
"""
dump all registers
def get_regs(self) -> Dict[str, int]:
"""Save current registers state.
"""
return {reg_name: self.ql.arch.regs.read(reg_name) for reg_name in self.regs}
return {reg_name: self.read_reg(reg_name) for reg_name in self.regs}
@Render.divider_printer("[ STACK ]")
def context_stack(self) -> None:
@@ -190,50 +206,55 @@ class ContextRender(Context, Render):
display context stack dump
"""
self.render_stack_dump(self.ql.arch.regs.arch_sp)
sp = self.cur_sp
stack_dump = []
for i in range(self.stack_num):
address = sp + i * self.asize
# attempt to read current stack entry
value = self.try_read_pointer(address)
# treat stack entry as a pointer and attempt to dereference it
deref = None if value is None else self.get_deref(value)
stack_dump.append((address, value, deref))
self.render_stack_dump(sp, stack_dump)
@Render.divider_printer("[ REGISTERS ]")
def context_reg(self, saved_states: Mapping["str", int]) -> None:
"""
display context registers dump
def context_reg(self) -> None:
"""Rendering registers context.
"""
return NotImplementedError
curr = self.get_regs()
prev = self.prev_regs
@Render.divider_printer("[ DISASM ]")
curr = self.swap_regs(curr)
prev = self.swap_regs(prev)
diff_reg = self.reg_diff(curr, prev)
self.render_regs_dump(curr, diff_reg)
self.print_mode_info()
@Render.divider_printer("[ DISASM ]", footer=True)
def context_asm(self) -> None:
"""
read context assembly and render with render_assembly
"""Disassemble srrounding instructions.
"""
lines = {}
past_list = []
from_addr = self.cur_addr - self.disasm_num
to_addr = self.cur_addr + self.disasm_num
address = self.cur_addr
prediction = self.predictor.predict()
cur_addr = from_addr
while cur_addr <= to_addr:
insn = self.disasm(cur_addr)
cur_addr += insn.size
past_list.append(insn)
# assuming a single instruction is in the same size of a native pointer.
# this is not true for all architectures.
ptr = address - self.pointersize * self.disasm_num
listing = []
bk_list = []
fd_list = []
cur_insn = None
for each in past_list:
if each.address < self.cur_addr:
bk_list.append(each)
# taking disasm_num instructions before, current, and disasm_num instructions after
for _ in range(self.disasm_num * 2 + 1):
insn = self.disasm(ptr)
listing.append(insn)
elif each.address > self.cur_addr:
fd_list.append(each)
ptr += insn.size
elif each.address == self.cur_addr:
cur_insn = each
lines.update({
"backward": bk_list,
"forward": fd_list,
"current": cur_insn,
})
self.render_assembly(lines)
self.render_assembly(listing, address, prediction)
qiling/debugger/qdb/render/render_arm.py
+46 -53
View File
@@ -3,73 +3,66 @@
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from typing import Iterator
from .render import *
from .render import Render, ContextRender
from ..arch import ArchARM, ArchCORTEX_M
from ..misc import InsnLike
class ContextRenderARM(ContextRender, ArchARM):
"""
context render for ARM
"""Context renderer for ARM architecture.
"""
def __init__(self, ql, predictor):
super().__init__(ql, predictor)
ArchARM.__init__(self)
self.disasm_num = 8
def print_mode_info(self) -> None:
cpsr = self.read_reg(self._flags_reg)
@staticmethod
def print_mode_info(bits):
flags = ArchARM.get_flags(bits)
flags = ArchARM.get_flags(cpsr)
mode = ArchARM.get_mode(cpsr)
print(f"[{flags.pop('mode')} mode] ", end="")
for key, val in flags.items():
if val:
print(f"{color.BLUE}{key.upper()} ", end="")
else:
print(f"{color.GREEN}{key.lower()} ", end="")
self.render_flags(flags, f'{mode} mode')
print(color.END)
def __disasm_all(self, rng: range) -> Iterator[InsnLike]:
addr = rng.start
@Render.divider_printer("[ REGISTERS ]")
def context_reg(self, saved_reg_dump):
"""
redering context registers
while addr in rng:
insn = self.disasm(addr)
yield insn
addr += insn.size
@Render.divider_printer("[ DISASM ]", footer=True)
def context_asm(self) -> None:
"""Disassemble srrounding instructions.
"""
cur_regs = self.dump_regs()
cur_regs = self.swap_reg_name(cur_regs)
diff_reg = self.reg_diff(cur_regs, saved_reg_dump)
self.render_regs_dump(cur_regs, diff_reg=diff_reg)
self.print_mode_info(self.ql.arch.regs.cpsr)
address = self.cur_addr
prediction = self.predictor.predict()
# arm thumb may mix narrow and wide instructions so we can never know for
# sure where we need to start reading instructions from. to work around
# that we assume all instructions are wide, and then take the most recent
# ones into consideration.
listing = []
begin = address - self.asize * self.disasm_num
end = address
# disassemble all instructions in range, but keep only the last ones
listing.extend(self.__disasm_all(range(begin, end)))
listing = listing[-self.disasm_num:]
begin = address
end = address + self.asize * (self.disasm_num + 1)
# disassemble all instructions in range, but keep only the first ones
listing.extend(self.__disasm_all(range(begin, end)))
listing = listing[:self.disasm_num * 2 + 1]
self.render_assembly(listing, address, prediction)
class ContextRenderCORTEX_M(ContextRenderARM, ArchCORTEX_M):
"""Context renderer for ARM Cortex-M architecture.
"""
context render for cortex_m
"""
def __init__(self, ql, predictor):
super().__init__(ql, predictor)
ArchCORTEX_M.__init__(self)
self.regs_a_row = 3
@Render.divider_printer("[ REGISTERS ]")
def context_reg(self, saved_reg_dump):
cur_regs = self.dump_regs()
cur_regs = self.swap_reg_name(cur_regs)
# for re-order
extra_dict = {
"xpsr": "xpsr",
"control": "control",
"primask": "primask",
"faultmask": "faultmask",
"basepri": "basepri",
}
cur_regs = self.swap_reg_name(cur_regs, extra_dict=extra_dict)
diff_reg = self.reg_diff(cur_regs, saved_reg_dump)
self.render_regs_dump(cur_regs, diff_reg=diff_reg)
self.print_mode_info(self.ql.arch.regs.cpsr)
qiling/debugger/qdb/render/render_intel.py
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env python3
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from typing import Optional
from .render import Render, ContextRender
from ..arch import ArchIntel, ArchX86, ArchX64
class ContextRenderIntel(ContextRender):
"""Context renderer base class for Intel architecture.
"""
def print_mode_info(self) -> None:
eflags = self.read_reg('eflags')
flags = ArchIntel.get_flags(eflags)
iopl = ArchIntel.get_iopl(eflags)
self.render_flags(flags, f'iopl: {iopl}')
@Render.divider_printer("[ DISASM ]", footer=True)
def context_asm(self) -> None:
"""Disassemble srrounding instructions.
"""
address = self.cur_addr
prediction = self.predictor.predict()
ptr = address
listing = []
# since intel architecture has instructions with varying sizes, it is
# difficult to tell what were the preceding instructions. for that reason
# we display instructions only from current address and on.
for _ in range(9):
insn = self.disasm(ptr)
listing.append(insn)
ptr += insn.size
self.render_assembly(listing, address, prediction)
class ContextRenderX86(ContextRenderIntel, ArchX86):
"""Context renderer for x86 architecture.
"""
class ContextRenderX64(ContextRenderIntel, ArchX64):
"""Context renderer for x86-64 architecture.
"""
qiling/debugger/qdb/render/render_mips.py
+5 -19
View File
@@ -3,27 +3,13 @@
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from .render import *
from .render import ContextRender
from ..arch import ArchMIPS
class ContextRenderMIPS(ContextRender, ArchMIPS):
"""
context render for MIPS
"""Context renderer for MIPS architecture.
"""
def __init__(self, ql, predictor):
super().__init__(ql, predictor)
ArchMIPS.__init__(self)
@Render.divider_printer("[ REGISTERS ]")
def context_reg(self, saved_reg_dump):
"""
redering context registers
"""
cur_regs = self.dump_regs()
cur_regs = self.swap_reg_name(cur_regs)
diff_reg = self.reg_diff(cur_regs, saved_reg_dump)
self.render_regs_dump(cur_regs, diff_reg=diff_reg)
def print_mode_info(self) -> None:
pass
qiling/debugger/qdb/render/render_x86.py
-68
View File
@@ -1,68 +0,0 @@
#!/usr/bin/env python3
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from .render import *
from ..arch import ArchX86
class ContextRenderX86(ContextRender, ArchX86):
"""
context render for X86
"""
def __init__(self, ql, predictor):
super().__init__(ql, predictor)
ArchX86.__init__(self)
@Render.divider_printer("[ REGISTERS ]")
def context_reg(self, saved_reg_dump):
cur_regs = self.dump_regs()
diff_reg = self.reg_diff(cur_regs, saved_reg_dump)
self.render_regs_dump(cur_regs, diff_reg=diff_reg)
flags = self.get_flags(self.ql.arch.regs.eflags)
print("EFLAGS: ", end="")
print(color.GREEN, end="")
for key, val in flags.items():
if val:
print(f"{color.BLUE}{key.upper()} ", end="")
else:
print(f"{color.GREEN}{key.lower()} ", end="")
print(color.END)
@Render.divider_printer("[ DISASM ]")
def context_asm(self):
lines = {}
past_list = []
cur_addr = self.cur_addr
while len(past_list) < 10:
line = self.disasm(cur_addr)
past_list.append(line)
cur_addr += line.size
fd_list = []
cur_insn = None
for each in past_list:
if each.address > self.cur_addr:
fd_list.append(each)
elif each.address == self.cur_addr:
cur_insn = each
"""
only forward and current instruction will be printed,
because we don't have a solid method to disasm backward instructions,
since it's x86 instruction length is variadic
"""
lines.update({
"current": cur_insn,
"forward": fd_list,
})
self.render_assembly(lines)
qiling/debugger/qdb/render/render_x8664.py
-58
View File
@@ -1,58 +0,0 @@
#!/usr/bin/env python3
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from .render import *
from ..arch import ArchX8664
class ContextRenderX8664(ContextRender, ArchX8664):
"""
Context render for X86_64
"""
def __init__(self, ql, predictor):
super().__init__(ql, predictor)
ArchX8664.__init__(self)
@Render.divider_printer("[ REGISTERS ]")
def context_reg(self, saved_reg_dump):
cur_regs = self.dump_regs()
diff_reg = self.reg_diff(cur_regs, saved_reg_dump)
self.render_regs_dump(cur_regs, diff_reg=diff_reg)
print(color.GREEN, "EFLAGS: [CF: {flags[CF]}, PF: {flags[PF]}, AF: {flags[AF]}, ZF: {flags[ZF]}, SF: {flags[SF]}, OF: {flags[OF]}]".format(flags=self.get_flags(self.ql.arch.regs.eflags)), color.END, sep="")
@Render.divider_printer("[ DISASM ]")
def context_asm(self):
lines = {}
past_list = []
cur_addr = self.cur_addr
while len(past_list) < 10:
line = self.disasm(cur_addr)
past_list.append(line)
cur_addr += line.size
fd_list = []
cur_insn = None
for each in past_list:
if each.address > self.cur_addr:
fd_list.append(each)
elif each.address == self.cur_addr:
cur_insn = each
"""
only forward and current instruction will be printed,
because we don't have a solid method to disasm backward instructions,
since it's x86 instruction length is variadic
"""
lines.update({
"current": cur_insn,
"forward": fd_list,
})
self.render_assembly(lines)
qiling/debugger/qdb/utils.py
+224 -205
View File
@@ -4,16 +4,16 @@
#
from __future__ import annotations
from typing import TYPE_CHECKING, Callable, Dict, Mapping, Tuple, Type
from capstone import CsInsn
from enum import Enum
from typing import TYPE_CHECKING, Any, Callable, Dict, List, Mapping, Optional, Tuple, Type, TypeVar, Union
from qiling.const import QL_ARCH
from .render import (
ContextRender,
ContextRenderX86,
ContextRenderX8664,
ContextRenderX64,
ContextRenderARM,
ContextRenderCORTEX_M,
ContextRenderMIPS
@@ -22,7 +22,7 @@ from .render import (
from .branch_predictor import (
BranchPredictor,
BranchPredictorX86,
BranchPredictorX8664,
BranchPredictorX64,
BranchPredictorARM,
BranchPredictorCORTEX_M,
BranchPredictorMIPS,
@@ -36,81 +36,69 @@ if TYPE_CHECKING:
from .qdb import QlQdb
def qdb_print(msgtype: QDB_MSG, msg: str) -> None:
"""
color printing
_K = TypeVar('_K')
_V = TypeVar('_V')
def qdb_print(level: QDB_MSG, msg: str) -> None:
"""Log printing.
"""
def print_error(msg):
return f"{color.RED}[!] {msg}{color.END}"
decorations = {
QDB_MSG.ERROR: ('!', color.RED),
QDB_MSG.INFO : ('+', color.CYAN),
}
def print_info(msg):
return f"{color.CYAN}[+] {msg}{color.END}"
tag, col = decorations[level]
color_coated = {
QDB_MSG.ERROR: print_error,
QDB_MSG.INFO : print_info,
}.get(msgtype)(msg)
print(color_coated)
print(f'{col}[{tag}] {msg}{color.END}')
def setup_address_marker():
class Marker:
"""provide the ability to mark an address as a more easier rememberable alias
"""
class Marker:
"""provide the ability to mark an address as a more easier rememberable alias
def __init__(self):
self._mark_list: Dict[str, int] = {}
def get_address(self, sym: str) -> Optional[int]:
"""
get the mapped address to a symbol if it's in the mark_list
"""
def __init__(self):
self._mark_list = {}
return self._mark_list.get(sym)
def get_symbol(self, sym):
"""
get the mapped address to a symbol if it's in the mark_list
"""
@property
def mark_list(self):
"""
get a list about what we marked
"""
return self._mark_list.get(sym, None)
return self._mark_list.items()
@property
def mark_list(self):
"""
get a list about what we marked
"""
def gen_sym_name(self) -> str:
"""
generating symbol name automatically
"""
return self._mark_list.items()
syms = len(self._mark_list)
def gen_sym_name(self):
"""
generating symbol name automatically
"""
# find the next available 'sym#'
return next((f'sym{i}' for i in range(syms) if f'sym{i}' not in self._mark_list), f'sym{syms}')
sym_name, idx = "sym0", 0
while sym_name in self._mark_list:
idx += 1
sym_name = f"sym{idx}"
def mark(self, loc: int, sym: Optional[str] = None) -> str:
"""
mark loc as sym
"""
return sym_name
sym = sym or self.gen_sym_name()
def mark_only_loc(self, loc):
"""
mark when location provided only
"""
if sym in self._mark_list:
return ''
sym_name = self.gen_sym_name()
self.mark(sym_name, loc)
return sym_name
self._mark_list[sym] = loc
def mark(self, sym: str, loc: int):
"""
mark loc as sym
"""
if sym not in self.mark_list:
self._mark_list.update({sym: loc})
else:
return f"dumplicated symbol name: {sym} at address: 0x{loc:08x}"
return Marker()
return sym
# helper functions for setting proper branch predictor and context render depending on different arch
@@ -120,7 +108,7 @@ def setup_branch_predictor(ql: Qiling) -> BranchPredictor:
preds: Dict[QL_ARCH, Type[BranchPredictor]] = {
QL_ARCH.X86: BranchPredictorX86,
QL_ARCH.X8664: BranchPredictorX8664,
QL_ARCH.X8664: BranchPredictorX64,
QL_ARCH.ARM: BranchPredictorARM,
QL_ARCH.CORTEX_M: BranchPredictorCORTEX_M,
QL_ARCH.MIPS: BranchPredictorMIPS
@@ -136,7 +124,7 @@ def setup_context_render(ql: Qiling, predictor: BranchPredictor) -> ContextRende
rends: Dict[QL_ARCH, Type[ContextRender]] = {
QL_ARCH.X86: ContextRenderX86,
QL_ARCH.X8664: ContextRenderX8664,
QL_ARCH.X8664: ContextRenderX64,
QL_ARCH.ARM: ContextRenderARM,
QL_ARCH.CORTEX_M: ContextRenderCORTEX_M,
QL_ARCH.MIPS: ContextRenderMIPS
@@ -146,121 +134,137 @@ def setup_context_render(ql: Qiling, predictor: BranchPredictor) -> ContextRende
return r(ql, predictor)
def run_qdb_script(qdb: QlQdb, filename: str) -> None:
with open(filename) as fd:
for line in iter(fd.readline, ""):
# skip commented and empty line
if line.startswith("#") or line == "\n":
continue
class MemDiff(Enum):
ADD = '+'
REM = '-'
MOD = '*'
cmd, arg, _ = qdb.parseline(line)
func = getattr(qdb, f"do_{cmd}")
if arg:
func(arg)
else:
func()
RamKey = Tuple[int, int]
RamVal = Tuple[int, str, bytes]
RamDiffKey = Tuple[int, int]
RamDiffVal = Tuple[MemDiff, Tuple[int, str, Union[bytes, Tuple]]]
class DiffedState:
"""
internal container for storing diffed state
"""
def __init__(self, reg, xreg, ram, loader):
self.reg: Dict[str, int] = reg
self.xreg: Dict[str, int] = xreg
self.ram: Dict[RamDiffKey, RamDiffVal] = ram
self.loader: Dict[str, Any] = loader
class State:
"""
internal container for storing raw state from qiling
"""
def __init__(self, saved: Mapping[str, Mapping]):
self.reg: Dict[str, int] = saved.get("reg") or {}
self.xreg: Dict[str, int] = saved.get("cpr") or saved.get("msr") or {}
mem = saved.get("mem") or {}
ram = mem.get("ram") or []
# saved ram lists might not match in order, we turn them into dicts to work around
# that. in these dicts every memory content is mapped to its memory entry's properties
self.ram: Dict[RamKey, RamVal] = {(lbound, ubound): (perms, label, data) for lbound, ubound, perms, label, data in ram}
self.loader: Dict[str, Any] = saved.get('loader') or {}
@staticmethod
def __dict_diff(d0: Mapping[_K, _V], d1: Mapping[_K, _V]) -> Dict[_K, _V]:
return {k: v for k, v in d0.items() if v != d1.get(k)}
def _diff_reg(self, other: State) -> Dict[str, int]:
return State.__dict_diff(self.reg, other.reg)
def _diff_xreg(self, other: State) -> Dict[str, int]:
return State.__dict_diff(self.xreg, other.xreg)
def _diff_ram(self, other: State) -> Dict[RamDiffKey, RamDiffVal]:
ram0 = self.ram
ram1 = other.ram
ram_diff: Dict[RamDiffKey, RamDiffVal] = {}
removed = [rng for rng in ram0 if rng not in ram1]
added = [rng for rng in ram1 if rng not in ram0]
modified = [rng for rng in ram0 if rng in ram1 and ram0[rng] != ram1[rng]]
# memory regions that got removed should be re-added
for rng in removed:
ram_diff[rng] = (MemDiff.ADD, ram0[rng])
# memory regions that got added should be removed
for rng in added:
_, label, _ = ram1[rng]
# though we discard data as it is not required anymore, label is still required
# to determine the method of removing the region: brk, mmap, or ordinary map
ram_diff[rng] = (MemDiff.REM, (-1, label, b''))
# memory regions that fot modified should be reverted back
for rng in modified:
perms0, label0, data0 = ram0[rng]
perms1, label1, data1 = ram1[rng]
perms = -1 if perms0 == perms1 else perms0
assert label0 == label1, 'memory region label changed unexpectedly'
assert len(data0) == len(data1), 'memory contents differ in size'
# scan both data chunks and keep the index and byte value of the unmatched ones.
# if memory contents are identical, this will result in an empty tuple
data_diff = tuple((i, b0) for i, (b0, b1) in enumerate(zip(data0, data1)) if b0 != b1)
ram_diff[rng] = (MemDiff.MOD, (perms, label0, data_diff))
# <DEBUG>
# for rng, (opcode, diff) in sorted(ram_diff.items()):
# lbound, ubound = rng
# perms, label, data = diff
#
# print(f'{opcode.name} {lbound:010x} - {ubound:010x} {perms:03b} {label:24s} ~{len(data)}')
# </DEBUG>
return ram_diff
def diff(self, other: State) -> DiffedState:
"""Diff between previous and current state.
"""
return DiffedState(
self._diff_reg(other),
self._diff_xreg(other),
self._diff_ram(other),
self.loader
)
class SnapshotManager:
"""for functioning differential snapshot
Supports Qdb features like:
1. record/replay debugging
2. memory access in gdb-style
"""Differential snapshot object.
"""
class State:
"""
internal container for storing raw state from qiling
"""
def __init__(self, saved_state):
self.reg, self.ram, self.xreg = SnapshotManager.transform(saved_state)
class DiffedState:
"""
internal container for storing diffed state
"""
def __init__(self, diffed_st):
self.reg, self.ram, self.xreg = diffed_st
@staticmethod
def transform(st):
"""
transform saved context into binary set
"""
reg = st.get("reg", {})
mem = st.get("mem", [])
xreg = st.get("cpr") or st.get("msr") or {}
ram = []
for mem_seg in mem["ram"]:
lbound, ubound, perms, label, raw_bytes = mem_seg
rb_set = {(idx, val) for idx, val in enumerate(raw_bytes)}
ram.append((lbound, ubound, perms, label, rb_set))
return (reg, ram, xreg)
def __init__(self, ql):
def __init__(self, ql: Qiling):
self.ql = ql
self.layers = []
self.layers: List[DiffedState] = []
def _save(self) -> State:
def save(self) -> State:
"""
acquire current State by wrapping saved context from ql.save()
"""
return self.State(self.ql.save())
return State(self.ql.save(reg=True, mem=True, loader=True))
def diff_reg(self, prev_reg, cur_reg):
"""
diff two register values
"""
diffed = filter(lambda t: t[0] != t[1], zip(prev_reg.items(), cur_reg.items()))
return {prev[0]: prev[1] for prev, _ in diffed}
def diff_ram(self, prev_ram, cur_ram):
"""
diff two ram data if needed
"""
if any((cur_ram is None, prev_ram is None, prev_ram == cur_ram)):
return
ram = []
paired = zip(prev_ram, cur_ram)
for each in paired:
# lbound, ubound, perm, label, data
*prev_others, prev_rb_set = each[0]
*cur_others, cur_rb_set = each[1]
if prev_others == cur_others and cur_rb_set != prev_rb_set:
diff_set = prev_rb_set - cur_rb_set
else:
continue
ram.append((*cur_others, diff_set))
return ram
def diff(self, before_st, after_st):
"""
diff between previous and current state
"""
# prev_st = self.layers.pop()
diffed_reg = self.diff_reg(before_st.reg, after_st.reg)
diffed_ram = self.diff_ram(before_st.ram, after_st.ram)
diffed_xreg = self.diff_reg(before_st.xreg, after_st.xreg)
# diffed_reg = self.diff_reg(prev_st.reg, cur_st.reg)
# diffed_ram = self.diff_ram(prev_st.ram, cur_st.ram)
return self.DiffedState((diffed_reg, diffed_ram, diffed_xreg))
def snapshot(func):
@staticmethod
def snapshot(func: Callable) -> Callable:
"""
decorator function for saving differential context on certian qdb command
"""
@@ -268,17 +272,16 @@ class SnapshotManager:
def magic(self: QlQdb, *args, **kwargs):
if self.rr:
# save State before execution
p_st = self.rr._save()
before = self.rr.save()
# certian execution to be snapshot
func(self, *args, **kwargs)
# save State after execution
q_st = self.rr._save()
after = self.rr.save()
# merge two saved States into a DiffedState
st = self.rr.diff(p_st, q_st)
self.rr.layers.append(st)
self.rr.layers.append(before.diff(after))
else:
func(self, *args, **kwargs)
@@ -289,49 +292,65 @@ class SnapshotManager:
helper function for restoring running state from an existing incremental snapshot
"""
prev_st = self.layers.pop()
cur_st = self._save()
prev_st = self.layers.pop() # DiffedState
curr_st = self.save() # State, expected to be identical to 'after' State in snapshot method
for reg_name, reg_value in prev_st.reg.items():
cur_st.reg[reg_name] = reg_value
curr_st.reg.update(prev_st.reg)
curr_st.xreg.update(prev_st.xreg)
for reg_name, reg_value in prev_st.xreg.items():
cur_st.xreg[reg_name] = reg_value
if prev_st.ram:
diff_ram = prev_st.ram
curr_ram = curr_st.ram
to_be_restored = {
"reg": cur_st.reg,
# we must begin by removing unwanted memory regions, otherwise we would not be able to
# add new ones in case they overlap. here we iterate over the diff dictionary but handle
# only remove opcodes
for rng, (opcode, props) in diff_ram.items():
lbound, ubound = rng
size = ubound - lbound
if opcode is MemDiff.REM:
# NOTE: it doesn't seem like distinguishing between brk, mmap, mmap annonymous
# and regular maps is actually required
self.ql.mem.unmap(lbound, size)
# doind a second pass, but this time handling add and modify opcodes
for rng, (opcode, props) in diff_ram.items():
lbound, ubound = rng
perms, label, data = props
size = ubound - lbound
if opcode is MemDiff.ADD:
# TODO: distinguish between brk, mmap, mmap annonymous and regular maps
self.ql.mem.map(lbound, size, perms, label)
self.ql.mem.write(lbound, data)
elif opcode is MemDiff.MOD:
if perms != -1:
self.ql.mem.protect(lbound, size, perms)
# is there a diff for this memory range?
if data:
# get current memory content
_, _, curr_data = curr_ram[rng]
curr_data = bytearray(curr_data)
# patch with existing diff
for i, b in data:
curr_data[i] = b
# write patched data
self.ql.mem.write(lbound, bytes(curr_data))
self.ql.restore({
'reg': curr_st.reg,
# though we have arch-specific context to restore, we want to keep this arch-agnostic.
# one way to work around that is to include 'xreg' both as msr (intel) and cpr (arm).
# only the relevant one will be picked up while the other one will be discarded
"msr": cur_st.xreg,
"cpr": cur_st.xreg
}
'msr': curr_st.xreg,
'cpr': curr_st.xreg,
# FIXME: not sure how this one even works. while curr_st is a fresh qiling snapshot,
# prev_st is a DiffedState which does not hold a complete state but only a diff between
# two points which seem to be unrelated here.
#
# this code only patches current memory content with the diff between points a and b while
# we may be already be at point c.
if getattr(prev_st, "ram", None) and prev_st.ram != cur_st.ram:
ram = []
# lbound, ubound, perm, label, data
for each in prev_st.ram:
*prev_others, prev_rb_set = each
for *cur_others, cur_rb_set in cur_st.ram:
if prev_others == cur_others:
cur_rb_dict = dict(cur_rb_set)
for idx, val in prev_rb_set:
cur_rb_dict[idx] = val
bs = bytes(dict(sorted(cur_rb_dict.items())).values())
ram.append((*cur_others, bs))
to_be_restored["mem"] = {
"ram": ram,
"mmio": {}
}
self.ql.restore(to_be_restored)
'loader': prev_st.loader
})
qiling/debugger/utils.py
-344
View File
@@ -1,344 +0,0 @@
#!/usr/bin/env python3
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from elftools.common.exceptions import ELFError
from elftools.common.py3compat import (
ifilter, byte2int, bytes2str, itervalues, str2bytes, iterbytes)
from elftools.elf.elffile import ELFFile
from elftools.elf.dynamic import DynamicSection, DynamicSegment
from elftools.elf.enums import ENUM_D_TAG
from elftools.elf.segments import InterpSegment
from elftools.elf.sections import NoteSection, SymbolTableSection
from elftools.elf.gnuversions import (
GNUVerSymSection, GNUVerDefSection,
GNUVerNeedSection,
)
from elftools.elf.relocation import RelocationSection
from elftools.elf.descriptions import (
describe_ei_class, describe_ei_data, describe_ei_version,
describe_ei_osabi, describe_e_type, describe_e_machine,
describe_e_version_numeric, describe_p_type, describe_p_flags,
describe_sh_type, describe_sh_flags,
describe_symbol_type, describe_symbol_bind, describe_symbol_visibility,
describe_symbol_shndx, describe_reloc_type, describe_dyn_tag,
describe_dt_flags, describe_dt_flags_1, describe_ver_flags, describe_note,
describe_attr_tag_arm
)
from elftools.elf.constants import E_FLAGS
from elftools.elf.constants import E_FLAGS_MASKS
from qiling import Qiling
class QlReadELF(object):
def __init__(self, ql:Qiling, elf_stream):
self.ql = ql
self.elffile = ELFFile(elf_stream)
self._versioninfo = None
def elf_file_header(self):
elf_header = {}
def add_info(key, value):
elf_header[key] = value
header = self.elffile.header
e_ident = header['e_ident']
add_info('Magic', ' '.join('%2.2x' % byte2int(b)
for b in self.elffile.e_ident_raw))
add_info('Class',describe_ei_class(e_ident['EI_CLASS']))
add_info('Data', describe_ei_data(e_ident['EI_DATA']))
add_info('Version', e_ident['EI_VERSION'])
add_info('OS/ABI', describe_ei_osabi(e_ident['EI_OSABI']))
add_info('ABI Version', e_ident['EI_ABIVERSION'])
add_info('Type', describe_e_type(header['e_type']))
add_info('Machine', describe_e_machine(header['e_machine']))
add_info('Version_e', describe_e_version_numeric(header['e_version']))
add_info('Entry point address', self._format_hex(header['e_entry']))
add_info('Start of program headers', header['e_phoff'])
add_info('Start of section headers', header['e_shoff'])
add_info('Flags', [self._format_hex(header['e_flags']),
self.decode_flags(header['e_flags'])])
add_info('Size of this header', header['e_ehsize'])
add_info('Size of program headers', header['e_phentsize'])
add_info('Number of program headers', header['e_phnum'])
add_info('Size of section headers', header['e_shentsize'])
add_info('Number of section headers', header['e_shnum'])
add_info('Section header string table index', header['e_shstrndx'])
return elf_header
def elf_program_headers(self):
program_headers = []
def add_info(dic):
program_headers.append(dic)
if self.elffile.num_segments() == 0:
return None
for segment in self.elffile.iter_segments():
program_hdr = {}
program_hdr['Type'] = describe_p_type(segment['p_type'])
program_hdr['Offset'] = self._format_hex(segment['p_offset'], fieldsize=6)
program_hdr['VirtAddr'] = self._format_hex(segment['p_vaddr'], fullhex=True)
program_hdr['PhysAddr'] = self._format_hex(segment['p_paddr'], fullhex=True)
program_hdr['FileSiz'] = self._format_hex(segment['p_filesz'], fieldsize=5)
program_hdr['MemSiz'] = self._format_hex(segment['p_memsz'], fieldsize=5)
program_hdr['Flg'] = describe_p_flags(segment['p_flags'])
program_hdr['Align'] = self._format_hex(segment['p_align'])
add_info(program_hdr)
return program_headers
def elf_section_headers(self):
section_headers = []
def add_info(dic):
section_headers.append(dic)
if self.elffile.num_sections() == 0:
return None
for nsec, section in enumerate(self.elffile.iter_sections()):
section_hdr = {}
section_hdr['index'] = nsec
section_hdr['Name'] = section.name
section_hdr['Type'] = describe_sh_type(section['sh_type'])
section_hdr['Addr'] = self._format_hex(section['sh_addr'], fieldsize=8, lead0x=False)
section_hdr['Offset'] = self._format_hex(section['sh_offset'], fieldsize=6, lead0x=False)
section_hdr['Size'] = self._format_hex(section['sh_size'], fieldsize=6, lead0x=False)
section_hdr['ES'] = self._format_hex(section['sh_entsize'], fieldsize=2, lead0x=False)
section_hdr['Flag'] = describe_sh_flags(section['sh_flags'])
section_hdr['Lk'] = section['sh_link']
section_hdr['Inf'] = section['sh_info']
section_hdr['Al'] = section['sh_addralign']
add_info(section_hdr)
return section_headers
def elf_symbol_tables(self):
symbol_tables_list = []
def add_info(dic):
symbol_tables_list.append(dic)
self._init_versioninfo()
symbol_tables = [s for s in self.elffile.iter_sections()
if isinstance(s, SymbolTableSection)]
if not symbol_tables and self.elffile.num_sections() == 0:
return None
for section in symbol_tables:
if not isinstance(section, SymbolTableSection):
continue
if section['sh_entsize'] == 0:
continue
for nsym, symbol in enumerate(section.iter_symbols()):
version_info = ''
if (section['sh_type'] == 'SHT_DYNSYM' and
self._versioninfo['type'] == 'GNU'):
version = self._symbol_version(nsym)
if (version['name'] != symbol.name and
version['index'] not in ('VER_NDX_LOCAL',
'VER_NDX_GLOBAL')):
if version['filename']:
# external symbol
version_info = '@%(name)s (%(index)i)' % version
else:
# internal symbol
if version['hidden']:
version_info = '@%(name)s' % version
else:
version_info = '@@%(name)s' % version
symbol_info = {}
symbol_info['index'] = nsym
symbol_info['Value'] = self._format_hex(
symbol['st_value'], fullhex=True, lead0x=False)
symbol_info['Size'] = symbol['st_size']
symbol_info['Type'] = describe_symbol_type(symbol['st_info']['type'])
symbol_info['Bind'] = describe_symbol_bind(symbol['st_info']['bind'])
symbol_info['Vis'] = describe_symbol_visibility(symbol['st_other']['visibility'])
symbol_info['Ndx'] = describe_symbol_shndx(symbol['st_shndx'])
symbol_info['Name'] = symbol.name
symbol_info['version_info'] = version_info
add_info(symbol_info)
return symbol_tables_list
def decode_flags(self, flags):
description = ""
if self.elffile['e_machine'] == "EM_ARM":
eabi = flags & E_FLAGS.EF_ARM_EABIMASK
flags &= ~E_FLAGS.EF_ARM_EABIMASK
if flags & E_FLAGS.EF_ARM_RELEXEC:
description += ', relocatable executabl'
flags &= ~E_FLAGS.EF_ARM_RELEXEC
if eabi == E_FLAGS.EF_ARM_EABI_VER5:
EF_ARM_KNOWN_FLAGS = E_FLAGS.EF_ARM_ABI_FLOAT_SOFT|E_FLAGS.EF_ARM_ABI_FLOAT_HARD|E_FLAGS.EF_ARM_LE8|E_FLAGS.EF_ARM_BE8
description += ', Version5 EABI'
if flags & E_FLAGS.EF_ARM_ABI_FLOAT_SOFT:
description += ", soft-float ABI"
elif flags & E_FLAGS.EF_ARM_ABI_FLOAT_HARD:
description += ", hard-float ABI"
if flags & E_FLAGS.EF_ARM_BE8:
description += ", BE8"
elif flags & E_FLAGS.EF_ARM_LE8:
description += ", LE8"
if flags & ~EF_ARM_KNOWN_FLAGS:
description += ', <unknown>'
else:
description += ', <unrecognized EABI>'
elif self.elffile['e_machine'] == "EM_MIPS":
if flags & E_FLAGS.EF_MIPS_NOREORDER:
description += ", noreorder"
if flags & E_FLAGS.EF_MIPS_PIC:
description += ", pic"
if flags & E_FLAGS.EF_MIPS_CPIC:
description += ", cpic"
if (flags & E_FLAGS.EF_MIPS_ABI2):
description += ", abi2"
if (flags & E_FLAGS.EF_MIPS_32BITMODE):
description += ", 32bitmode"
if (flags & E_FLAGS_MASKS.EFM_MIPS_ABI_O32):
description += ", o32"
elif (flags & E_FLAGS_MASKS.EFM_MIPS_ABI_O64):
description += ", o64"
elif (flags & E_FLAGS_MASKS.EFM_MIPS_ABI_EABI32):
description += ", eabi32"
elif (flags & E_FLAGS_MASKS.EFM_MIPS_ABI_EABI64):
description += ", eabi64"
if (flags & E_FLAGS.EF_MIPS_ARCH) == E_FLAGS.EF_MIPS_ARCH_1:
description += ", mips1"
if (flags & E_FLAGS.EF_MIPS_ARCH) == E_FLAGS.EF_MIPS_ARCH_2:
description += ", mips2"
if (flags & E_FLAGS.EF_MIPS_ARCH) == E_FLAGS.EF_MIPS_ARCH_3:
description += ", mips3"
if (flags & E_FLAGS.EF_MIPS_ARCH) == E_FLAGS.EF_MIPS_ARCH_4:
description += ", mips4"
if (flags & E_FLAGS.EF_MIPS_ARCH) == E_FLAGS.EF_MIPS_ARCH_5:
description += ", mips5"
if (flags & E_FLAGS.EF_MIPS_ARCH) == E_FLAGS.EF_MIPS_ARCH_32R2:
description += ", mips32r2"
if (flags & E_FLAGS.EF_MIPS_ARCH) == E_FLAGS.EF_MIPS_ARCH_64R2:
description += ", mips64r2"
if (flags & E_FLAGS.EF_MIPS_ARCH) == E_FLAGS.EF_MIPS_ARCH_32:
description += ", mips32"
if (flags & E_FLAGS.EF_MIPS_ARCH) == E_FLAGS.EF_MIPS_ARCH_64:
description += ", mips64"
return description
def _format_hex(self, addr, fieldsize=None, fullhex=False, lead0x=True,
alternate=False):
""" Format an address into a hexadecimal string.
fieldsize:
Size of the hexadecimal field (with leading zeros to fit the
address into. For example with fieldsize=8, the format will
be %08x
If None, the minimal required field size will be used.
fullhex:
If True, override fieldsize to set it to the maximal size
needed for the elfclass
lead0x:
If True, leading 0x is added
alternate:
If True, override lead0x to emulate the alternate
hexadecimal form specified in format string with the #
character: only non-zero values are prefixed with 0x.
This form is used by readelf.
"""
if alternate:
if addr == 0:
lead0x = False
else:
lead0x = True
fieldsize -= 2
s = '0x' if lead0x else ''
if fullhex:
fieldsize = 8 if self.elffile.elfclass == 32 else 16
if fieldsize is None:
field = '%x'
else:
field = '%' + '0%sx' % fieldsize
return s + field % addr
def _init_versioninfo(self):
""" Search and initialize informations about version related sections
and the kind of versioning used (GNU or Solaris).
"""
if self._versioninfo is not None:
return
self._versioninfo = {'versym': None, 'verdef': None,
'verneed': None, 'type': None}
for section in self.elffile.iter_sections():
if isinstance(section, GNUVerSymSection):
self._versioninfo['versym'] = section
elif isinstance(section, GNUVerDefSection):
self._versioninfo['verdef'] = section
elif isinstance(section, GNUVerNeedSection):
self._versioninfo['verneed'] = section
elif isinstance(section, DynamicSection):
for tag in section.iter_tags():
if tag['d_tag'] == 'DT_VERSYM':
self._versioninfo['type'] = 'GNU'
break
if not self._versioninfo['type'] and (
self._versioninfo['verneed'] or self._versioninfo['verdef']):
self._versioninfo['type'] = 'Solaris'
def _symbol_version(self, nsym):
""" Return a dict containing information on the
or None if no version information is available
"""
self._init_versioninfo()
symbol_version = dict.fromkeys(('index', 'name', 'filename', 'hidden'))
if (not self._versioninfo['versym'] or
nsym >= self._versioninfo['versym'].num_symbols()):
return None
symbol = self._versioninfo['versym'].get_symbol(nsym)
index = symbol.entry['ndx']
if not index in ('VER_NDX_LOCAL', 'VER_NDX_GLOBAL'):
index = int(index)
if self._versioninfo['type'] == 'GNU':
# In GNU versioning mode, the highest bit is used to
# store wether the symbol is hidden or not
if index & 0x8000:
index &= ~0x8000
symbol_version['hidden'] = True
if (self._versioninfo['verdef'] and
index <= self._versioninfo['verdef'].num_versions()):
_, verdaux_iter = \
self._versioninfo['verdef'].get_version(index)
symbol_version['name'] = next(verdaux_iter).name
else:
verneed, vernaux = \
self._versioninfo['verneed'].get_version(index)
symbol_version['name'] = vernaux.name
symbol_version['filename'] = verneed.name
symbol_version['index'] = index
return symbol_version
qiling/extensions/coverage/formats/base.py
+12 -11
View File
@@ -3,9 +3,14 @@
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from abc import ABC, abstractmethod
from __future__ import annotations
from qiling import Qiling
from abc import ABC, abstractmethod
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from qiling import Qiling
class QlBaseCoverage(ABC):
@@ -15,25 +20,21 @@ class QlBaseCoverage(ABC):
all the methods marked with the @abstractmethod decorator.
"""
FORMAT_NAME: str
def __init__(self, ql: Qiling):
super().__init__()
self.ql = ql
@property
@staticmethod
@abstractmethod
def FORMAT_NAME() -> str:
raise NotImplementedError
@abstractmethod
def activate(self):
def activate(self) -> None:
pass
@abstractmethod
def deactivate(self):
def deactivate(self) -> None:
pass
@abstractmethod
def dump_coverage(self, coverage_file: str):
def dump_coverage(self, coverage_file: str) -> None:
pass
qiling/extensions/coverage/formats/drcov.py
+56 -23
View File
@@ -3,12 +3,20 @@
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from ctypes import Structure
from ctypes import c_uint32, c_uint16
from __future__ import annotations
from ctypes import Structure, c_uint32, c_uint16
from functools import lru_cache
from typing import TYPE_CHECKING, BinaryIO, Dict, Tuple
from .base import QlBaseCoverage
if TYPE_CHECKING:
from qiling import Qiling
from qiling.loader.loader import QlLoader
# Adapted from https://www.ayrx.me/drcov-file-format
class bb_entry(Structure):
_fields_ = [
@@ -40,36 +48,61 @@ class QlDrCoverage(QlBaseCoverage):
FORMAT_NAME = "drcov"
def __init__(self, ql):
def __init__(self, ql: Qiling):
super().__init__(ql)
self.drcov_version = 2
self.drcov_flavor = 'drcov'
self.basic_blocks = []
self.basic_blocks: Dict[int, bb_entry] = {}
self.bb_callback = None
@staticmethod
def block_callback(ql, address, size, self):
for mod_id, mod in enumerate(ql.loader.images):
if mod.base <= address <= mod.end:
ent = bb_entry(address - mod.base, size, mod_id)
self.basic_blocks.append(ent)
break
@lru_cache(maxsize=64)
def _get_img_base(self, loader: QlLoader, address: int) -> Tuple[int, int]:
"""Retrieve the containing image of a given address.
def activate(self):
self.bb_callback = self.ql.hook_block(self.block_callback, user_data=self)
Addresses are expected to be aligned to page boundary, and cached for faster retrieval.
"""
def deactivate(self):
self.ql.hook_del(self.bb_callback)
return next((i, img.base) for i, img in enumerate(loader.images) if img.base <= address < img.end)
def block_callback(self, ql: Qiling, address: int, size: int):
if address not in self.basic_blocks:
try:
# we rely on the fact that images are allocated on page size boundary and
# use it to speed up image retrieval. we align the basic block address to
# page boundary, knowing basic blocks within the same page belong to the
# same image. then we use the aligned address to retreive the containing
# image. returned values are cached so subsequent retrievals for basic
# blocks within the same page will return the cached value instead of
# going through the retreival process again (up to maxsize cached pages)
i, img_base = self._get_img_base(ql.loader, address & ~(0x1000 - 1))
except StopIteration:
pass
else:
self.basic_blocks[address] = bb_entry(address - img_base, size, i)
def activate(self) -> None:
self.bb_callback = self.ql.hook_block(self.block_callback)
def deactivate(self) -> None:
if self.bb_callback:
self.ql.hook_del(self.bb_callback)
def dump_coverage(self, coverage_file: str) -> None:
def __write_line(bio: BinaryIO, line: str) -> None:
bio.write(f'{line}\n'.encode())
def dump_coverage(self, coverage_file):
with open(coverage_file, "wb") as cov:
cov.write(f"DRCOV VERSION: {self.drcov_version}\n".encode())
cov.write(f"DRCOV FLAVOR: {self.drcov_flavor}\n".encode())
cov.write(f"Module Table: version {self.drcov_version}, count {len(self.ql.loader.images)}\n".encode())
cov.write("Columns: id, base, end, entry, checksum, timestamp, path\n".encode())
__write_line(cov, f"DRCOV VERSION: {self.drcov_version}")
__write_line(cov, f"DRCOV FLAVOR: {self.drcov_flavor}")
__write_line(cov, f"Module Table: version {self.drcov_version}, count {len(self.ql.loader.images)}")
__write_line(cov, "Columns: id, base, end, entry, checksum, timestamp, path")
for mod_id, mod in enumerate(self. ql.loader.images):
cov.write(f"{mod_id}, {mod.base}, {mod.end}, 0, 0, 0, {mod.path}\n".encode())
cov.write(f"BB Table: {len(self.basic_blocks)} bbs\n".encode())
for bb in self.basic_blocks:
__write_line(cov, f"{mod_id}, {mod.base}, {mod.end}, 0, 0, 0, {mod.path}")
__write_line(cov, f"BB Table: {len(self.basic_blocks)} bbs")
for bb in self.basic_blocks.values():
cov.write(bytes(bb))
qiling/extensions/coverage/formats/drcov_exact.py
+3 -7
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
#
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
@@ -17,10 +17,6 @@ class QlDrCoverageExact(QlDrCoverage):
FORMAT_NAME = "drcov_exact"
def __init__(self, ql):
super().__init__(ql)
def activate(self):
def activate(self) -> None:
# We treat every instruction as a block on its own.
self.bb_callback = self.ql.hook_code(self.block_callback, user_data=self)
self.bb_callback = self.ql.hook_code(self.block_callback)
qiling/extensions/coverage/formats/ezcov.py
+35 -23
View File
@@ -1,19 +1,30 @@
#!/usr/bin/env python3
#
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from collections import namedtuple
from os.path import basename
from __future__ import annotations
import os
from typing import Any, TYPE_CHECKING, List, NamedTuple
from .base import QlBaseCoverage
if TYPE_CHECKING:
from qiling import Qiling
# Adapted from https://github.com/nccgroup/Cartographer/blob/main/EZCOV.md#coverage-data
class bb_entry(namedtuple('bb_entry', 'offset size mod_id')):
def csvline(self):
offset = '0x{:08x}'.format(self.offset)
class bb_entry(NamedTuple):
offset: int
size: int
mod_id: Any
def as_csv(self) -> str:
offset = f'{self.offset:#010x}'
mod_id = f"[ {self.mod_id if self.mod_id is not None else ''} ]"
return f"{offset},{self.size},{mod_id}\n"
class QlEzCoverage(QlBaseCoverage):
@@ -27,29 +38,30 @@ class QlEzCoverage(QlBaseCoverage):
FORMAT_NAME = "ezcov"
def __init__(self, ql):
def __init__(self, ql: Qiling):
super().__init__(ql)
self.ezcov_version = 1
self.ezcov_flavor = 'ezcov'
self.basic_blocks = []
self.bb_callback = None
self.ezcov_flavor = 'ezcov'
self.basic_blocks: List[bb_entry] = []
self.bb_callback = None
@staticmethod
def block_callback(ql, address, size, self):
mod = ql.loader.find_containing_image(address)
if mod is not None:
ent = bb_entry(address - mod.base, size, basename(mod.path))
self.basic_blocks.append(ent)
def block_callback(self, ql: Qiling, address: int, size: int):
img = ql.loader.find_containing_image(address)
def activate(self):
self.bb_callback = self.ql.hook_block(self.block_callback, user_data=self)
if img is not None:
self.basic_blocks.append(bb_entry(address - img.base, size, os.path.basename(img.path)))
def deactivate(self):
self.ql.hook_del(self.bb_callback)
def activate(self) -> None:
self.bb_callback = self.ql.hook_block(self.block_callback)
def dump_coverage(self, coverage_file):
def deactivate(self) -> None:
if self.bb_callback:
self.ql.hook_del(self.bb_callback)
def dump_coverage(self, coverage_file: str) -> None:
with open(coverage_file, "w") as cov:
cov.write(f"EZCOV VERSION: {self.ezcov_version}\n")
cov.write("# Qiling EZCOV exporter tool\n")
for bb in self.basic_blocks:
cov.write(bb.csvline())
cov.writelines(bb.as_csv() for bb in self.basic_blocks)
qiling/extensions/tracing/formats/base.py
+7 -11
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
#
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
# This code structure is copied and modified from the coverage extension
@@ -12,24 +12,20 @@ class QlBaseTrace(ABC):
To add support for a new coverage format, just derive from this class and implement
all the methods marked with the @abstractmethod decorator.
"""
FORMAT_NAME: str
def __init__(self):
super().__init__()
@property
@staticmethod
@abstractmethod
def FORMAT_NAME():
raise NotImplementedError
@abstractmethod
def activate(self):
def activate(self) -> None:
pass
@abstractmethod
def deactivate(self):
def deactivate(self) -> None:
pass
@abstractmethod
def dump_trace(self, trace_file):
def dump_trace(self, trace_file: str) -> None:
pass
qiling/hw/hw.py
+108 -59
View File
@@ -1,26 +1,70 @@
#!/usr/bin/env python3
#
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
import ctypes
from functools import cached_property
from typing import Any, Dict, List, Optional, Tuple
from qiling.core import Qiling
from qiling import Qiling
from qiling.hw.peripheral import QlPeripheral
from qiling.utils import ql_get_module_function
from qiling.exception import QlErrorModuleFunctionNotFound
# should adhere to the QlMmioHandler interface, but not extend it directly to
# avoid potential pickling issues
class QlPripheralHandler:
def __init__(self, hwman: "QlHwManager", base: int, size: int, label: str) -> None:
self._hwman = hwman
self._base = base
self._size = size
self._label = label
def __getstate__(self):
state = self.__dict__.copy()
del state['_hwman'] # remove non-pickleable reference
return state
@cached_property
def _mmio(self) -> bytearray:
"""Get memory buffer used to back non-mapped hardware mmio regions.
"""
return bytearray(self._size)
def read(self, ql: Qiling, offset: int, size: int) -> int:
address = self._base + offset
hardware = self._hwman.find(address)
if hardware:
return hardware.read(address - hardware.base, size)
else:
ql.log.debug('[%s] read non-mapped hardware [%#010x]', self._label, address)
return int.from_bytes(self._mmio[offset:offset + size], byteorder='little')
def write(self, ql: Qiling, offset: int, size: int, value: int) -> None:
address = self._base + offset
hardware = self._hwman.find(address)
if hardware:
hardware.write(address - hardware.base, size, value)
else:
ql.log.debug('[%s] write non-mapped hardware [%#010x] = %#010x', self._label, address, value)
self._mmio[offset:offset + size] = value.to_bytes(size, 'little')
class QlHwManager:
def __init__(self, ql: Qiling):
self.ql = ql
self.entity = {}
self.region = {}
self.entity: Dict[str, QlPeripheral] = {}
self.region: Dict[str, List[Tuple[int, int]]] = {}
self.stepable = {}
def create(self, label: str, struct: str=None, base: int=None, kwargs: dict={}) -> "QlPeripheral":
def create(self, label: str, struct: Optional[str] = None, base: Optional[int] = None, kwargs: Optional[Dict[str, Any]] = None) -> QlPeripheral:
""" Create the peripheral accroding the label and envs.
struct: Structure of the peripheral. Use defualt ql structure if not provide.
@@ -30,39 +74,45 @@ class QlHwManager:
if struct is None:
struct, base, kwargs = self.load_env(label.upper())
try:
entity = ql_get_module_function('qiling.hw', struct)(self.ql, label, **kwargs)
self.entity[label] = entity
if hasattr(entity, 'step'):
self.stepable[label] = entity
if kwargs is None:
kwargs = {}
try:
entity = ql_get_module_function('qiling.hw', struct)(self.ql, label, **kwargs)
except QlErrorModuleFunctionNotFound:
self.ql.log.warning(f'could not create {struct}({label}): implementation not found')
else:
assert isinstance(entity, QlPeripheral)
assert isinstance(base, int)
self.entity[label] = entity
self.region[label] = [(lbound + base, rbound + base) for (lbound, rbound) in entity.region]
return entity
except QlErrorModuleFunctionNotFound:
self.ql.log.debug(f'The {struct}({label}) has not been implemented')
def delete(self, label: str):
# FIXME: what should we do if struct is not implemented? is it OK to return None , or we fail?
def delete(self, label: str) -> None:
""" Remove the peripheral
"""
if label in self.entity:
self.entity.pop(label)
self.region.pop(label)
if label in self.stepable:
self.stepable.pop(label)
def load_env(self, label: str):
if label in self.entity:
del self.entity[label]
if label in self.region:
del self.region[label]
def load_env(self, label: str) -> Tuple[str, int, Dict[str, Any]]:
""" Get peripheral information (structure, base address, initialization list) from env.
Args:
label (str): Peripheral Label
"""
args = self.ql.env[label]
return args['struct'], args['base'], args.get("kwargs", {})
def load_all(self):
@@ -70,48 +120,30 @@ class QlHwManager:
if args['type'] == 'peripheral':
self.create(label.lower(), args['struct'], args['base'], args.get("kwargs", {}))
def find(self, address: int):
# TODO: this is wasteful. device mapping is known at creation time. at least we could cache lru entries
def find(self, address: int) -> Optional[QlPeripheral]:
""" Find the peripheral at `address`
"""
for label in self.entity.keys():
for lbound, rbound in self.region[label]:
if lbound <= address < rbound:
return self.entity[label]
return None
def step(self):
""" Update all peripheral's state
""" Update all peripheral's state
"""
for entity in self.stepable.values():
entity.step()
def setup_mmio(self, begin, size, info=""):
mmio = ctypes.create_string_buffer(size)
for ent in self.entity.values():
if hasattr(ent, 'step'):
ent.step()
def mmio_read_cb(ql, offset, size):
address = begin + offset
hardware = self.find(address)
if hardware:
return hardware.read(address - hardware.base, size)
else:
ql.log.debug('%s Read non-mapped hardware [0x%08x]' % (info, address))
buf = ctypes.create_string_buffer(size)
ctypes.memmove(buf, ctypes.addressof(mmio) + offset, size)
return int.from_bytes(buf.raw, byteorder='little')
def setup_mmio(self, begin: int, size: int, info: str) -> None:
dev = QlPripheralHandler(self, begin, size, info)
def mmio_write_cb(ql, offset, size, value):
address = begin + offset
hardware = self.find(address)
if hardware:
hardware.write(address - hardware.base, size, value)
else:
ql.log.debug('%s Write non-mapped hardware [0x%08x] = 0x%08x' % (info, address, value))
ctypes.memmove(ctypes.addressof(mmio) + offset, (value).to_bytes(size, 'little'), size)
self.ql.mem.map_mmio(begin, size, mmio_read_cb, mmio_write_cb, info=info)
self.ql.mem.map_mmio(begin, size, dev, info)
def show_info(self):
self.ql.log.info(f'{"Start":8s} {"End":8s} {"Label":8s} {"Class"}')
@@ -131,8 +163,25 @@ class QlHwManager:
return self.entity.get(key)
def save(self):
return {label : entity.save() for label, entity in self.entity.items()}
return {
'entity': {label: entity.save() for label, entity in self.entity.items()},
'region': self.region
}
def restore(self, saved_state):
for label, data in saved_state.items():
entity = saved_state['entity']
assert isinstance(entity, dict)
region = saved_state['region']
assert isinstance(region, dict)
for label, data in entity.items():
self.entity[label].restore(data)
self.region = region
# a dirty hack to rehydrate non-pickleable hwman
# a proper fix would require a deeper refactoring to how peripherals are created and managed
for ph in self.ql.mem.mmio_cbs.values():
if isinstance(ph, QlPripheralHandler):
setattr(ph, '_hwman', self)
qiling/loader/blob.py
+13 -10
View File
@@ -4,8 +4,8 @@
#
from qiling import Qiling
from qiling.loader.loader import QlLoader
from qiling.os.memory import QlMemoryHeap
from qiling.loader.loader import QlLoader, Image
class QlLoaderBLOB(QlLoader):
def __init__(self, ql: Qiling):
@@ -14,15 +14,18 @@ class QlLoaderBLOB(QlLoader):
self.load_address = 0
def run(self):
self.load_address = self.ql.os.entry_point # for consistency
self.load_address = self.ql.os.load_address
self.entry_point = self.ql.os.entry_point
self.ql.mem.map(self.ql.os.entry_point, self.ql.os.code_ram_size, info="[code]")
self.ql.mem.write(self.ql.os.entry_point, self.ql.code)
code_begins = self.load_address
code_size = self.ql.os.code_ram_size
code_ends = code_begins + code_size
heap_address = self.ql.os.entry_point + self.ql.os.code_ram_size
heap_size = int(self.ql.os.profile.get("CODE", "heap_size"), 16)
self.ql.os.heap = QlMemoryHeap(self.ql, heap_address, heap_address + heap_size)
self.ql.mem.map(code_begins, code_size, info="[code]")
self.ql.mem.write(code_begins, self.ql.code)
self.ql.arch.regs.arch_sp = heap_address - 0x1000
# allow image-related functionalities
self.images.append(Image(code_begins, code_ends, 'blob_code'))
return
# FIXME: stack pointer should be a configurable profile setting
self.ql.arch.regs.arch_sp = code_ends - 0x1000
qiling/loader/elf.py
+14 -2
View File
@@ -7,7 +7,7 @@ import io
import os
from enum import IntEnum
from typing import AnyStr, Optional, Sequence, Mapping, Tuple
from typing import Any, AnyStr, Optional, Sequence, Mapping, Tuple
from elftools.common.utils import preserve_stream_pos
from elftools.elf.constants import P_FLAGS, SH_FLAGS
@@ -330,7 +330,7 @@ class QlLoaderELF(QlLoader):
hwcap_values = {
(QL_ARCH.ARM, QL_ENDIAN.EL, 32): 0x001fb8d7,
(QL_ARCH.ARM, QL_ENDIAN.EB, 32): 0xd7b81f00,
(QL_ARCH.ARM64, QL_ENDIAN.EL, 64): 0x078bfbfd
(QL_ARCH.ARM64, QL_ENDIAN.EL, 64): 0x078bfafd
}
# determine hwcap value by arch properties; if not found default to 0
@@ -701,3 +701,15 @@ class QlLoaderELF(QlLoader):
elfdata_mapping.extend(sec.data())
return bytes(elfdata_mapping)
def save(self) -> Mapping[str, Any]:
saved = super().save()
saved['brk_address'] = self.brk_address
return saved
def restore(self, saved_state: Mapping[str, Any]):
self.brk_address = saved_state['brk_address']
super().restore(saved_state)
qiling/loader/mcu.py
+33 -29
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
#
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
# Built on top of Unicorn emulator (www.unicorn-engine.org)
# Built on top of Unicorn emulator (www.unicorn-engine.org)
import io
@@ -27,7 +27,7 @@ class IhexParser:
if addr != begin + len(stream):
self.segments.append((begin, stream))
begin, stream = addr, data
else:
stream += data
@@ -36,13 +36,13 @@ class IhexParser:
def parse_line(self, line):
if len(line) < 9:
return
desc = line[7: 9]
size = int(line[1: 3], 16)
size = int(line[1: 3], 16)
addr = bytes.fromhex(line[3: 7])
data = bytes.fromhex(line[9: 9 + size * 2])
data = bytes.fromhex(line[9: 9 + size * 2])
if desc == '00': # Data
offset = int.from_bytes(addr, byteorder='big')
self.mem.append((self.base + offset, data))
@@ -52,20 +52,20 @@ class IhexParser:
elif desc == '04': # Extended Linear Address
self.base = int.from_bytes(data, byteorder='big') * 0x10000
class QlLoaderMCU(QlLoader):
def __init__(self, ql:Qiling):
super().__init__(ql)
super().__init__(ql)
self.entry_point = 0
self.load_address = 0
self.filetype = self.guess_filetype()
if self.filetype == 'elf':
with open(self.ql.path, 'rb') as infile:
self.elf = ELFFile(io.BytesIO(infile.read()))
elif self.filetype == 'bin':
self.map_address = self.argv[1]
@@ -74,8 +74,8 @@ class QlLoaderMCU(QlLoader):
def guess_filetype(self):
if self.ql.path.endswith('.elf'):
return 'elf'
return 'elf'
if self.ql.path.endswith('.bin'):
return 'bin'
@@ -83,7 +83,7 @@ class QlLoaderMCU(QlLoader):
return 'hex'
return 'elf'
def reset(self):
if self.filetype == 'elf':
for segment in self.elf.iter_segments(type='PT_LOAD'):
@@ -99,7 +99,7 @@ class QlLoaderMCU(QlLoader):
for begin, data in self.ihex.segments:
self.ql.mem.write(begin, data)
self.ql.arch.init_context()
self.entry_point = self.ql.arch.regs.read('pc')
@@ -109,30 +109,34 @@ class QlLoaderMCU(QlLoader):
def load_env(self):
for name, args in self.env.items():
memtype = args['type']
if memtype == 'memory':
size = args['size']
base = args['base']
self.ql.mem.map(base, size, info=f'[{name}]')
if memtype == 'remap':
# elif memtype == 'remap':
# size = args['size']
# base = args['base']
# alias = args['alias']
# self.ql.hw.setup_remap(alias, base, size, info=f'[{name}]')
elif memtype == 'mmio':
size = args['size']
base = args['base']
alias = args['alias']
self.ql.hw.setup_remap(alias, base, size, info=f'[{name}]')
self.ql.hw.setup_mmio(base, size, name)
if memtype == 'mmio':
size = args['size']
base = args['base']
self.ql.hw.setup_mmio(base, size, info=f'[{name}]')
if memtype == 'core':
elif memtype == 'core':
self.ql.hw.create(name.lower())
else:
self.ql.log.debug(f'ignoring unknown memory type "{memtype}" for {name}')
def run(self):
self.load_profile()
self.load_env()
## Handle interrupt from instruction execution
self.ql.hook_intr(self.ql.arch.unicorn_exception_handler)
self.reset()
qiling/loader/pe.py
+162 -4
View File
@@ -10,7 +10,8 @@ import pefile
import pickle
import secrets
import ntpath
from typing import TYPE_CHECKING, Any, Dict, MutableMapping, NamedTuple, Optional, Mapping, Sequence, Tuple, Union
from collections import namedtuple
from typing import TYPE_CHECKING, Any, Dict, List, MutableMapping, NamedTuple, Optional, Mapping, Sequence, Tuple, Union
from unicorn import UcError
from unicorn.x86_const import UC_X86_REG_CR4, UC_X86_REG_CR8
@@ -29,6 +30,13 @@ if TYPE_CHECKING:
from logging import Logger
from qiling import Qiling
class ForwardedExport(NamedTuple):
source_dll: str
source_ordinal: str
source_symbol: str
target_dll: str
target_symbol: str
class QlPeCacheEntry(NamedTuple):
ba: int
@@ -79,6 +87,16 @@ class Process:
export_symbols: MutableMapping[int, Dict[str, Any]]
libcache: Optional[QlPeCache]
# maps image base to RVA of its function table
function_table_lookup: Dict[int, int]
# maps image base to its list of function table entries
function_tables: MutableMapping[int, List]
# List of exports which have been forwarded from
# one DLL to another.
forwarded_exports: List[ForwardedExport]
def __init__(self, ql: Qiling):
self.ql = ql
@@ -105,6 +123,108 @@ class Process:
vpath = ntpath.join(dirname, basename)
return self.ql.os.path.virtual_to_host_path(vpath), basename.casefold()
def init_function_tables(self, pe: pefile.PE, image_base: int):
"""Parse function table data for the given PE file.
Only really relevant for non-x86 images.
Args:
pe: the PE image whose function data should be parsed
image_base: the absolute address at which the image was loaded
"""
if self.ql.arch.type is not QL_ARCH.X86:
# Check if the PE file has an exception directory
if hasattr(pe, 'DIRECTORY_ENTRY_EXCEPTION'):
exception_dir = pe.OPTIONAL_HEADER.DATA_DIRECTORY[
pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_EXCEPTION']
]
self.function_table_lookup[image_base] = exception_dir.VirtualAddress
runtime_function_list = list(pe.DIRECTORY_ENTRY_EXCEPTION)
if image_base not in self.function_tables:
self.function_tables[image_base] = []
self.function_tables[image_base].extend(runtime_function_list)
self.ql.log.debug(f'Parsed {len(runtime_function_list)} exception directory entries')
else:
self.ql.log.debug(f'Image has no exception directory; skipping exception data')
def lookup_function_entry(self, base_addr: int, control_pc: int):
"""Look up a RUNTIME_FUNCTION entry and its index in a module's
function table, such that the given program counter falls within
the entry's begin and end range.
Args:
base_addr: The base address of the image whose exception directory to search.
control_pc: The program counter.
Returns:
A tuple (index, runtime_function)
"""
function_table = self.function_tables[base_addr]
# Initiate a search of the function table for a RUNTIME_FUNCTION
# entry such that the provided PC falls within its start and end range.
return next(((i, rtfunc) for i, rtfunc in enumerate(function_table)
if rtfunc.struct.BeginAddress <= control_pc - base_addr < rtfunc.struct.EndAddress),
(None, None))
def resolve_forwarded_exports(self):
while self.forwarded_exports:
forwarded_export = self.forwarded_exports.pop()
source_dll = forwarded_export.source_dll
source_ordinal = forwarded_export.source_ordinal
source_symbol = forwarded_export.source_symbol
target_dll = forwarded_export.target_dll
target_symbol = forwarded_export.target_symbol
if not source_symbol:
# Some DLLs (shlwapi.dll) have a bunch of forwarded
# exports with ordinals but no symbols.
# These are really annoying to deal with, but they are
# used extremely rarely, so we will ignore them.
continue
target_iat = self.import_address_table.get(target_dll)
if not target_iat:
# If IAT was not found, it is probably a virtual library.
continue
# If we have an existing entry in the process IAT for the code
# this entry forwards to, then we will point the symbol there
# rather than the symbol string in the exporter's data section.
forward_ea = target_iat.get(target_symbol)
if not forward_ea:
self.ql.log.warning(f"Forwarding symbol {source_dll}.{source_symbol} to {target_dll}.{target_symbol}: Failed to resolve address")
continue
self.import_address_table[source_dll][source_symbol] = forward_ea
self.import_address_table[source_dll][source_ordinal] = forward_ea
# Register the new address as having the source symbol/ordinal.
# This way, hooks on forward source symbols will function
# correctly.
self.import_symbols[forward_ea] = {
'name' : source_symbol,
'ordinal' : source_ordinal,
'dll' : source_dll.split('.')[0]
}
# TODO: With the above code, hooks on functions which are
# forward targets may not work correctly.
# The most correct way to resolve this would be to add
# support for addresses to be associated with multiple symbols.
self.ql.log.debug(f"Forwarding symbol {source_dll}.{source_symbol} to {target_dll}.{target_symbol}: Resolved symbol to ({forward_ea:#x})")
def load_dll(self, name: str, is_driver: bool = False) -> int:
dll_path, dll_name = self.__get_path_elements(name)
@@ -195,6 +315,9 @@ class Process:
with ShowProgress(self.ql.log, 0.1337):
dll.relocate_image(image_base)
# initialize the function tables only after possible relocation
self.init_function_tables(dll, image_base)
data = bytearray(dll.get_memory_mapped_image())
assert image_size >= len(data)
@@ -203,6 +326,31 @@ class Process:
for sym in dll.DIRECTORY_ENTRY_EXPORT.symbols:
ea = image_base + sym.address
if sym.forwarder:
# Some exports are forwarders, meaning they
# actually refer to code in other libraries.
#
# For example, calls to
# kernel32.InterlockedPushEntrySList
# should be forwarded to
# ntdll.RtlInterlockedPushEntrySList
#
# If we do not properly account for forwarders then
# calls to these symbols will land in the exporter's
# data section and cause a lot of problems.
forward_str = sym.forwarder
if b'.' in forward_str:
target_dll_name, target_symbol_name = forward_str.split(b'.', 1)
target_dll_filename = (target_dll_name.lower() + b'.dll').decode()
# Remember the forwarded export for later.
forwarded_export = ForwardedExport(dll_name, sym.ordinal, sym.name,
target_dll_filename, target_symbol_name)
self.forwarded_exports.append(forwarded_export)
import_symbols[ea] = {
'name' : sym.name,
'ordinal' : sym.ordinal,
@@ -227,6 +375,8 @@ class Process:
self.import_address_table[dll_name] = import_table
self.import_symbols.update(import_symbols)
self.resolve_forwarded_exports()
dll_base = image_base
dll_len = image_size
@@ -281,8 +431,8 @@ class Process:
# the blacklist may be revisited from time to time to see if any of the file
# can be safely unlisted.
blacklist = {
32 : ('gdi32.dll',),
64 : ('gdi32.dll',)
32 : ('gdi32.dll','user32.dll',),
64 : ('gdi32.dll','user32.dll',)
}[self.ql.arch.bits]
if dll_name in blacklist:
@@ -674,12 +824,14 @@ class QlLoaderPE(QlLoader, Process):
def run(self):
self.init_dlls = (
'ntdll.dll',
'kernel32.dll',
'kernelbase.dll', # kernel32 forwards some exports to kernelbase
'kernel32.dll', # for efficiency, load kernelbase first
'user32.dll'
)
self.sys_dlls = (
'ntdll.dll',
'kernelbase.dll',
'kernel32.dll',
'mscoree.dll',
'ucrtbase.dll'
@@ -709,6 +861,9 @@ class QlLoaderPE(QlLoader, Process):
self.export_symbols = {}
self.import_address_table = {}
self.ldr_list = []
self.function_tables = {}
self.function_table_lookup = {}
self.forwarded_exports = []
self.pe_image_address = 0
self.pe_image_size = 0
self.dll_size = 0
@@ -841,6 +996,9 @@ class QlLoaderPE(QlLoader, Process):
# set up call frame for DllMain
self.ql.os.fcall.call_native(self.entry_point, args, None)
# Initialize the function tables
super().init_function_tables(pe, image_base)
elif pe is None:
self.ql.mem.map(self.entry_point, self.ql.os.code_ram_size, info="[shellcode]")
qiling/os/blob/blob.py
+16 -5
View File
@@ -8,6 +8,8 @@ from qiling.cc import QlCC, intel, arm, mips, riscv, ppc
from qiling.const import QL_ARCH, QL_OS
from qiling.os.fcall import QlFunctionCall
from qiling.os.os import QlOs
from qiling.os.memory import QlMemoryHeap
class QlOsBlob(QlOs):
""" QlOsBlob for bare barines.
@@ -21,7 +23,7 @@ class QlOsBlob(QlOs):
type = QL_OS.BLOB
def __init__(self, ql: Qiling):
super(QlOsBlob, self).__init__(ql)
super().__init__(ql)
self.ql = ql
@@ -39,11 +41,20 @@ class QlOsBlob(QlOs):
self.fcall = QlFunctionCall(ql, cc)
def run(self):
if self.ql.entry_point:
# if entry point was set explicitly, override the default one
if self.ql.entry_point is not None:
self.entry_point = self.ql.entry_point
self.exit_point = self.ql.loader.load_address + len(self.ql.code)
if self.ql.exit_point:
self.exit_point = self.ql.exit_point
self.exit_point = self.load_address + len(self.ql.code)
# if exit point was set explicitly, override the default one
if self.ql.exit_point is not None:
self.exit_point = self.ql.exit_point
# if heap info is provided in profile, create heap
heap_base = self.profile.getint('CODE', 'heap_address', fallback=None)
heap_size = self.profile.getint('CODE', 'heap_size', fallback=None)
if heap_base is not None and heap_size is not None:
self.heap = QlMemoryHeap(self.ql, heap_base, heap_base + heap_size)
self.ql.emu_start(self.entry_point, self.exit_point, self.ql.timeout, self.ql.count)
qiling/os/memory.py
+27 -11
View File
@@ -6,7 +6,7 @@
import bisect
import os
import re
from typing import Any, Callable, Iterator, List, Mapping, Optional, Pattern, Sequence, Tuple, Union
from typing import Any, Callable, Dict, Iterator, List, Mapping, Optional, Pattern, Protocol, Sequence, Tuple, Union
from unicorn import UC_PROT_NONE, UC_PROT_READ, UC_PROT_WRITE, UC_PROT_EXEC, UC_PROT_ALL
@@ -20,6 +20,22 @@ MmioReadCallback = Callable[[Qiling, int, int], int]
MmioWriteCallback = Callable[[Qiling, int, int, int], None]
class QlMmioHandler(Protocol):
"""A simple MMIO handler boilerplate that can be used to implement memory mapped devices.
This should be extended to implement mapped devices state machines. Note that the read and write
methods are optional, where their existance indicates whether the device supports the corresponding
operation. That is, an unimplemented method means the corresponding operation will be silently
dropped.
"""
def read(self, ql: Qiling, offset: int, size: int) -> int:
...
def write(self, ql: Qiling, offset: int, size: int, value: int) -> None:
...
class QlMemoryManager:
"""
some ideas and code from:
@@ -29,7 +45,7 @@ class QlMemoryManager:
def __init__(self, ql: Qiling, pagesize: int = 0x1000):
self.ql = ql
self.map_info: List[MapInfoEntry] = []
self.mmio_cbs = {}
self.mmio_cbs: Dict[Tuple[int, int], QlMmioHandler] = {}
bit_stuff = {
64: (1 << 64) - 1,
@@ -57,7 +73,7 @@ class QlMemoryManager:
addr += 1
c = self.read(addr, 1)
return ret.decode()
return ret.decode('latin1')
def __write_string(self, addr: int, s: str, encoding: str):
self.write(addr, bytes(s, encoding) + b'\x00')
@@ -272,7 +288,7 @@ class QlMemoryManager:
for lbound, ubound, perm, label, is_mmio in self.map_info:
if is_mmio:
mem_dict['mmio'].append((lbound, ubound, perm, label, *self.mmio_cbs[(lbound, ubound)]))
mem_dict['mmio'].append((lbound, ubound, perm, label, self.mmio_cbs[(lbound, ubound)]))
else:
data = self.read(lbound, ubound - lbound)
mem_dict['ram'].append((lbound, ubound, perm, label, bytes(data)))
@@ -294,12 +310,12 @@ class QlMemoryManager:
self.ql.log.debug(f'writing {len(data):#x} bytes at {lbound:#08x}')
self.write(lbound, data)
for lbound, ubound, perms, label, read_cb, write_cb in mem_dict['mmio']:
for lbound, ubound, perms, label, handler in mem_dict['mmio']:
self.ql.log.debug(f"restoring mmio range: {lbound:#08x} {ubound:#08x} {label}")
size = ubound - lbound
if not self.is_mapped(lbound, size):
self.map_mmio(lbound, size, read_cb, write_cb, info=label)
self.map_mmio(lbound, size, handler, label)
def read(self, addr: int, size: int) -> bytearray:
"""Read bytes from memory.
@@ -619,15 +635,15 @@ class QlMemoryManager:
self.ql.uc.mem_map(addr, size, perms)
self.add_mapinfo(addr, addr + size, perms, info or '[mapped]', is_mmio=False)
def map_mmio(self, addr: int, size: int, read_cb: Optional[MmioReadCallback], write_cb: Optional[MmioWriteCallback], info: str = '[mmio]'):
def map_mmio(self, addr: int, size: int, handler: QlMmioHandler, info: str = '[mmio]'):
# TODO: mmio memory overlap with ram? Is that possible?
# TODO: Can read_cb or write_cb be None? How uc handle that access?
prot = UC_PROT_NONE
if read_cb:
if hasattr(handler, 'read'):
prot |= UC_PROT_READ
if write_cb:
if hasattr(handler, 'write'):
prot |= UC_PROT_WRITE
# generic mmio read wrapper
@@ -642,10 +658,10 @@ class QlMemoryManager:
cb(self.ql, offset, size, value)
self.ql.uc.mmio_map(addr, size, __mmio_read, read_cb, __mmio_write, write_cb)
self.ql.uc.mmio_map(addr, size, __mmio_read, handler.read, __mmio_write, handler.write)
self.add_mapinfo(addr, addr + size, prot, info, is_mmio=True)
self.mmio_cbs[(addr, addr + size)] = (read_cb, write_cb)
self.mmio_cbs[(addr, addr + size)] = handler
class Chunk:
qiling/os/os.py
+1
View File
@@ -89,6 +89,7 @@ class QlOs:
if self.ql.code:
# this shellcode entrypoint does not work for windows
# windows shellcode entry point will comes from pe loader
self.load_address = self.profile.getint('CODE', 'load_address')
self.entry_point = self.profile.getint('CODE', 'entry_point')
self.code_ram_size = self.profile.getint('CODE', 'ram_size')
qiling/os/posix/const.py
+26 -21
View File
@@ -17,6 +17,9 @@ THREAD_EVENT_EXIT_GROUP_EVENT = 6
# File Open Limits
NR_OPEN = 1024
# number of signals
NSIG = 32
SOCK_TYPE_MASK = 0x0f
class linux_x86_socket_types(Enum):
@@ -459,6 +462,8 @@ class QlPrettyFlag(Flag):
# open flags #
################################
FLAG_UNSUPPORTED = -1
class macos_x86_open_flags(QlPrettyFlag):
O_RDONLY = 0x000000
O_WRONLY = 0x000001
@@ -473,8 +478,8 @@ class macos_x86_open_flags(QlPrettyFlag):
O_EXCL = 0x000800
O_NOCTTY = 0x020000
O_DIRECTORY = 0x100000
O_BINARY = 0x000000
O_LARGEFILE = 0x000000
O_BINARY = FLAG_UNSUPPORTED
O_LARGEFILE = FLAG_UNSUPPORTED
class linux_x86_open_flags(QlPrettyFlag):
@@ -491,8 +496,8 @@ class linux_x86_open_flags(QlPrettyFlag):
O_EXCL = 0x000080
O_NOCTTY = 0x000100
O_DIRECTORY = 0x010000
O_BINARY = 0x000000
O_LARGEFILE = 0x000000
O_BINARY = FLAG_UNSUPPORTED
O_LARGEFILE = FLAG_UNSUPPORTED
class linux_arm_open_flags(QlPrettyFlag):
@@ -509,7 +514,7 @@ class linux_arm_open_flags(QlPrettyFlag):
O_EXCL = 0x000080
O_NOCTTY = 0x000100
O_DIRECTORY = 0x004000
O_BINARY = 0x000000
O_BINARY = FLAG_UNSUPPORTED
O_LARGEFILE = 0x020000
@@ -527,7 +532,7 @@ class linux_mips_open_flags(QlPrettyFlag):
O_EXCL = 0x000400
O_NOCTTY = 0x000800
O_DIRECTORY = 0x010000
O_BINARY = 0x000000
O_BINARY = FLAG_UNSUPPORTED
O_LARGEFILE = 0x002000
@@ -545,8 +550,8 @@ class linux_riscv_open_flags(QlPrettyFlag):
O_EXCL = 0x000080
O_NOCTTY = 0x000100
O_DIRECTORY = 0x010000
O_BINARY = 0x000000
O_LARGEFILE = 0x000000
O_BINARY = FLAG_UNSUPPORTED
O_LARGEFILE = FLAG_UNSUPPORTED
class linux_ppc_open_flags(QlPrettyFlag):
@@ -563,7 +568,7 @@ class linux_ppc_open_flags(QlPrettyFlag):
O_EXCL = 0x000080
O_NOCTTY = 0x000100
O_DIRECTORY = 0x004000
O_BINARY = 0x000000
O_BINARY = FLAG_UNSUPPORTED
O_LARGEFILE = 0x010000
@@ -581,26 +586,26 @@ class freebsd_x86_open_flags(QlPrettyFlag):
O_EXCL = 0x000800
O_NOCTTY = 0x008000
O_DIRECTORY = 0x20000
O_BINARY = 0x000000
O_LARGEFILE = 0x000000
O_BINARY = FLAG_UNSUPPORTED
O_LARGEFILE = FLAG_UNSUPPORTED
class windows_x86_open_flags(QlPrettyFlag):
O_RDONLY = 0x000000
O_WRONLY = 0x000001
O_RDWR = 0x000002
O_NONBLOCK = 0x000000
O_NONBLOCK = FLAG_UNSUPPORTED
O_APPEND = 0x000008
O_ASYNC = 0x000000
O_SYNC = 0x000000
O_NOFOLLOW = 0x000000
O_ASYNC = FLAG_UNSUPPORTED
O_SYNC = FLAG_UNSUPPORTED
O_NOFOLLOW = FLAG_UNSUPPORTED
O_CREAT = 0x000100
O_TRUNC = 0x000200
O_EXCL = 0x000400
O_NOCTTY = 0x000000
O_DIRECTORY = 0x000000
O_NOCTTY = FLAG_UNSUPPORTED
O_DIRECTORY = FLAG_UNSUPPORTED
O_BINARY = 0x008000
O_LARGEFILE = 0x000000
O_LARGEFILE = FLAG_UNSUPPORTED
class qnx_arm_open_flags(QlPrettyFlag):
@@ -611,13 +616,13 @@ class qnx_arm_open_flags(QlPrettyFlag):
O_APPEND = 0x00008
O_ASYNC = 0x10000
O_SYNC = 0x00020
O_NOFOLLOW = 0x000000
O_NOFOLLOW = FLAG_UNSUPPORTED
O_CREAT = 0x00100
O_TRUNC = 0x00200
O_EXCL = 0x00400
O_NOCTTY = 0x00800
O_DIRECTORY = 0x000000
O_BINARY = 0x000000
O_DIRECTORY = FLAG_UNSUPPORTED
O_BINARY = FLAG_UNSUPPORTED
O_LARGEFILE = 0x08000
qiling/os/posix/const_mapping.py
+3 -3
View File
@@ -114,12 +114,12 @@ def ql_open_flag_mapping(ql: Qiling, flags: int) -> int:
# convert emulated os flags to hosting os flags.
# flags names are consistent across all classes, even if they are not supported, to maintain compatibility
for ef in emul_flags:
# test whether flag i set, excluding unsupported flags and 0 values
if ef and flags & ef.value:
# test whether flag is set, excluding unsupported flags
if (ef.value != FLAG_UNSUPPORTED) and (flags & ef.value):
hf = host_flags[ef.name or '']
# if flag is also supported on the host, set it
if hf:
if hf.value != FLAG_UNSUPPORTED:
ret |= hf.value
# NOTE: not sure why this one is needed
qiling/os/posix/posix.py
+10 -2
View File
@@ -10,7 +10,7 @@ from qiling import Qiling
from qiling.const import QL_ARCH, QL_INTERCEPT
from qiling.exception import QlErrorSyscallNotFound
from qiling.os.os import QlOs
from qiling.os.posix.const import NR_OPEN, errors
from qiling.os.posix.const import NR_OPEN, NSIG, errors
from qiling.os.posix.msq import QlMsq
from qiling.os.posix.shm import QlShm
from qiling.os.posix.syscall.abi import QlSyscallABI, arm, intel, mips, ppc, riscv
@@ -49,7 +49,6 @@ class QlOsPosix(QlOs):
def __init__(self, ql: Qiling):
super().__init__(ql)
self.sigaction_act = [0] * 256
conf = self.profile['KERNEL']
self.uid = self.euid = conf.getint('uid')
@@ -92,6 +91,11 @@ class QlOsPosix(QlOs):
self._shm = QlShm()
self._msq = QlMsq()
self._sig = [None] * NSIG
# a bitmap representing the blocked signals. a set bit at index i means signal i is blocked.
# note that SIGKILL and SIGSTOP cannot be blocked.
self.blocked_signals = 0
def __get_syscall_mapper(self, archtype: QL_ARCH):
qlos_path = f'.os.{self.type.name.lower()}.map_syscall'
@@ -264,3 +268,7 @@ class QlOsPosix(QlOs):
@property
def msq(self):
return self._msq
@property
def sig(self):
return self._sig
qiling/os/posix/syscall/signal.py
+171 -9
View File
@@ -3,27 +3,189 @@
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
from qiling import Qiling
from __future__ import annotations
import ctypes
from typing import TYPE_CHECKING, Type
from qiling.const import QL_ARCH
from qiling.os import struct
from qiling.os.posix.const import NSIG
# TODO: MIPS differs in too many details around signals; MIPS implementation is better extracted out
if TYPE_CHECKING:
from qiling import Qiling
from qiling.arch.arch import QlArch
@struct.cache
def __make_sigset(arch: QlArch):
native_type = struct.get_native_type(arch.bits)
sigset_type = {
QL_ARCH.X86: native_type,
QL_ARCH.X8664: native_type,
QL_ARCH.ARM: native_type,
QL_ARCH.ARM64: native_type,
QL_ARCH.MIPS: ctypes.c_uint32 * (128 // (4 * 8)),
QL_ARCH.CORTEX_M: native_type
}
if arch.type not in sigset_type:
raise NotImplementedError(f'sigset definition is missing for {arch.type.name}')
return sigset_type[arch.type]
@struct.cache
def __make_sigaction(arch: QlArch) -> Type[struct.BaseStruct]:
native_type = struct.get_native_type(arch.bits)
Struct = struct.get_aligned_struct(arch.bits, arch.endian)
sigset_type = __make_sigset(arch)
# # FIXME: untill python 3.11 ctypes Union does not support an endianess that is different from
# the hosting paltform. if a LE system is emulating a BE one or vice versa, this will fail. to
# work around that we avoid using a union and refer to the inner field as 'sa_handler' regardless.
#
# Union = struct.get_aligned_union(arch.bits)
#
# class sighandler_union(Union):
# _fields_ = (
# ('sa_handler', native_type),
# ('sa_sigaction', native_type)
# )
# <WORKAROUND> see FIXME above
class sighandler_union(Struct):
_fields_ = (
('sa_handler', native_type),
)
# </WORKAROUND>
# see: https://elixir.bootlin.com/linux/v5.19.17/source/arch/arm/include/uapi/asm/signal.h
class arm_sigaction(Struct):
_anonymous_ = ('_u',)
_fields_ = (
('_u', sighandler_union),
('sa_mask', sigset_type),
('sa_flags', native_type),
('sa_restorer', native_type)
)
# see: https://elixir.bootlin.com/linux/v5.19.17/source/arch/x86/include/uapi/asm/signal.h
class x86_sigaction(Struct):
_anonymous_ = ('_u',)
_fields_ = (
('_u', sighandler_union),
('sa_mask', sigset_type),
('sa_flags', native_type),
('sa_restorer', native_type)
)
class x8664_sigaction(Struct):
_fields_ = (
('sa_handler', native_type),
('sa_flags', native_type),
('sa_restorer', native_type),
('sa_mask', sigset_type)
)
# see: https://elixir.bootlin.com/linux/v5.19.17/source/arch/mips/include/uapi/asm/signal.h
class mips_sigaction(Struct):
_fields_ = (
('sa_flags', ctypes.c_uint32),
('sa_handler', native_type),
('sa_mask', sigset_type)
)
sigaction_struct = {
QL_ARCH.X86: x86_sigaction,
QL_ARCH.X8664: x8664_sigaction,
QL_ARCH.ARM: arm_sigaction,
QL_ARCH.ARM64: arm_sigaction,
QL_ARCH.MIPS: mips_sigaction,
QL_ARCH.CORTEX_M: arm_sigaction
}
if arch.type not in sigaction_struct:
raise NotImplementedError(f'sigaction definition is missing for {arch.type.name}')
return sigaction_struct[arch.type]
def ql_syscall_rt_sigaction(ql: Qiling, signum: int, act: int, oldact: int):
if oldact:
arr = ql.os.sigaction_act[signum] or [0] * 5
data = b''.join(ql.pack32(key) for key in arr)
SIGKILL = 9
SIGSTOP = 23 if ql.arch.type is QL_ARCH.MIPS else 19
ql.mem.write(oldact, data)
if signum not in range(NSIG) or signum in (SIGKILL, SIGSTOP):
return -1 # EINVAL
sigaction = __make_sigaction(ql.arch)
if oldact:
old = ql.os.sig[signum] or sigaction()
old.save_to(ql.mem, oldact)
if act:
ql.os.sigaction_act[signum] = [ql.mem.read_ptr(act + 4 * i, 4) for i in range(5)]
ql.os.sig[signum] = sigaction.load_from(ql.mem, act)
return 0
def ql_syscall_rt_sigprocmask(ql: Qiling, how: int, nset: int, oset: int, sigsetsize: int):
# SIG_BLOCK = 0x0
# SIG_UNBLOCK = 0x1
def __sigprocmask(ql: Qiling, how: int, newset: int, oldset: int):
SIG_BLOCK = 0
SIG_UNBLOCK = 1
SIG_SETMASK = 2
SIGKILL = 9
SIGSTOP = 19
if oldset:
ql.mem.write_ptr(newset, ql.os.blocked_signals)
if newset:
set_mask = ql.mem.read_ptr(newset)
if how == SIG_BLOCK:
ql.os.blocked_signals |= set_mask
elif how == SIG_UNBLOCK:
ql.os.blocked_signals &= ~set_mask
elif how == SIG_SETMASK:
ql.os.blocked_signals = set_mask
else:
return -1 # EINVAL
# silently drop attempts to block SIGKILL and SIGSTOP
ql.os.blocked_signals &= ~((1 << SIGKILL) | (1 << SIGSTOP))
return 0
def __sigprocmask_mips(ql: Qiling, how: int, newset: int, oldset: int):
SIG_BLOCK = 1
SIG_UNBLOCK = 2
SIG_SETMASK = 3
SIGKILL = 9
SIGSTOP = 23
# TODO: to implement
return 0
def ql_syscall_rt_sigprocmask(ql: Qiling, how: int, newset: int, oldset: int):
impl = __sigprocmask_mips if ql.arch.type is QL_ARCH.MIPS else __sigprocmask
return impl(ql, how, newset, oldset)
def ql_syscall_signal(ql: Qiling, sig: int, sighandler: int):
return 0
qiling/os/posix/syscall/unistd.py
+16
View File
@@ -152,6 +152,22 @@ def ql_syscall_capset(ql: Qiling, hdrp: int, datap: int):
def ql_syscall_kill(ql: Qiling, pid: int, sig: int):
if sig not in range(NSIG):
return -1 # EINVAL
if pid > 0 and pid != ql.os.pid:
return -1 # ESRCH
sigaction = ql.os.sig[sig]
# sa_handler is:
# SIG_DFL for the default action.
# SIG_IGN to ignore this signal.
# handler pointer
# if sa_flags & SA_SIGINFO:
# call sa_sigaction instead of sa_handler
return 0
qiling/os/windows/const.py
+2
View File
@@ -38,6 +38,7 @@ STATUS_INVALID_HANDLE = 0xC0000008
STATUS_PROCEDURE_NOT_FOUND = 0xC000007A
STATUS_DLL_NOT_FOUND = 0xC0000135
STATUS_PORT_NOT_SET = 0xC0000353
STATUS_STACK_BUFFER_OVERRUN = 0xC0000409
STATUS_NO_YIELD_PERFORMED = 0x40000024
# ...
@@ -638,6 +639,7 @@ ProcessProtectionInformation = 61
ProcessDebugObjectHandle = 30
ProcessDebugFlags = 31
ProcessExecuteFlags = 34
ProcessCookie = 36
ProcessImageInformation = 37
ProcessMitigationPolicy = 52
ProcessFaultInformation = 63
qiling/os/windows/dlls/kernel32/errhandlingapi.py
+44 -36
View File
@@ -44,15 +44,6 @@ def hook_GetLastError(ql: Qiling, address: int, params):
def hook_SetLastError(ql: Qiling, address: int, params):
ql.os.last_error = params['dwErrCode']
# LONG UnhandledExceptionFilter(
# _EXCEPTION_POINTERS *ExceptionInfo
# );
@winsdkapi(cc=STDCALL, params={
'ExceptionInfo' : POINTER
})
def hook_UnhandledExceptionFilter(ql: Qiling, address: int, params):
return 1
# UINT SetErrorMode(
# UINT uMode
# );
@@ -63,33 +54,6 @@ def hook_SetErrorMode(ql: Qiling, address: int, params):
# TODO maybe this need a better implementation
return 0
# __analysis_noreturn VOID RaiseException(
# DWORD dwExceptionCode,
# DWORD dwExceptionFlags,
# DWORD nNumberOfArguments,
# const ULONG_PTR *lpArguments
# );
@winsdkapi(cc=STDCALL, params={
'dwExceptionCode' : DWORD,
'dwExceptionFlags' : DWORD,
'nNumberOfArguments' : DWORD,
'lpArguments' : POINTER
})
def hook_RaiseException(ql: Qiling, address: int, params):
nNumberOfArguments = params['nNumberOfArguments']
lpArguments = params['lpArguments']
handle = ql.os.handle_manager.search("TopLevelExceptionHandler")
if handle is None:
ql.log.warning(f'RaiseException: top level exception handler not found')
return
exception_handler = handle.obj
args = [(PARAM_INTN, ql.mem.read_ptr(lpArguments + i * ql.arch.pointersize)) for i in range(nNumberOfArguments)] if lpArguments else []
ql.os.fcall.call_native(exception_handler, args, None)
# PVOID AddVectoredExceptionHandler(
# ULONG First,
# PVECTORED_EXCEPTION_HANDLER Handler
@@ -151,3 +115,47 @@ def hook_RemoveVectoredExceptionHandler(ql: Qiling, address: int, params):
hook.remove()
return 0
# VOID RaiseException(
# DWORD dwExceptionCode,
# DWORD dwExceptionFlags,
# DWORD nNumberOfArguments,
# CONST ULONG_PTR* lpArguments
# );
@winsdkapi(cc=STDCALL, params={
'dwExceptionCode': DWORD,
'dwExceptionFlags': DWORD,
'nNumberOfArguments': DWORD,
'lpArguments': PVOID
}, passthru=True)
def hook_RaiseException(ql: Qiling, address: int, params):
# On x86_64, RaiseException will call RtlRaiseException,
# which calls the exception dispatcher directly. The native
# exception dispatching code mostly works correctly
# for software exceptions, so we shall simply continue
# through to the native dispatcher in this case.
if ql.arch.type is not QL_ARCH.X86:
return
# On x86, the situation is different. RtlRaiseException
# will call ZwRaiseException, which uses a syscall.
# However, Qiling doesn't really support Windows syscalls
# right now.
# We will treat all exceptions as unhandled exceptions,
# which is better than nothing.
# TODO: Get kernel exception dispatching working properly,
# then first-chance software exceptions, SEH, and C++
# exceptions can work on 32-bit Windows too.
nNumberOfArguments = params['nNumberOfArguments']
lpArguments = params['lpArguments']
handle = ql.os.handle_manager.search("TopLevelExceptionHandler")
if handle is None:
ql.log.warning(f'RaiseException: top level exception handler not found')
return
exception_handler = handle.obj
args = [(PARAM_INTN, ql.mem.read_ptr(lpArguments + i * ql.arch.pointersize)) for i in range(nNumberOfArguments)] if lpArguments else []
ql.os.fcall.call_native(exception_handler, args, None)
qiling/os/windows/dlls/kernel32/heapapi.py
+73 -6
View File
@@ -49,6 +49,17 @@ def hook_HeapCreate(ql: Qiling, address: int, params):
return ql.os.heap.alloc(dwInitialSize)
def _HeapAlloc(ql: Qiling, address: int, params):
dwFlags = params["dwFlags"]
dwBytes = params["dwBytes"]
ptr = ql.os.heap.alloc(dwBytes)
if ptr and (dwFlags & HEAP_ZERO_MEMORY):
__zero_mem(ql.mem, ptr, dwBytes)
return ptr
# DECLSPEC_ALLOCATOR LPVOID HeapAlloc(
# HANDLE hHeap,
# DWORD dwFlags,
@@ -60,15 +71,45 @@ def hook_HeapCreate(ql: Qiling, address: int, params):
'dwBytes' : SIZE_T
})
def hook_HeapAlloc(ql: Qiling, address: int, params):
dwFlags = params["dwFlags"]
dwBytes = params["dwBytes"]
return _HeapAlloc(ql, address, params)
ptr = ql.os.heap.alloc(dwBytes)
# DECLSPEC_ALLOCATOR LPVOID HeapReAlloc(
# HANDLE hHeap,
# DWORD dwFlags,
# _Frees_ptr_opt_ LPVOID lpMem,
# SIZE_T dwBytes
# );
@winsdkapi(cc=STDCALL, params={
'hHeap' : HANDLE,
'dwFlags' : DWORD,
'lpMem': LPVOID,
'dwBytes' : SIZE_T
})
def hook_HeapReAlloc(ql: Qiling, address: int, params):
base = params["lpMem"]
newSize = params["dwBytes"]
if ptr and (dwFlags & HEAP_ZERO_MEMORY):
__zero_mem(ql.mem, ptr, dwBytes)
if not base:
return _HeapAlloc(ql, address, params)
if newSize == 0:
ql.os.heap.free(base)
return 0
return ptr
oldSize = ql.os.heap.size(base)
oldData = bytes(ql.mem.read(base, oldSize))
ql.os.heap.free(base)
if newSize < oldSize:
oldData = oldData[0:newSize]
newBase = ql.os.heap.alloc(newSize)
if newBase:
ql.mem.write(newBase, oldData)
return newBase
# SIZE_T HeapSize(
# HANDLE hHeap,
@@ -120,3 +161,29 @@ def hook_HeapSetInformation(ql: Qiling, address: int, params):
@winsdkapi(cc=STDCALL, params={})
def hook_GetProcessHeap(ql: Qiling, address: int, params):
return ql.os.heap.start_address
# BOOL HeapValidate(
# HANDLE hHeap,
# DWORD dwFlags,
# LPCVOID lpMem
# );
@winsdkapi(cc=STDCALL, params={
'hHeap': PVOID,
'dwFlags': DWORD,
'lpMem': PVOID
})
def hook_HeapValidate(ql: Qiling, address: int, params):
hHeap = params['hHeap']
lpMem = params['lpMem']
if not hHeap:
return 0
# TODO: Maybe _find is a heap manager implementation
# detail, in which case we shouldn't rely on it.
chunk = ql.os.heap._find(lpMem)
if not chunk:
return 0
return chunk.inuse
qiling/os/windows/dlls/kernel32/winbase.py
-18
View File
@@ -159,24 +159,6 @@ def hook__lwrite(ql: Qiling, address: int, params):
def hook_FatalExit(ql: Qiling, address: int, params):
ql.emu_stop()
# PVOID EncodePointer(
# _In_ PVOID Ptr
# );
@winsdkapi(cc=STDCALL, params={
'Ptr' : PVOID
})
def hook_EncodePointer(ql: Qiling, address: int, params):
return params['Ptr']
# PVOID DecodePointer(
# _In_ PVOID Ptr
# );
@winsdkapi(cc=STDCALL, params={
'Ptr' : PVOID
})
def hook_DecodePointer(ql: Qiling, address: int, params):
return params['Ptr']
# UINT WinExec(
# LPCSTR lpCmdLine,
# UINT uCmdShow
qiling/os/windows/dlls/kernel32/winnls.py
+13 -12
View File
@@ -80,18 +80,19 @@ def hook_IsValidCodePage(ql: Qiling, address: int, params):
return 1
def __LCMapString(ql: Qiling, address: int, params, wstring: bool):
lpSrcStr: str = params["lpSrcStr"]
lpSrcStr: int = params["lpSrcStr"]
cchSrc: int = params["cchSrc"]
lpDestStr: int = params["lpDestStr"]
cchDest: int = params["cchDest"]
enc = "utf-16le" if wstring else "utf-8"
res = f'{lpSrcStr}\x00'
char_size = 2 if wstring else 1
byte_count = cchSrc * char_size
if cchDest and lpDestStr:
# TODO maybe do some other check, for now is working
ql.mem.write(lpDestStr, res.encode(enc))
source_bytes = ql.mem.read(lpSrcStr, byte_count)
ql.mem.write(lpDestStr, bytes(source_bytes))
return len(res)
return cchSrc
# int LCMapStringW(
# LCID Locale,
@@ -104,9 +105,9 @@ def __LCMapString(ql: Qiling, address: int, params, wstring: bool):
@winsdkapi(cc=STDCALL, params={
'Locale' : LCID,
'dwMapFlags' : DWORD,
'lpSrcStr' : LPCWSTR,
'lpSrcStr' : POINTER,
'cchSrc' : INT,
'lpDestStr' : LPWSTR,
'lpDestStr' : POINTER,
'cchDest' : INT
})
def hook_LCMapStringW(ql: Qiling, address: int, params):
@@ -123,9 +124,9 @@ def hook_LCMapStringW(ql: Qiling, address: int, params):
@winsdkapi(cc=STDCALL, params={
'Locale' : LCID,
'dwMapFlags' : DWORD,
'lpSrcStr' : LPCSTR,
'lpSrcStr' : POINTER,
'cchSrc' : INT,
'lpDestStr' : LPSTR,
'lpDestStr' : POINTER,
'cchDest' : INT
})
def hook_LCMapStringA(ql: Qiling, address: int, params):
@@ -145,9 +146,9 @@ def hook_LCMapStringA(ql: Qiling, address: int, params):
@winsdkapi(cc=STDCALL, params={
'lpLocaleName' : LPCWSTR,
'dwMapFlags' : DWORD,
'lpSrcStr' : LPCWSTR,
'lpSrcStr' : POINTER,
'cchSrc' : INT,
'lpDestStr' : LPWSTR,
'lpDestStr' : POINTER,
'cchDest' : INT,
'lpVersionInformation' : LPNLSVERSIONINFO,
'lpReserved' : LPVOID,
qiling/os/windows/dlls/msvcrt.py
+118 -92
View File
@@ -10,7 +10,7 @@ from qiling import Qiling
from qiling.exception import QlErrorNotImplemented
from qiling.os.const import *
from qiling.os.windows.fncc import *
from qiling.os.windows.const import LOCALE
from qiling.os.windows.const import *
from qiling.os.windows.handle import Handle
# void __set_app_type (
@@ -135,10 +135,9 @@ def hook__controlfp(ql: Qiling, address: int, params):
# );
@winsdkapi(cc=CDECL, params={
'func' : POINTER
})
}, passthru=True)
def hook_atexit(ql: Qiling, address: int, params):
ret = 0
return ret
return
# char*** __p__environ(void)
@winsdkapi(cc=CDECL, params={})
@@ -174,17 +173,6 @@ def hook_puts(ql: Qiling, address: int, params):
def hook__cexit(ql: Qiling, address: int, params):
pass
# void __cdecl _initterm(
# PVFV *,
# PVFV *
# );
@winsdkapi(cc=CDECL, params={
'pfbegin' : POINTER,
'pfend' : POINTER
})
def hook__initterm(ql: Qiling, address: int, params):
return 0
# void exit(
# int const status
# );
@@ -194,17 +182,6 @@ def hook__initterm(ql: Qiling, address: int, params):
def hook_exit(ql: Qiling, address: int, params):
ql.emu_stop()
# int __cdecl _initterm_e(
# PVFV *,
# PVFV *
# );
@winsdkapi(cc=CDECL, params={
'pfbegin' : POINTER,
'pfend' : POINTER
})
def hook__initterm_e(ql: Qiling, address: int, params):
return 0
# char*** __cdecl __p___argv (void);
@winsdkapi(cc=CDECL, params={})
def hook___p___argv(ql: Qiling, address: int, params):
@@ -233,20 +210,12 @@ def hook___p___argc(ql: Qiling, address: int, params):
return ret
# TODO: this one belongs to ucrtbase.dll
@winsdkapi(cc=CDECL, params={})
@winsdkapi(cc=CDECL, params={}, passthru=True)
def hook__get_initial_narrow_environment(ql: Qiling, address: int, params):
ret = 0
for i, (k, v) in enumerate(ql.env.items()):
entry = bytes(f'{k}={v}', 'ascii') + b'\x00'
p_entry = ql.os.heap.alloc(len(entry))
ql.mem.write(p_entry, entry)
if i == 0:
ret = p_entry
return ret
# If the native version of this function does not
# get to run, then debug versions of the CRT DLLs can fail
# their initialization.
return
# int sprintf ( char * str, const char * format, ... );
@winsdkapi(cc=CDECL, params={
@@ -303,13 +272,6 @@ def hook_wprintf(ql: Qiling, address: int, params):
return count
# MSVCRT_FILE * CDECL MSVCRT___acrt_iob_func(unsigned idx)
@winsdkapi(cc=CDECL, params={
'idx': UINT
})
def hook___acrt_iob_func(ql: Qiling, address: int, params):
return 0
def __stdio_common_vfprintf(ql: Qiling, address: int, params, wstring: bool):
format = params['_Format']
arglist = params['_ArgList']
@@ -368,6 +330,18 @@ def __stdio_common_vsprintf(ql: Qiling, address: int, params, wstring: bool):
def hook___stdio_common_vsprintf(ql: Qiling, address: int, params):
return __stdio_common_vsprintf(ql, address, params, False)
@winsdkapi(cc=CDECL, params={
'_Options' : PARAM_INT64,
'_Buffer' : POINTER,
'_BufferCount' : SIZE_T,
'_MaxCount' : SIZE_T,
'_Format' : STRING,
'_Locale' : DWORD,
'_ArgList' : POINTER
})
def hook___stdio_common_vsnprintf(ql: Qiling, address: int, params):
return __stdio_common_vsprintf(ql, address, params, False)
@winsdkapi(cc=CDECL, params={
'_Options' : PARAM_INT64,
'_Buffer' : POINTER,
@@ -379,6 +353,18 @@ def hook___stdio_common_vsprintf(ql: Qiling, address: int, params):
def hook___stdio_common_vswprintf(ql: Qiling, address: int, params):
return __stdio_common_vsprintf(ql, address, params, True)
@winsdkapi(cc=CDECL, params={
'_Options' : PARAM_INT64,
'_Buffer' : POINTER,
'_BufferCount' : SIZE_T,
'_MaxCount' : SIZE_T,
'_Format' : WSTRING,
'_Locale' : DWORD,
'_ArgList' : POINTER
})
def hook___stdio_common_vsnwprintf(ql: Qiling, address: int, params):
return __stdio_common_vsprintf(ql, address, params, True)
# all the "_s" versions are aliases to their non-"_s" counterparts
@winsdkapi(cc=CDECL, params={
@@ -412,6 +398,18 @@ def hook___stdio_common_vfwprintf_s(ql: Qiling, address: int, params):
def hook___stdio_common_vsprintf_s(ql: Qiling, address: int, params):
return hook___stdio_common_vsprintf.__wrapped__(ql, address, params)
@winsdkapi(cc=CDECL, params={
'_Options' : PARAM_INT64,
'_Buffer' : POINTER,
'_BufferCount' : SIZE_T,
'_MaxCount' : SIZE_T,
'_Format' : STRING,
'_Locale' : DWORD,
'_ArgList' : POINTER
})
def hook___stdio_common_vsnprintf_s(ql: Qiling, address: int, params):
return hook___stdio_common_vsnprintf.__wrapped__(ql, address, params)
@winsdkapi(cc=CDECL, params={
'_Options' : PARAM_INT64,
'_Buffer' : POINTER,
@@ -423,6 +421,18 @@ def hook___stdio_common_vsprintf_s(ql: Qiling, address: int, params):
def hook___stdio_common_vswprintf_s(ql: Qiling, address: int, params):
return hook___stdio_common_vswprintf.__wrapped__(ql, address, params)
@winsdkapi(cc=CDECL, params={
'_Options' : PARAM_INT64,
'_Buffer' : POINTER,
'_BufferCount' : SIZE_T,
'_MaxCount' : SIZE_T,
'_Format' : WSTRING,
'_Locale' : DWORD,
'_ArgList' : POINTER
})
def hook___stdio_common_vsnwprintf_s(ql: Qiling, address: int, params):
return hook___stdio_common_vsnwprintf.__wrapped__(ql, address, params)
@winsdkapi(cc=CDECL, params={})
def hook___lconv_init(ql: Qiling, address: int, params):
return 0
@@ -478,43 +488,42 @@ def hook_strncmp(ql: Qiling, address: int, params):
return result
def __malloc(ql: Qiling, address: int, params):
size = params['size']
return ql.os.heap.alloc(size)
@winsdkapi(cc=CDECL, params={
'size' : UINT
})
}, passthru=True)
def hook__malloc_base(ql: Qiling, address: int, params):
return __malloc(ql, address, params)
return
# void* malloc（unsigned int size)
@winsdkapi(cc=CDECL, params={
'size' : UINT
})
}, passthru=True)
def hook_malloc(ql: Qiling, address: int, params):
size = params['size']
return
return ql.os.heap.alloc(size)
def __free(ql: Qiling, address: int, params):
address = params['address']
ql.os.heap.free(address)
# void* __cdecl _realloc_base(
# void* const block,
# size_t const size
# )
@winsdkapi(cc=CDECL, params={
'block' : POINTER,
'size' : UINT
}, passthru=True)
def hook__realloc_base(ql: Qiling, address: int, params):
return
@winsdkapi(cc=CDECL, params={
'address': POINTER
})
}, passthru=True)
def hook__free_base(ql: Qiling, address: int, params):
return __free(ql, address, params)
return
# void* free（void *address)
@winsdkapi(cc=CDECL, params={
'address': POINTER
})
}, passthru=True)
def hook_free(ql: Qiling, address: int, params):
return __free(ql, address, params)
return
# _onexit_t _onexit(
# _onexit_t function
@@ -530,6 +539,27 @@ def hook__onexit(ql: Qiling, address: int, params):
return addr
# _onexit_t __dllonexit(
# _onexit_t func,
# _PVFV ** pbegin,
# _PVFV ** pend
# );
@winsdkapi(cc=STDCALL, params={
'function': POINTER,
'pbegin': POINTER,
'pend': POINTER
})
def hook___dllonexit(ql: Qiling, address: int, params):
function = params['function']
if function:
addr = ql.os.heap.alloc(ql.arch.pointersize)
ql.mem.write_ptr(addr, function)
return addr
return 0
# void *memset(
# void *dest,
# int c,
@@ -539,32 +569,16 @@ def hook__onexit(ql: Qiling, address: int, params):
'dest' : POINTER,
'c' : INT,
'count' : SIZE_T
})
}, passthru=True)
def hook_memset(ql: Qiling, address: int, params):
dest = params["dest"]
c = params["c"]
count = params["count"]
ql.mem.write(dest, bytes([c] * count))
return dest
def __calloc(ql: Qiling, address: int, params):
num = params['num']
size = params['size']
count = num * size
ret = ql.os.heap.alloc(count)
ql.mem.write(ret, bytes([0] * count))
return ret
return
@winsdkapi(cc=CDECL, params={
'num' : SIZE_T,
'size' : SIZE_T
})
}, passthru=True)
def hook__calloc_base(ql: Qiling, address: int, params):
return __calloc(ql, address, params)
return
# void *calloc(
# size_t num,
@@ -573,9 +587,9 @@ def hook__calloc_base(ql: Qiling, address: int, params):
@winsdkapi(cc=CDECL, params={
'num' : SIZE_T,
'size' : SIZE_T
})
}, passthru=True)
def hook_calloc(ql: Qiling, address: int, params):
return __calloc(ql, address, params)
return
# void * memmove(
# void *dest,
@@ -586,12 +600,9 @@ def hook_calloc(ql: Qiling, address: int, params):
'dest' : POINTER,
'src' : POINTER,
'num' : SIZE_T
})
}, passthru=True)
def hook_memmove(ql: Qiling, address: int, params):
data = ql.mem.read(params['src'], params['num'])
ql.mem.write(params['dest'], bytes(data))
return params['dest']
return
# int _ismbblead(
# unsigned int c
@@ -644,3 +655,18 @@ def hook__time64(ql: Qiling, address: int, params):
ql.mem.write_ptr(dst, time_wasted, 8)
return time_wasted
# void abort( void );
@winsdkapi(cc=CDECL, params={})
def hook_abort(ql: Qiling, address: int, params):
# During testing, it was found that programs terminating abnormally
# via abort() terminated with exit code=STATUS_STACK_BUFFER_OVERRUN.
# According to Microsoft's devblog, this does not necessarily mean
# that a stack buffer overrun occurred.
# Rather, it can indicate abnormal program termination in a variety of
# situations, including abort().
# https://devblogs.microsoft.com/oldnewthing/20190108-00/?p=100655
#
ql.os.exit_code = STATUS_STACK_BUFFER_OVERRUN
ql.emu_stop()
qiling/os/windows/dlls/ntdll.py
+446 -1
View File
@@ -17,6 +17,7 @@ from qiling.os.windows.handle import *
from qiling.os.windows import structs
from qiling.os.windows import utils
from unicorn.x86_const import *
# void *memcpy(
# void *dest,
@@ -39,6 +40,7 @@ def hook_memcpy(ql: Qiling, address: int, params):
return dest
def _QueryInformationProcess(ql: Qiling, address: int, params):
handle = params["ProcessHandle"]
flag = params["ProcessInformationClass"]
obuf_ptr = params["ProcessInformation"]
obuf_len = params['ProcessInformationLength']
@@ -68,6 +70,24 @@ def _QueryInformationProcess(ql: Qiling, address: int, params):
res_data = bytes(pci_obj)
elif flag == ProcessCookie:
hCurrentProcess = (1 << ql.arch.bits) - 1
if handle != hCurrentProcess:
# If a process attempts to query the cookie of another
# process, then QueryInformationProcess returns an error.
return STATUS_INVALID_PARAMETER
# TODO: Change this to something else,
# maybe a static randomly generated value.
res_data = ql.pack32(0x00000001)
if obuf_len != len(res_data):
# If the buffer length is not ULONG size
# then QueryInformationProcess returns an error.
return STATUS_INFO_LENGTH_MISMATCH
else:
# TODO: support more info class ("flag") values
ql.log.info(f'QueryInformationProcess: no implementation for info class {flag:#04x}')
@@ -452,4 +472,429 @@ def hook_wcsstr(ql: Qiling, address: int, params):
@winsdkapi(cc=STDCALL, params={})
def hook_CsrGetProcessId(ql: Qiling, address: int, params):
pid = ql.os.profile["PROCESSES"].getint("csrss.exe", fallback=12345)
return pid
return pid
# NTSYSAPI PVOID RtlPcToFileHeader(
# [in] PVOID PcValue,
# [out] PVOID *BaseOfImage
# );
@winsdkapi(cc=STDCALL, params={
'PcValue' : PVOID,
'BaseOfImage': PVOID
})
def hook_RtlPcToFileHeader(ql: Qiling, address: int, params):
pc = params["PcValue"]
base_of_image_ptr = params["BaseOfImage"]
containing_image = ql.loader.find_containing_image(pc)
base_addr = containing_image.base if containing_image else 0
ql.mem.write_ptr(base_of_image_ptr, base_addr)
return base_addr
def _FindImageBaseAndFunctionTable(ql: Qiling, control_pc: int, image_base_ptr: int):
"""
Helper function to locate a containing image for `control_pc` as well as its
function table, while writing the image base to `image_base_ptr` (if non-zero).
Returns:
(base_addr, function_table_addr)
if no image is found, otherwise
(0, 0)
"""
containing_image = ql.loader.find_containing_image(control_pc)
if containing_image:
base_addr = containing_image.base
else:
base_addr = 0
# Write base address to the ImageBase pointer, if provided
if image_base_ptr != 0:
ql.mem.write_ptr(image_base_ptr, base_addr)
# If we don’t have a valid base, abort now
if base_addr == 0:
return 0, 0
# Look up the function-table RVA and compute the absolute address
function_table_rva = ql.loader.function_table_lookup.get(base_addr)
function_table_addr = base_addr + function_table_rva if function_table_rva else 0
return base_addr, function_table_addr
# NTSYSAPI PRUNTIME_FUNCTION RtlLookupFunctionEntry(
# [in] DWORD64 ControlPc,
# [out] PDWORD64 ImageBase,
# [out] PUNWIND_HISTORY_TABLE HistoryTable
# );
@winsdkapi(cc=STDCALL, params={
'ControlPc': PVOID,
'ImageBase': PVOID,
'HistoryTable': PVOID
})
def hook_RtlLookupFunctionEntry(ql: Qiling, address: int, params):
control_pc = params["ControlPc"]
image_base_ptr = params["ImageBase"]
# TODO: Make use of the history table to optimize this function.
# Alternatively, we could add caching to the loader, seeing as the
# loader is responsible for lookups in the function table.
# For simplicity, we are going to ignore the history table.
# history_table_ptr = params["HistoryTable"]
# This function should not be getting called on x86.
if ql.arch.type is QL_ARCH.X86:
raise QlErrorNotImplemented("RtlLookupFunctionEntry is not implemented for x86")
base_addr, function_table_addr = _FindImageBaseAndFunctionTable(ql, control_pc, image_base_ptr)
# If no function table was found, abort.
if function_table_addr == 0:
return 0
# Look up the RUNTIME_FUNCTION entry; we are interested in the index in the table
# so that we can compute the address.
runtime_function_idx, runtime_function = ql.loader.lookup_function_entry(base_addr, control_pc)
# If a suitable function entry was found,
# compute its address and return.
if runtime_function:
return function_table_addr + runtime_function_idx * 12 # sizeof(RUNTIME_FUNCTION)
return 0
# NTSYSAPI
# PRUNTIME_FUNCTION
# RtlLookupFunctionTable (
# IN PVOID ControlPc,
# OUT PVOID *ImageBase,
# OUT PULONG SizeOfTable
# );
@winsdkapi(cc=STDCALL, params={
'ControlPc': PVOID,
'ImageBase': PVOID,
'SizeOfTable': PVOID
})
def hook_RtlLookupFunctionTable(ql: Qiling, address: int, params):
control_pc = params["ControlPc"]
image_base_ptr = params["ImageBase"]
size_of_table_ptr = params["SizeOfTable"]
# This function should not be getting called on x86.
if ql.arch.type is QL_ARCH.X86:
raise QlErrorNotImplemented("RtlLookupFunctionTable is not implemented for x86")
base_addr, function_table_addr = _FindImageBaseAndFunctionTable(ql, control_pc, image_base_ptr)
# If no function table was found, abort.
if function_table_addr == 0:
ql.mem.write_ptr(size_of_table_ptr, 0, 4)
return 0
# If a valid pointer for the size was provided,
# we want to figure out the size of the table.
if size_of_table_ptr != 0:
# Look up the function table from the loader,
# and get the number of entries.
function_table = ql.loader.function_tables[base_addr]
# compute the total size of the table
size_of_table = len(function_table) * 12 # sizeof(RUNTIME_FUNCTION)
# Write the size to memory at the provided pointer.
ql.mem.write_ptr(size_of_table_ptr, size_of_table, 4)
return function_table_addr
@winsdkapi(cc=STDCALL, params={})
def hook_LdrControlFlowGuardEnforced(ql: Qiling, address: int, params):
# There are some checks in ntdll for whether CFG is enabled.
# We simply bypass these checks by returning 0.
# May not be necessary, but we do it just in case.
return 0
# NTSYSAPI
# NTSTATUS
# ZwRaiseException (
# IN PEXCEPTION_RECORD ExceptionRecord,
# IN PCONTEXT ContextRecord,
# IN BOOLEAN FirstChance
# );
@winsdkapi(cc=STDCALL, params={
'ExceptionRecord': PVOID,
'ContextRecord': PVOID,
'FirstChance': BOOLEAN
}, passthru=True)
def hook_ZwRaiseException(ql: Qiling, address: int, params):
exception_ptr = params['ExceptionRecord']
context_ptr = params['ContextRecord']
first_chance = params['FirstChance']
# The native ZwRaiseException simply uses a syscall to start
# the kernel exception dispatcher. However, Windows syscalls
# are not really working in Qiling right now.
# For now, we just provide a workaround for second-chance
# exceptions to work.
# TODO: Get some kind of solution for kernel exception
# dispatching. This is also needed for first-chance exceptions
# to work properly on 32-bit Windows.
if first_chance:
raise QlErrorNotImplemented("ZwRaiseException is not implemented for first-chance exceptions.")
# In Windows, an unhandled exception triggers the
# top-level unhandled exception filter, after which the process
# is terminated and error reporting services are called.
# Regardless of whether an unhandled exception filter is present,
# the process terminates with the same error code that was raised.
# Our strategy for this hook is to forward second-chance exceptions
# to the registered unhandled exception filter, if one exists.
if exception_ptr:
exception_code = ql.mem.read_ptr(exception_ptr, 4) # exception code is always DWORD
ql.log.debug(f"[ZwRaiseException] ExceptionCode: 0x{exception_code:08X}")
else:
ql.log.debug("[ZwRaiseException] ExceptionRecord is NULL")
ql.log.debug(f" ContextRecord: 0x{context_ptr:016X}")
ql.log.debug(f" FirstChance: {first_chance}")
handle = ql.os.handle_manager.search("TopLevelExceptionHandler")
if handle is None:
ql.log.debug(f'[ZwRaiseException] No top-level exception filter was found.')
ql.log.info(f'The process exited with code 0x{exception_code:08X}.')
ql.os.exit_code = exception_code
ql.emu_stop()
return
ret_addr = ql.stack_read(0)
exception_filter = handle.obj
# allocate some memory for the EXCEPTION_POINTERS struct
epointers_struct = structs.make_exception_pointers(ql.arch.bits)
exception_pointers_ptr = ql.os.heap.alloc(epointers_struct.sizeof())
with epointers_struct.ref(ql.mem, exception_pointers_ptr) as epointers_obj:
epointers_obj.ExceptionRecord = exception_ptr
epointers_obj.ContextRecord = context_ptr
exception_filter = handle.obj
ql.log.debug(f'[ZwRaiseException] Resuming execution at the top-level exception filter at 0x{exception_filter:08X}.')
# Hack: We are going to fake that the caller of ZwRaiseException
# actually called the unhandled exception filter instead.
# We will create a hook which will be triggered when the unhandled
# exception filter returns, so that we may terminate execution.
def __post_exception_filter(ql: Qiling):
# Free the exception pointers struct we allocated earlier.
# Might not be needed, since we are going to terminate the process
# soon, but we might as well free it.
ql.os.heap.free(exception_pointers_ptr)
ql.log.debug(f'[ZwRaiseException] Returned from unhandled exception filter at 0x{exception_filter:08X}.')
ql.log.info(f'The process exited with code 0x{exception_code:08X}.')
ql.os.exit_code = exception_code
ql.emu_stop()
ql.hook_address(__post_exception_filter, ret_addr)
exception_filter_args = [(POINTER, exception_pointers_ptr)]
# Resume execution at the registered unhandled exception filter.
# If a program is using a custom unhandled exception filter as an anti-debugging
# trick, then the exception filter might not return.
# TODO: This relies on the hook being marked 'passthru' so that Qiling
# doesn't rewind after it returns. However, this is not entirely intended
# behavior of passthru, so this is a bit of a hack. Maybe find some
# way to rewrite without passthru.
ql.os.fcall.call_native(exception_filter, exception_filter_args, ret_addr)
# NTSTATUS EtwNotificationRegister(
# LPCGUID ProviderGuid,
# ULONG Type,
# PVOID CallbackFunction,
# PVOID CallbackContext,
# PVOID* RegistrationHandle
# );
@winsdkapi(cc=STDCALL, params={
'ProviderGuid': PVOID,
'Type': DWORD,
'CallbackFunction': PVOID,
'CallbackContext': PVOID,
'RegistrationHandle': PVOID
})
def hook_EtwNotificationRegister(ql: Qiling, address: int, params):
reg_handle_ptr = params['RegistrationHandle']
# It is very important to have a hook for this function
# because it is called by some Windows DLLs (sechost.dll,
# advapi32.dll) during initialization when the global
# CRT lock is held.
# If a DllMain aborts here, then the global CRT lock is never
# freed and any attempt to lock the global CRT lock *anywhere*
# will crash us.
# TODO: See if a more thorough implementation
# is needed for this function.
# For now, just create a dummy handle, and return it.
handle = Handle()
ql.os.handle_manager.append(handle)
if reg_handle_ptr:
ql.mem.write_ptr(reg_handle_ptr, handle.id)
return STATUS_SUCCESS
# NTSYSAPI
# VOID RtlRaiseException(
# PEXCEPTION_RECORD ExceptionRecord
# );
@winsdkapi(cc=STDCALL, params={
'ExceptionRecord': PVOID
}, passthru=True)
def hook_RtlRaiseException(ql: Qiling, address: int, params):
return
# NTSYSAPI
# PRUNTIME_FUNCTION RtlVirtualUnwind(
# DWORD HandlerType,
# DWORD64 ImageBase,
# DWORD64 ControlPc,
# PRUNTIME_FUNCTION FunctionEntry,
# PCONTEXT ContextRecord,
# PVOID* HandlerData,
# PDWORD64 EstablisherFrame,
# PKNONVOLATILE_CONTEXT_POINTERS ContextPointers
# );
@winsdkapi(cc=STDCALL, params={
'HandlerType': DWORD,
'ImageBase': PVOID,
'ControlPc': PVOID,
'FunctionEntry': PVOID,
'ContextRecord': PVOID,
'HandlerData': PVOID,
'EstablisherFrame': PVOID,
'ContextPointers': PVOID
}, passthru=True)
def hook_RtlVirtualUnwind(ql: Qiling, address: int, params):
return
# NTSYSAPI
# VOID RtlUnwindEx(
# PVOID TargetFrame,
# PVOID TargetIp,
# PEXCEPTION_RECORD ExceptionRecord,
# PVOID ReturnValue,
# PCONTEXT OriginalContext,
# PUNWIND_HISTORY_TABLE HistoryTable
# );
@winsdkapi(cc=STDCALL, params={
'TargetFrame': PVOID,
'TargetIp': PVOID,
'ExceptionRecord': PVOID,
'ReturnValue': PVOID,
'OriginalContext': PVOID,
'HistoryTable': PVOID
}, passthru=True)
def hook_RtlUnwindEx(ql: Qiling, address: int, params):
return
# NTSYSAPI
# BOOLEAN RtlDispatchException(
# PEXCEPTION_RECORD ExceptionRecord,
# PCONTEXT ContextRecord
# );
@winsdkapi(cc=STDCALL, params={
'ExceptionRecord': PVOID,
'ContextRecord': PVOID
}, passthru=True)
def hook_RtlDispatchException(ql: Qiling, address: int, params):
return
# NTSYSAPI
# VOID RtlRestoreContext(
# PCONTEXT ContextRecord,
# PEXCEPTION_RECORD ExceptionRecord
# );
@winsdkapi(cc=CDECL, params={
'ContextRecord': PVOID,
'ExceptionRecord': PVOID
}, passthru=True)
def hook_RtlRestoreContext(ql: Qiling, address: int, params):
return
# NTSYSAPI
# VOID RtlCaptureContext(
# PCONTEXT ContextRecord
# );
@winsdkapi(cc=STDCALL, params={
'ContextRecord': PVOID
}, passthru=True)
def hook_RtlCaptureContext(ql: Qiling, address: int, params):
return
# NTSYSAPI
# VOID RtlCaptureContext2(
# PCONTEXT ContextRecord,
# ULONG Flags
# );
@winsdkapi(cc=STDCALL, params={
'ContextRecord': PVOID,
'Flags': DWORD
}, passthru=True)
def hook_RtlCaptureContext2(ql: Qiling, address: int, params):
return
# NTSYSAPI
# NTSTATUS RtlInitializeExtendedContext2(
# USHORT Version,
# USHORT ContextFlags,
# ULONG ExtensionCount,
# ULONG *ExtensionSizes,
# ULONG BufferSize,
# PVOID Buffer,
# PCONTEXT Context,
# ULONG *LengthReturned
# );
@winsdkapi(cc=STDCALL, params={
'Version': WORD,
'ContextFlags': WORD,
'ExtensionCount': DWORD,
'ExtensionSizes': PVOID,
'BufferSize': DWORD,
'Buffer': PVOID,
'Context': PVOID,
'LengthReturned': PVOID
}, passthru=True)
def hook_RtlInitializeExtendedContext2(ql: Qiling, address: int, params):
return
# NTSYSAPI
# NTSTATUS RtlGetExtendedContextLength2(
# USHORT Version,
# USHORT ContextFlags,
# ULONG ExtensionCount,
# ULONG *ExtensionSizes,
# PULONG Length
# );
@winsdkapi(cc=STDCALL, params={
'Version': WORD,
'ContextFlags': WORD,
'ExtensionCount': DWORD,
'ExtensionSizes': PVOID,
'Length': PVOID
}, passthru=True)
def hook_RtlGetExtendedContextLength2(ql: Qiling, address: int, params):
return
qiling/os/windows/structs.py
+16
View File
@@ -1545,3 +1545,19 @@ def make_win32_find_data(archbits: int, *, wide: bool):
)
return WIN32_FIND_DATA
# https://learn.microsoft.com/en-us/windows/win32/api/winnt/ns-winnt-exception_pointers
def make_exception_pointers(archbits: int):
"""Generate an EXCEPTION_POINTERS structure class.
"""
native_type = struct.get_native_type(archbits)
Struct = struct.get_aligned_struct(archbits)
class EXCEPTION_POINTERS(Struct):
_fields_ = (
('ExceptionRecord', native_type),
('ContextRecord', native_type)
)
return EXCEPTION_POINTERS
qiling/profiles/linux.ql
+1
View File
@@ -1,6 +1,7 @@
[CODE]
# ram_size 0xa00000 is 10MB
ram_size = 0xa00000
load_address = 0x1000000
entry_point = 0x1000000
qiling/profiles/windows.ql
+1
View File
@@ -23,6 +23,7 @@ KI_USER_SHARED_DATA = 0x7ffe0000
[CODE]
# ram_size 0xa00000 is 10MB
ram_size = 0xa00000
load_address = 0x1000000
entry_point = 0x1000000
[KERNEL]
tests/__init__.py
View File
tests/profiles/blob_raw.ql
+4
View File
@@ -0,0 +1,4 @@
[CODE]
load_address = 0x10000000
entry_point = 0x10000008
ram_size = 0xa00000
tests/profiles/uboot_bin.ql
+2
View File
@@ -1,6 +1,8 @@
[CODE]
ram_size = 0xa00000
load_address = 0x80800000
entry_point = 0x80800000
heap_address = 0xa0000000
heap_size = 0x300000
tests/qdb_scripts/arm.qdb
+31 -7
View File
@@ -1,13 +1,37 @@
# This line is demonstrate comment in qdb script
# break on entry to main
b 0x000103fc
x/10wx 0x7ff3cee4
x $sp
x $sp + 0x10
x/5i 0x047ba9e0
b 0x047ba9ec
# break on call to puts
b 0x00010414
# run till main
c
s
# show stack entries
x/8xw $sp
# run till puts
c
# show argument passed to puts
info args 1
# show instructions passed call till end of function
x/4i ($pc + 4)
# step over call to puts
n
# show snapshot diff
info snapshot
# step backwards to start of main
p
p
# re-run till the end of program to test that nothing breaks
c
c
# quit
q
tests/qdb_scripts/arm_static.qdb
+37
View File
@@ -0,0 +1,37 @@
# break on entry to main
b 0x000102e4
# break on call to puts
b 0x000102ee
# run till main
c
# show stack entries
x/8xw $sp
# run till puts
c
# show argument passed to puts
info args 1
# show instructions passed call till end of function
x/3i ($pc + 4)
# step over call to puts
n
# show snapshot diff
info snapshot
# step backwards to start of main
p
p
# re-run till the end of program to test that nothing breaks
c
c
# quit
q
tests/qdb_scripts/mips32el.qdb
+31 -7
View File
@@ -1,13 +1,37 @@
# This line is demonstrate comment in qdb script
# break on entry to main
b 0x565555e0
x/10wx 0x7ff3cec0
x $sp
x $sp + 0x10
x/5i 0x047bac40
b 0x047bac50
# break on call to puts
b 0x56555600
# run till main
c
s
# show stack entries
x/8xw $sp
# run till puts
c
# show argument passed to puts
info args 1
# show instructions passed call till end of function
x/5i ($pc + 4)
# step over call to puts
n
# show snapshot diff
info snapshot
# step backwards to start of main
p
p
# re-run till the end of program to test that nothing breaks
c
c
# quit
q
tests/qdb_scripts/x86.qdb
+32 -6
View File
@@ -1,11 +1,37 @@
# This line is demonstrate comment in qdb script
# break on entry to main
b 0x5655551d
x/4wx 0x7ff3cee0
x $esp
x $esp + 0x4
x/5i 0x047bac70
s
# break on call to printf
b 0x56555542
# run till main
c
# show stack entries
x/8xw $esp
# run till printf
c
# show argument passed to printf
info args 1
# show instructions passed call till end of function
x/8i ($eip + 5)
# step over call to printf
n
# show snapshot diff
info snapshot
# step backwards to start of main
p
p
# re-run till the end of program to test that nothing breaks
c
c
# quit
q
tests/test_blob.py
+89 -12
View File
@@ -10,13 +10,17 @@ sys.path.append("..")
from qiling.core import Qiling
from qiling.const import QL_ARCH, QL_OS, QL_VERBOSE
from qiling.os.const import STRING
from qiling.os.const import STRING, POINTER, SIZE_T
class BlobTest(unittest.TestCase):
def test_uboot_arm(self):
def my_getenv(ql, *args, **kwargs):
env = {"ID": b"000000000000000", "ethaddr": b"11:22:33:44:55:66"}
def my_getenv(ql: Qiling):
env = {
"ID": b"000000000000000",
"ethaddr": b"11:22:33:44:55:66"
}
params = ql.os.resolve_fcall_params({'key': STRING})
value = env.get(params["key"], b"")
@@ -26,12 +30,23 @@ class BlobTest(unittest.TestCase):
ql.arch.regs.r0 = value_addr
ql.arch.regs.arch_pc = ql.arch.regs.lr
def check_password(ql, *args, **kwargs):
passwd_output = ql.mem.read(ql.arch.regs.r0, ql.arch.regs.r2)
passwd_input = ql.mem.read(ql.arch.regs.r1, ql.arch.regs.r2)
self.assertEqual(passwd_output, passwd_input)
def check_password(ql: Qiling):
params = ql.os.resolve_fcall_params({
'ptr1': POINTER, # points to real password
'ptr2': POINTER, # points to user provided password
'size': SIZE_T # comparison length
})
def partial_run_init(ql):
ptr1 = params['ptr1']
ptr2 = params['ptr2']
size = params['size']
real_password = ql.mem.read(ptr1, size)
user_password = ql.mem.read(ptr2, size)
self.assertSequenceEqual(real_password, user_password, seq_type=bytearray)
def partial_run_init(ql: Qiling):
# argv prepare
ql.arch.regs.arch_sp -= 0x30
arg0_ptr = ql.arch.regs.arch_sp
@@ -56,16 +71,78 @@ class BlobTest(unittest.TestCase):
ql = Qiling(code=uboot_code[0x40:], archtype=QL_ARCH.ARM, ostype=QL_OS.BLOB, profile="profiles/uboot_bin.ql", verbose=QL_VERBOSE.DEBUG)
image_base_addr = ql.loader.load_address
ql.hook_address(my_getenv, image_base_addr + 0x13AC0)
ql.hook_address(check_password, image_base_addr + 0x48634)
imgbase = ql.loader.images[0].base
ql.hook_address(my_getenv, imgbase + 0x13AC0)
ql.hook_address(check_password, imgbase + 0x48634)
partial_run_init(ql)
ql.run(image_base_addr + 0x486B4, image_base_addr + 0x48718)
ql.run(imgbase + 0x486B4, imgbase + 0x48718)
del ql
def test_blob_raw(self):
def run_checksum_emu(input_data_buffer: bytes) -> int:
"""
Callable function that takes input data buffer and returns the checksum.
"""
BASE_ADDRESS = 0x10000000
CHECKSUM_FUNC_ADDR = BASE_ADDRESS + 0x8
END_ADDRESS = 0x100000ba
DATA_ADDR = 0xa0000000
STACK_ADDR = 0xb0000000
with open("../examples/rootfs/blob/example_raw.bin", "rb") as f:
raw_code = f.read()
ql = Qiling(code=raw_code, archtype=QL_ARCH.ARM, ostype=QL_OS.BLOB, profile="profiles/blob_raw.ql", verbose=QL_VERBOSE.DEBUG, thumb=True)
input_data_len = len(input_data_buffer)
# Map memory for data and stack
ql.mem.map(STACK_ADDR, 0x2000)
ql.mem.map(DATA_ADDR, ql.mem.align_up(input_data_len + 0x100))
# Write input data
ql.mem.write(DATA_ADDR, input_data_buffer)
# Set up registers
ql.arch.regs.sp = STACK_ADDR + 0x2000 - 4
ql.arch.regs.r0 = DATA_ADDR
ql.arch.regs.r1 = input_data_len
ql.arch.regs.pc = CHECKSUM_FUNC_ADDR
ql.arch.regs.lr = 0xbebebebe
ql.run(begin=CHECKSUM_FUNC_ADDR, end=END_ADDRESS)
result = ql.arch.regs.r0
return result
def calculate_expected_checksum(input_data_buffer: bytes) -> int:
"""
Python implementation of the expected checksum calculation.
"""
input_data_len = len(input_data_buffer)
expected_checksum = 0
if input_data_len >= 1 and input_data_buffer[0] == 0xDE: # MAGIC_VALUE_1
for i in range(min(input_data_len, 4)):
expected_checksum += input_data_buffer[i]
expected_checksum += 0x10
elif input_data_len >= 2 and input_data_buffer[1] == 0xAD: # MAGIC_VALUE_2
for i in range(input_data_len):
expected_checksum ^= input_data_buffer[i]
expected_checksum += 0x20
else:
for i in range(input_data_len):
expected_checksum += input_data_buffer[i]
return expected_checksum & 0xFF
test_input = b"\x01\x02\x03\x04\x05"
self.assertEqual(run_checksum_emu(test_input), calculate_expected_checksum(test_input))
if __name__ == "__main__":
unittest.main()
tests/test_qdb.py
+39 -22
View File
@@ -1,41 +1,58 @@
#!/usr/bin/env python3
#
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
import sys, unittest
import sys
import unittest
sys.path.append("..")
from qiling import Qiling
from qiling.const import QL_VERBOSE
class DebuggerTest(unittest.TestCase):
def test_qdb_mips32el_hello(self):
rootfs = "../examples/rootfs/mips32el_linux"
path = rootfs + "/bin/mips32el_hello"
def __test_common(self, vpath: str, rootfs: str, script: str) -> None:
"""Load a common setup for all test cases.
"""
ql = Qiling([path], rootfs)
ql.debugger = "qdb::rr:qdb_scripts/mips32el.qdb"
ql.run()
del ql
ql = Qiling([f'{rootfs}{vpath}'], rootfs, verbose=QL_VERBOSE.DEBUG)
ql.debugger = f'qdb::rr:{script}'
try:
ql.run()
except SystemExit as ex:
self.assertEqual(ex.code, 0)
def test_qdb_mips32el_hello(self):
self.__test_common(
r'/bin/mips32el_hello',
r'../examples/rootfs/mips32el_linux',
r'qdb_scripts/mips32el.qdb'
)
def test_qdb_arm_hello(self):
rootfs = "../examples/rootfs/arm_linux"
path = rootfs + "/bin/arm_hello"
self.__test_common(
r'/bin/arm_hello',
r'../examples/rootfs/arm_linux',
r'qdb_scripts/arm.qdb'
)
ql = Qiling([path], rootfs)
ql.debugger = "qdb::rr:qdb_scripts/arm.qdb"
ql.run()
del ql
def test_qdb_arm_hello_static(self):
self.__test_common(
r'/bin/arm_hello_static',
r'../examples/rootfs/arm_linux',
r'qdb_scripts/arm_static.qdb'
)
def test_qdb_x86_hello(self):
rootfs = "../examples/rootfs/x86_linux"
path = rootfs + "/bin/x86_hello"
self.__test_common(
r'/bin/x86_hello',
r'../examples/rootfs/x86_linux',
r'qdb_scripts/x86.qdb'
)
ql = Qiling([path], rootfs)
ql.debugger = "qdb::rr:qdb_scripts/x86.qdb"
ql.run()
del ql
if __name__ == "__main__":
if __name__ == '__main__':
unittest.main()
tests/test_windows_cpp_x86.py
+64
View File
@@ -0,0 +1,64 @@
#!/usr/bin/env python3
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
import sys, unittest
sys.path.append("..")
from qiling import Qiling
from qiling.const import QL_VERBOSE
from qiling.extensions import pipe
def good_bad_count(test_str: str, good_str="GOOD", bad_str="BAD"):
good_count = test_str.count(good_str)
bad_count = test_str.count(bad_str)
return good_count, bad_count
class CppTests_x86(unittest.TestCase):
def test_cpp_helloworld(self):
""" Test a basic C++ Hello World program which prints "Hello World!"
to the console using std::cout.
"""
ql = Qiling(["../examples/rootfs/x86_windows/bin/except/CppHelloWorld_x86.exe"], "../examples/rootfs/x86_windows/", verbose=QL_VERBOSE.DEFAULT)
ql.os.stdout = pipe.SimpleStringBuffer()
ql.run()
conout = ql.os.stdout.read()
self.assertEqual(conout, b"Hello World!\x0d\x0a")
del ql
def test_cpp_types(self):
""" This program tests several C++ type-related runtime features.
- typeid
- dynamic_cast
- virtual methods
- virtual destructors
"""
ql = Qiling(["../examples/rootfs/x86_windows/bin/except/TestCppTypes_x86.exe"], "../examples/rootfs/x86_windows/", verbose=QL_VERBOSE.DEFAULT)
ql.os.stdout = pipe.SimpleStringBuffer()
ql.run()
conout = ql.os.stdout.read().decode('utf-8')
good_count, bad_count = good_bad_count(conout)
# the test program should print
# - 'GOOD' 12 times
# - 'BAD' 0 times
self.assertEqual(good_count, 12)
self.assertEqual(bad_count, 0)
del ql
if __name__ == '__main__':
unittest.main()
tests/test_windows_cpp_x8664.py
+181
View File
@@ -0,0 +1,181 @@
#!/usr/bin/env python3
#
# Cross Platform and Multi Architecture Advanced Binary Emulation Framework
#
import sys, unittest
sys.path.append("..")
from qiling import Qiling
from qiling.const import QL_VERBOSE
from qiling.extensions import pipe
def good_bad_count(test_str: str, good_str="GOOD", bad_str="BAD"):
good_count = test_str.count(good_str)
bad_count = test_str.count(bad_str)
return good_count, bad_count
class CppTests_x8664(unittest.TestCase):
def test_cpp_helloworld(self):
""" Test a basic C++ Hello World program which prints "Hello World!"
to the console using std::cout.
"""
ql = Qiling(["../examples/rootfs/x8664_windows/bin/except/CppHelloWorld.exe"], "../examples/rootfs/x8664_windows/", verbose=QL_VERBOSE.DEFAULT)
ql.os.stdout = pipe.SimpleStringBuffer()
ql.run()
conout = ql.os.stdout.read()
self.assertEqual(conout, b"Hello World!\x0d\x0a")
del ql
def test_cpp_types(self):
""" This program tests several C++ type-related runtime features.
- typeid
- dynamic_cast
- virtual methods
- virtual destructors
"""
ql = Qiling(["../examples/rootfs/x8664_windows/bin/except/TestCppTypes.exe"], "../examples/rootfs/x8664_windows/", verbose=QL_VERBOSE.DEFAULT)
ql.os.stdout = pipe.SimpleStringBuffer()
ql.run()
conout = ql.os.stdout.read().decode('utf-8')
good_count, bad_count = good_bad_count(conout)
# the test program should print
# - 'GOOD' 12 times
# - 'BAD' 0 times
self.assertEqual(good_count, 12)
self.assertEqual(bad_count, 0)
del ql
def test_soft_seh(self):
""" Test software SEH.
This test program uses __try..__catch and calls RaiseException with
a custom code. If software SEH is functioning correctly, the program
should be able to invoke its __catch-block and continue execution after.
"""
ql = Qiling(["../examples/rootfs/x8664_windows/bin/except/TestSoftSEH.exe"], "../examples/rootfs/x8664_windows/", verbose=QL_VERBOSE.DEFAULT)
ql.os.stdout = pipe.SimpleStringBuffer()
ql.run()
conout = ql.os.stdout.read().decode('utf-8')
good_count, bad_count = good_bad_count(conout)
# the test program should print
# - 'GOOD' 4 times
# - 'BAD' 0 times
self.assertEqual(good_count, 4)
self.assertEqual(bad_count, 0)
# If the exception handler was not invoked for some reason,
# the program may terminate abnormally with a non-zero exit
# code.
self.assertEqual(ql.os.exit_code, 0)
del ql
def test_soft_cppex(self):
""" Test software C++ exceptions.
This test program tests try..catch in various ways. If exception dispatching
and stack unwinding are functioning correctly, the program will run to completion.
- Simple try..catch
- Try..catch with throw data
- Nested try..catch with throw data
"""
ql = Qiling(["../examples/rootfs/x8664_windows/bin/except/TestCppEx.exe"], "../examples/rootfs/x8664_windows/", verbose=QL_VERBOSE.DEFAULT)
ql.os.stdout = pipe.SimpleStringBuffer()
ql.run()
conout = ql.os.stdout.read().decode('utf-8')
good_count, bad_count = good_bad_count(conout, 'y', 'n')
# the test program should print
# - 'y' 14 times
# - 'n' 0 times
self.assertEqual(good_count, 14)
self.assertEqual(bad_count, 0)
# If the exception handler was not invoked for some reason,
# the program may terminate abnormally with a non-zero exit
# code.
self.assertEqual(ql.os.exit_code, 0)
del ql
def test_cppex_unhandled_filtered(self):
""" Test unhandled C++ exceptions.
This program registers its own unhandled exception filter via
SetUnhandledExceptionFilter, then throws an uncaught exception.
If unhandled exception filters are functioning correctly,
the program's custom exception filter will be reached, but
execution will NOT resume after the exception.
Instead, the program is expected to terminate abnormally
with status code 0xE06D7363 (C++ runtime exception).
"""
ql = Qiling(["../examples/rootfs/x8664_windows/bin/except/TestCppExUnhandled.exe"], "../examples/rootfs/x8664_windows/", verbose=QL_VERBOSE.DEFAULT)
ql.os.stdout = pipe.SimpleStringBuffer()
ql.run()
conout = ql.os.stdout.read().decode('utf-8')
good_count, bad_count = good_bad_count(conout)
# the test program should print
# - 'GOOD' 3 times
# - 'BAD' 0 times
self.assertEqual(good_count, 3)
self.assertEqual(bad_count, 0)
# The program should have terminated abnormally
# with status code 0xE06D7363 (C++ runtime exception).
self.assertEqual(ql.os.exit_code, 0xE06D7363)
del ql
def test_cppex_unhandled_unfiltered(self):
""" Test unhandled C++ exceptions.
This program throws an uncaught C++ exception.
The program is expected to terminate abnormally
with status code 0xC0000409 (STATUS_STACK_BUFFER_OVERRUN).
"""
ql = Qiling(["../examples/rootfs/x8664_windows/bin/except/TestCppExUnhandled2.exe"], "../examples/rootfs/x8664_windows/", verbose=QL_VERBOSE.DEFAULT)
ql.os.stdout = pipe.SimpleStringBuffer()
ql.run()
conout = ql.os.stdout.read().decode('utf-8')
good_count, bad_count = good_bad_count(conout)
# the test program should print
# - 'GOOD' 1 time
# - 'BAD' 0 times
self.assertEqual(good_count, 1)
self.assertEqual(bad_count, 0)
# The program is expected to terminate abnormally
# with status code 0xC0000409 (STATUS_STACK_BUFFER_OVERRUN)
# https://devblogs.microsoft.com/oldnewthing/20190108-00/?p=100655
#
self.assertEqual(ql.os.exit_code, 0xC0000409)
del ql
if __name__ == '__main__':
unittest.main()