mirror of
https://github.com/qilingframework/qiling.git
synced 2026-10-12 02:21:40 +00:00
updated ARCHITECTURE files
This commit is contained in:
xwings
+3
-1
@@ -1,9 +1,11 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# Qiling Architecture
|
||||
|
||||
Generated with [eatmycode](https://github.com/xwings/eatmycode).
|
||||
|
||||
## Read First
|
||||
|
||||
Before planning code changes or reviewing code, read
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# Agent Rules
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# Firmware Routes
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# Operating System Routes
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# Runtime Routes
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# CPU Architecture and Calling Conventions
|
||||
@@ -21,7 +21,7 @@ or cross-platform compatibility certification.
|
||||
| --- | --- |
|
||||
| [arch/arch.py](../../qiling/arch/arch.py): `QlArch` | CPU, pointer width, stack, context interface |
|
||||
| [arch/register.py](../../qiling/arch/register.py): `QlRegisterManager` | Name-to-Unicorn registers and PC/SP aliases |
|
||||
| [arch/](../../qiling/arch/), [arch/models.py](../../qiling/arch/models.py) | Concrete x86/ARM/MIPS/RISC-V/PPC/Cortex-M adapters and CPU enums |
|
||||
| [arch/](../../qiling/arch/), [arch/models.py](../../qiling/arch/models.py), `*_const.py` | Concrete x86/ARM/MIPS/RISC-V/PPC/Cortex-M adapters, CPU enums, register maps and `EXCP` exception codes |
|
||||
| [cc/__init__.py](../../qiling/cc/__init__.py): `QlCC`, `QlCommonBaseCC` | Register/stack slots, result, return address, frame unwind |
|
||||
| [arch/cortex_m.py](../../qiling/arch/cortex_m.py): `QlArchCORTEX_M` | Exception/vector state and task-aware Unicorn |
|
||||
| [test_cpu_models.py](../../tests/test_cpu_models.py), [test_riscv.py](../../tests/test_riscv.py), [test_shellcode.py](../../tests/test_shellcode.py) | Model selection and instruction/ABI samples |
|
||||
@@ -43,8 +43,13 @@ versions come from the root manifest, not individual adapters.
|
||||
multiword arguments, shadow space, register exhaustion, stack return
|
||||
addresses and unwind rules differ (`cc/__init__.py`, `cc/intel.py`).
|
||||
- Endianness, CPU mode and Thumb PC handling affect execution and instruction
|
||||
decoding. Preserve `effective_pc` and Cortex-M exception return behavior
|
||||
when changing ARM mode; inspect the core Thumb workaround too.
|
||||
decoding. Preserve the ARM adapter's `effective_pc` (consumers fall back
|
||||
to `arch_pc` via `getattr`) and Cortex-M exception return behavior when
|
||||
changing ARM mode; inspect the core Thumb workaround too.
|
||||
- Interrupt numbers delivered to `hook_intno` are QEMU exception codes.
|
||||
**Observed:** ARM/ARM64 and MIPS Linux traps use the `EXCP` enums in
|
||||
`cortex_m_const.py` and `mips_const.py`; x86/RISC-V/PPC still pass
|
||||
literals. Prefer an existing enum and extend it for new codes.
|
||||
- Concrete adapters instantiate engine/assembler/disassembler objects;
|
||||
supported enums do not imply support for every OS/CPU combination.
|
||||
Cortex-M uses `MultiTaskUnicorn` and hardware interrupt state.
|
||||
@@ -73,8 +78,9 @@ never substitute host pointer size for a guest ABI.
|
||||
|
||||
From `tests/`: `python -m unittest test_cpu_models test_riscv test_shellcode`
|
||||
with root dependencies and relevant rootfs samples. All selected modules
|
||||
passed during rebuild; CPU-model suite has 7 tests. Their scope is selected
|
||||
models/instructions and guest execution, not instruction-set conformance.
|
||||
passed during the latest refresh (11 CPU-model/RISC-V and 8 shellcode
|
||||
cases). Their scope is selected models/instructions and guest execution,
|
||||
not instruction-set conformance.
|
||||
For PPC changes use `test_elf.ELFTest.test_elf_linux_powerpc`; for ABI changes
|
||||
run the consuming OS API/syscall regression as well.
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# Bare-metal Execution
|
||||
@@ -71,8 +71,8 @@ semantics into unrelated POSIX/Windows run loops.
|
||||
## Verification
|
||||
|
||||
From `tests/`: `python -m unittest test_mcu.MCUTest.test_mcu_snapshot_stm32f411 test_mcu.MCUTest.test_mcu_usart_input_stm32f411 test_blob.BlobTest.test_uboot_arm`
|
||||
passed 3 cases during rebuild. Root dependencies and matching STM32/U-Boot
|
||||
fixtures are required. Run `python test_mcu.py` for broader device/timing
|
||||
passed 3 cases during the latest refresh. Root dependencies and matching
|
||||
STM32/U-Boot fixtures are required. Run `python test_mcu.py` for broader device/timing
|
||||
changes and `python test_blob.py` for raw behavior; the full suites are not
|
||||
certified by the selected cases.
|
||||
|
||||
@@ -80,7 +80,8 @@ certified by the selected cases.
|
||||
|
||||
`test_blob.BlobTest.test_blob_raw` fails because rootfs lacks
|
||||
`blob/example_raw.bin`; [fixture source](../../examples/src/blob/Makefile)
|
||||
exists but was not built or copied into the submodule during this rebuild.
|
||||
exists but is not built or copied into the submodule; the case still
|
||||
errors with a missing-file error.
|
||||
Fast-mode faults currently stop cleanly instead of delivering hardware
|
||||
HardFault. Timeout units differ by execution path; full scheduler timing
|
||||
and MCU exception fidelity remain unverified.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# CLI, Packaging and Development Tooling
|
||||
@@ -95,10 +95,12 @@ CLI/version/profile behavior and TUI import outside the checkout. For
|
||||
checkout behavior from `tests/`: `python test_qltool.py`; rootfs samples
|
||||
are required for argument, JSON, filter and coverage cases.
|
||||
|
||||
Rebuild evidence: sdist plus wheel-from-sdist build passed, artifact metadata
|
||||
checks and `poetry check --lock` passed, 8 selected CLI tests passed, and 3 installed-wheel cases
|
||||
passed in a second fresh environment. See [root verification](../../ARCHITECTURE.md#verification)
|
||||
for common setup and Linux aggregate. No configured lint/type-check command exists.
|
||||
Latest refresh evidence: sdist plus wheel-from-sdist build, `twine check
|
||||
--strict` and `poetry check --lock` passed; 5 checkout CLI cases and 3
|
||||
installed-wheel cases passed in a second fresh environment; the wheel
|
||||
contains `qltui.py`, the 7 profiles and 43 GDB XML files. See
|
||||
[root verification](../../ARCHITECTURE.md#verification) for common setup
|
||||
and Linux aggregate. No configured lint/type-check command exists.
|
||||
|
||||
## Known Gaps
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# Core Runtime
|
||||
@@ -45,6 +45,9 @@ by `hookcallback` and re-raised by `emu_start`, not silently consumed.
|
||||
- Factories derive modules/class names from enums with explicit format/arch
|
||||
mappings. A new enum alone is insufficient (`utils.py`). Hook callback
|
||||
signatures and `QL_HOOK_BLOCK` behavior must survive dispatcher changes.
|
||||
Interrupt, memory-fault and invalid-instruction events that no hook
|
||||
handles raise `QlErrorCoreHook` (`core_hooks.py`); OS personalities
|
||||
choose which interrupt numbers they hook.
|
||||
- Profiles use defaults plus OS/user overrides; MCU uses YAML-derived maps
|
||||
(`profile_setup`). Parsing belongs here; the consumer owns each key.
|
||||
- `save` includes only selected components. `restore` applies only included
|
||||
@@ -80,7 +83,8 @@ not a claim that the tree is an acyclic import graph.
|
||||
|
||||
From `tests/`, after root setup: `python -m unittest test_shellcode`;
|
||||
`python -m unittest test_elf.ELFTest.test_memory_search` checks memory-facing
|
||||
facade behavior. Both passed during rebuild on Linux/Python 3.13/Unicorn 2.1.3.
|
||||
facade behavior. Both passed during the latest refresh on Linux/Python
|
||||
3.13/Unicorn 2.1.3.
|
||||
For snapshot changes use matching cases in `test_elf.py` and
|
||||
`test_mcu.MCUTest.test_mcu_snapshot_stm32f411` (the MCU case also passed).
|
||||
Fixtures are relative to `tests/`. Passing these does not certify every
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# Debuggers
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# DOS and BIOS Interrupts
|
||||
@@ -69,8 +69,8 @@ not apply to DOS interrupt leaves.
|
||||
## Verification
|
||||
|
||||
From `tests/`: `python -m unittest test_dos` passed one program test during
|
||||
rebuild with the existing DOS rootfs fixture. Use `python test_dos_exe.py`
|
||||
for EXE-loader changes; it was not part of the verified subset. Terminal
|
||||
the latest refresh with the existing DOS rootfs fixture. Use
|
||||
`python test_dos_exe.py` for EXE-loader changes; it was not part of the verified subset. Terminal
|
||||
cases need usable curses/TTY support. A successful sample does not certify
|
||||
all BIOS services or error leaves.
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# Instrumentation and Integrations
|
||||
@@ -77,7 +77,7 @@ From `tests/`: `python -m unittest test_history` passed 4 tests;
|
||||
`python test_uefi.py` passed 2 direct integration tests including sanitizer
|
||||
behavior. For coverage output run
|
||||
`python -m unittest test_qltool.Qltool_Test.test_qltool_coverage` with UEFI
|
||||
fixtures; that coverage case also passed during rebuild. For r2 changes use `python test_r2.py` after root setup with `RE`;
|
||||
fixtures; that coverage case also passed during the latest refresh. For r2 changes use `python test_r2.py` after root setup with `RE`;
|
||||
this optional check was not run. AFL/IDA require their external runtimes and
|
||||
matching harnesses; a base-package import does not validate them.
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# Hardware Models and MMIO
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# Linux Kernel Proxy
|
||||
@@ -12,7 +12,8 @@ Read when: changing qiling/os/posix/kernel_proxy/, forwarded syscalls, proxy des
|
||||
|
||||
Owns opt-in forwarding of selected Linux syscalls through a helper process,
|
||||
argument/buffer transport and proxy-owned descriptors. **In progress:**
|
||||
Phase 0 code exists and 21 tests passed, but actual guest dispatch and
|
||||
Phase 0 code exists; 20 of 21 tests pass and the pipe2 write-back case
|
||||
errors on descriptor cleanup (Known Gaps). Actual guest dispatch and
|
||||
cross-ABI buffer semantics have unresolved gaps. Later phases in
|
||||
[TODO.md](../../TODO.md) remain designs, not implemented capability.
|
||||
|
||||
@@ -74,8 +75,10 @@ No additional runtime package is required beyond the existing manifest.
|
||||
|
||||
## Verification
|
||||
|
||||
From `tests/`: `python -m unittest test_kernel_proxy` passed 21 tests on
|
||||
Linux x86_64/Python 3.13 with current hello fixtures. Helpers invoke real
|
||||
From `tests/`: `python -m unittest test_kernel_proxy` ran 21 tests on
|
||||
Linux x86_64 (WSL2)/Python 3.13 with current hello fixtures during the
|
||||
latest refresh: 20 passed and `test_ptr_out_writes_back_to_guest_memory`
|
||||
errored with `EBADF` after its assertions passed. Helpers invoke real
|
||||
Linux syscalls. These tests exercise IPC and direct hook invocation; they
|
||||
do not establish complete guest-trap integration or cross-architecture
|
||||
struct conversion. Run an actual emulated syscall case for dispatch changes.
|
||||
@@ -85,8 +88,10 @@ struct conversion. Run an actual emulated syscall case for dispatch changes.
|
||||
`KernelProxy._make_forwarder` returns `(ql, *args)`, while POSIX dispatch
|
||||
counts only ordinary positional parameters; tests calling the hook directly
|
||||
can miss missing guest arguments. `test_ptr_out_writes_back_to_guest_memory`
|
||||
closes helper-owned fd numbers with parent `os.close`; its passing result
|
||||
can depend on coincident parent fd allocation and does not prove cleanup.
|
||||
closes helper-owned fd numbers with parent `os.close`; depending on parent
|
||||
fd allocation this fails with `EBADF` or closes an unrelated parent
|
||||
descriptor. Its marshalling assertions pass; fix cleanup to go through the
|
||||
proxy fd wrapper or the helper before treating the suite as green.
|
||||
Buffer quotas, ABI translation and concurrent IPC access need evidence;
|
||||
none should be implied by Phase 0 passing tests. Preserve scope and record
|
||||
required fixes before claiming later TODO milestones.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# Executable Loaders
|
||||
@@ -75,8 +75,8 @@ OS imports in loaders are intentional construction dependencies.
|
||||
## Verification
|
||||
|
||||
From `tests/`: `python -m unittest test_elf.ELFTest.test_elf_linux_x8664 test_blob.BlobTest.test_uboot_arm`
|
||||
and `python test_uefi.py`. These passed during rebuild with local fixtures.
|
||||
Use direct UEFI script execution: its test bodies are guarded by `__main__`.
|
||||
and `python test_uefi.py`. These passed during the latest refresh with
|
||||
local fixtures. Use direct UEFI script execution: its test bodies are guarded by `__main__`.
|
||||
For changed formats, use the corresponding test files in Code Map; Windows
|
||||
and macOS need their system-library fixtures. Passing ELF does not verify PE.
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# Shared Operating System Services
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# POSIX Personalities and Syscalls
|
||||
@@ -53,6 +53,15 @@ Fixture Makefiles specify cross-compilers locally; no common C standard is decla
|
||||
- Linux `/proc/self` mappings are installed only when not overridden.
|
||||
Profile identity and network keys affect observable guest behavior.
|
||||
macOS kernel and QNX message interfaces retain their own state/layouts.
|
||||
- Linux registers `hook_intno` traps per architecture: the syscall
|
||||
interrupt (via `EXCP` enums on ARM/ARM64/MIPS, literals elsewhere) plus
|
||||
ARM/ARM64 `UDEF` and MIPS `RI` CPU exceptions.
|
||||
`hook_cpu_exception` logs the SIGILL equivalent and calls `ql.stop()`,
|
||||
so falling into non-code bytes ends emulation cleanly instead of raising
|
||||
`QlErrorCoreHook`; no guest signal handler runs (`linux/linux.py`).
|
||||
- Syscall result structs follow `ql.arch.endian`: `statx` selects the
|
||||
`*EB` big-endian ctypes variants for EB guests (`syscall/stat.py`).
|
||||
Check endian handling in any handler that writes ctypes layouts.
|
||||
|
||||
## Dependencies and Boundaries
|
||||
|
||||
@@ -68,14 +77,17 @@ own dispatch. Filesystem/socket/fork operations expose host resources.
|
||||
| --- | --- | --- |
|
||||
| New/fixed syscall | Mapping table, ABI, OS override/common handler | Update this contract; matching `test_posix`/ELF test and affected OS |
|
||||
| Thread/futex/signal | Thread contexts, yielding, futex waiters | `test_elf_multithread.py`; mark unsupported semantics explicitly |
|
||||
| CPU exception/trap | `hook_intno` registrations, `hook_cpu_exception`, `EXCP` enums | Shellcode illegal-instruction cases; read arch for new exception codes |
|
||||
| FD/path/socket | Host object lifetime, errno, guest structure layouts | OS-base checks and syscall regression; proxy checks if shared operations change |
|
||||
| macOS/QNX/kernel API | Local maps/structs, loader imports | Platform-specific test and loader owner |
|
||||
|
||||
## Verification
|
||||
|
||||
From `tests/`: `python -m unittest test_elf.ELFTest.test_elf_linux_x8664 test_riscv test_qnx`
|
||||
passed during rebuild. It checks sample Linux/RISC-V/QNX flows, not every
|
||||
handler. For POSIX changes run `python test_posix.py` (also imports ELF,
|
||||
and `python -m unittest test_shellcode.TestShellcode.test_linux_arm64_illegal_instruction test_shellcode.TestShellcode.test_linux_mips32_illegal_instruction test_elf.ELFTest.test_linux_statx_bigendian`
|
||||
passed during the latest refresh. They check sample Linux/RISC-V/QNX flows,
|
||||
SIGILL-style termination and big-endian `statx`, not every handler.
|
||||
For POSIX changes run `python test_posix.py` (also imports ELF,
|
||||
RISC-V and CLI suites), and the relevant thread/kernel/network/platform
|
||||
script. Follow [shared test-resource rules](cli-build.md#verification) when
|
||||
running these suites. Root Linux aggregate is the broader CI gate; its fixtures, extracted
|
||||
@@ -84,7 +96,9 @@ kernel sample and network/host requirements apply. Full aggregate was not run.
|
||||
## Known Gaps
|
||||
|
||||
macOS CI is commented out. Some thread classes are unset for RISC-V/PPC;
|
||||
signal syscall state is not proof of full delivery. Existing kernel-module
|
||||
signal syscall state is not proof of full delivery. CPU-exception hooks
|
||||
exist only for ARM/ARM64/MIPS; x86, RISC-V and PPC undefined instructions
|
||||
still surface as `QlErrorCoreHook`. Existing kernel-module
|
||||
archives need fixture preparation. `TODO.md` hybrid phases are plans; proxy
|
||||
Phase 0 implementation is documented separately. Current representative
|
||||
checks do not establish kernel fidelity or host isolation.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# UEFI Services
|
||||
@@ -68,8 +68,8 @@ UEFI shares PE parsing infrastructure, not Windows API semantics.
|
||||
## Verification
|
||||
|
||||
From `tests/`: **`python test_uefi.py`** with root dependencies and
|
||||
`examples/rootfs/x8664_efi` fixtures. Both tests passed during rebuild;
|
||||
they exercise interception and sanitized heap behavior. Do not substitute
|
||||
`examples/rootfs/x8664_efi` fixtures. Both tests passed during the latest
|
||||
refresh; they exercise interception and sanitized heap behavior. Do not substitute
|
||||
`python -m unittest test_uefi`: actual bodies are guarded by `__main__`.
|
||||
For package/coverage output changes also use the CLI coverage test through
|
||||
[extensions](extensions.md).
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
eatmycode_version: "2.0.0"
|
||||
eatmycode_version: "2.1.0"
|
||||
---
|
||||
|
||||
# Windows API Emulation
|
||||
|
||||
Reference in New Issue
Block a user