The WASM branch is ready to be included in the rest of Manticore. It doesn't include several advanced features, including a symbolic memory model, symbolic floating point support, or a standard library implementation (WASI/emscripten), but it does have full symbolic semantics for all of the integer instructions.
### Commit log:
* Empty WASM Module
Will run on a .wasm file and do nothing forever
* Read and parse sections
/shrug
* Can partially allocate modules
Just pushing for the weekend
* Blacken
* Can allocate with exports now
* Static initialization
Doesn't evaluate any of the starting expressions, but that's okay
* Make type hints a little more pycharm-friendly
* First part of invocation
* Blacken...
* Barebones execution poc
Can execute the following snippet:
```
main(){
return 42;
}
```
* blacken
* Add some loads and stores
* Add global operations
* Move float methods further down
* Support internal function calls
```
18:20:42 (mc) ehennenfent@nessie:~/wasm ➜ cat loop_check.c
#include <stdio.h>
#include <stdbool.h>
bool check(int arg){
if (((arg << 2) ^ 16) == 36) {
return true;
}
else{
return false;
}
}
int main(){
printf("Hello world!\n");
int g = getchar();
printf("Got: %d\n", g);
int y = 0;
for (int i = 0; i < 20; i++){
y++;
if (y > 18){
return check(g) ? 0 : -1;
}
}
return -1;
}
18:20:54 (mc) ehennenfent@nessie:~/wasm ➜ manticore loop_check.wasm
65: i32.const (16)
11: end
65: i32.const (32)
11: end
Initialization Complete
65: i32.const (0)
65: i32.const (0)
40: i32.load (Offset 4)
65: i32.const (16)
107: i32.sub
34: tee_local (Local 1)
54: i32.store (Offset 4)
65: i32.const (32)
16: call (Func Idx 2)
Called stub function with args: (32,)
HostFunc returned [13]
26: drop
32: get_local (Local 1)
16: call (Func Idx 0)
Called stub function with args: ()
HostFunc returned [13]
34: tee_local (Local 0)
54: i32.store (Offset 0)
65: i32.const (16)
32: get_local (Local 1)
16: call (Func Idx 1)
Called stub function with args: (16, 16)
HostFunc returned [13]
26: drop
65: i32.const (0)
32: get_local (Local 1)
65: i32.const (16)
106: i32.add
54: i32.store (Offset 4)
65: i32.const (-1)
65: i32.const (0)
32: get_local (Local 0)
65: i32.const (1073741823)
113: i32.and
65: i32.const (13)
71: i32.ne
27: select
11: end
[0]
2019-07-25 18:20:58,138: [13842] m.c.manticore:INFO: Generated testcase No. 0 - test
2019-07-25 18:20:58,143: [13842] m.c.manticore:INFO: Results in /home/ehennenfent/wasm/mcore_1evptpip
```
* Improve pickle-friendliness
* Convert wasm module instructions into internal type s
* Fix class naming conflict
Thanks Pickle
* Support bitvecs as an integer type
* Refactor execution to handle single instructions at a time
* Add Atomic Stacks
It's a very inefficient implementation, but we'll make it better
* Handle Concretize Exceptions
Needs a lot of fine tuning but it's a start
* Propagate symbols all the way to return
* Add test generation infrastructure
Still very basic but it's a start
* Attempt to make Travis happy
* Fix missing generation step
* Ignore stack exhaustsion and invalid modules
* Add basic shift instructions
* Add lots of arithmetic instructions
* Fix global initialization
* Add optimization for fixed size left shift
* Make Travis tests less verbose
* Blacken
* Properly handle modulo in shifting
* Add several control flow instructions
* Blacken
* Partially Fix return instruction
* Fix missing return value from look_forward
* Update look_forward to handle nested blocks
* Fix type sigs and executor copy/paste errors, deduplicate test names
* Fix an instruction exit issue
* Fix local ordering
* Blacken
* test_address now passes for wasm
Fixed issue with json2mc.py issue with testname uniqueness
Started some work on floating point
* i hate git
* i hate git
* Adding linenumber to testcase function name
minor float progress
* f32_cmp passed for wasm
* f64_cmp passes for wasm
* f32_bitwise passes except for the problem with assertEqual not working with nan
* f64_bitwise passes except for the problem with assertEqual not working with nan
* Fixed the tests to handle [nan] comparisons
* better handling of nans and infinites
* added integer backing value for f64
* Blacken
* Blacken
* f64.nearest
* Fix argument ordering and implement N-bit memory stores
* Use subtests
* Switch print to logger
* Modify control flow instructions to match spec
"Continuation" != the thing that goes inside the Label
* Teporarily disable call instructions to prevent infinite loop
Still need to figure out control flow for nested function calls
* Blacken
* Fixed infinite loop in _call via block depth tracking
The behavior of `end` needs to depend on how many labels and activations are on the stack. Every time we push or pop a label, we increment or decrement the counter for the current call frame.
* Blacken
* Initial `br` implementation
Probably still bugged
* Support if blocks without an else instruction
* Add call_indirect
* Add grow_memory
* i32_wrap_i64
* Add several arithmetic operators
* Copy i32.ctz impl from x86 TZCNT
* Fix off-by-one in br
I think this is the correct fix (it makes all the _br tests pass) but I'm leaving the TODO there just in case.
* Blacken...
* Fix lingering issue with if block splitting
* Fix bad args to sextend in i64_extend_s_i32
* Fix indirect call null check and Float binop return types
* Improve br_table indexing
See comment
* Remove vestigial argument popping in invoke
* Fix looking forward in _return and _block
* Blacken
* Generate trap and action tests
* Fix extraneous call to exec_for_test
* Fix missing traps in load instructions
* Fix c1=0 case in ctz/clz
Hopefully this also fixes the fact that the original CTZ implementations seems to have been intended for little-endian architectures
* Handle traps in-stride during test execution
* Treat zero division as trap
* Use ctypes to convert integers
* Blacken
* Apparently I was wrong about the CTZ endianness thing
* Better range checks for offsets
This really hammers home why checking whether ea + size is _greater than_ len(mem.data) is strange if mem.data is 0-indexed, but idk, that's what the spec says.
* Make stub functions return the correct number of results
* ctypes can't handle floats
* Add option to run the start function
* Correctly handle functions with unacceptable names
* Blacken
* Treat NotImplemented as a Skip
Also counts individual subtests directly instead of screwing around with trying to grep for them in the log.
* Fix unhandled negative indices
* Fixed some integer instruction bugs
i64/32:
div_s/u
shr_s/u
shl
clz
* Fix global and memory imports
* CC
* Make default loglevel debug
* Rework conversion from int
* Make float_store work
* _actually_ fix float_store
* Restore old float creation method
* Blacken
* Add range checks to float load/store
* Force cast to unsigned in _u instructions
* Fix yet another I32 -> I64 copy paste error
How many times do we have to teach you this lesson, ~old~ young man?
* Skip problematic float tests for now
* Handle floating point inaccuracies when dividing large ints
* Skip conversion and endianness tests
* Rotl, rotl, popcnt
* Blacken
* Add some conversion stubs
* Maybe fix conversions
* Improve reinterpret and float demotion
* Re-enable endianness tests
* Include WASM logging in verbosity controls
* Propagate constraints to executor
* Blacken
* Add prototype symbolic tests
* Add api for invocation to Manticore object
* Temporarily disable native/evm tests
There's no reason to tie up Travis for 40 minutes every time I push a commit.
* Fix wasmworld import
* Concretize br_if
* Pass arg generator to invoke
* Download updated WABT
It's fixed now!
* Generate Symbolic tests (partial)
* Follow m.run format
* Blacken
* Reinitialize Manticore every 50 tests
This should cut down on the performance issues we've been seeing
* Revert "Temporarily disable native/evm tests"
This reverts commit 7abd6c1333.
* I32 and I64 pass Symbolic
* Create new MC on reinit
* Blacken
* Fix return handling in test generation
* Concretize operand in `if_`
* Concretize memory addresses before access
In the long run, we should have a symbolic memory model
* Better memory access concretization (and br_table)
* Concretize float converions and indirect call operands
* Run start method
* Handle size errors in wrap+i64 and select
* Add timeout (and skip extra-problematic tests)
* Blacken
* Aggressively concretize floats
* Recast ints that should have been floats
* Better trap handling
* Unstash float concretization
* Fix log-lived 'trap' attr
* Blacken
* Concretize F32 when promoting to F64
_shouldn't_ strictly be necesssary, but the lazy conversion means that one of the tests fails. Until we get symbolic floating point, this will have to do.
* Docstrings for platforms/wasm.py
* Docstrings for ManticoreWASM
* Add docs for the module structure
* Add WASM to RTD
* missing docstring for Module itself
* Fix sphinx errors
* Bump RTD python version
I'm sorry and I swear I'll fix it later
* Reformat docstrings for sphinx
* Pacify RTD by adding newlines before params
* Missed a few
* Document the types
* Document executor
* A few more type docstrings
* Partial runtime_structure docs
* Finish documenting runtime structure
* Fix trailing whitespace
* Add collatz conjecture example
* Include undocumented members in RTD
* Add a bunch of events
* Appropriately connect event publishers
* Generate useful testcases
* Blacken
* Attempt at 3.6 backporting
* Optimize AtomicStack
* Sort solve_n output in test_solve_n
Very unclear why this is happening, but it's breaking a travis build
https://travis-ci.com/trailofbits/manticore/jobs/246451527
* Roll back to trusty?
Unsure if this is what's causing the EVM issues
* Create custom trap types
* Lazily evaluate overflows
May need to revert this if it slows down the Travis tests. It's 0.7% slower in the example I'm looking at, but that may not carry over.
* Blacken
* Address Dominik's feedback, remove stale TODOs
* Expand & Test Examples
* Fix relative path
* Fix relative path?
* capitalize Path
* Fix relative path for real
* Blacken...
* Test finalize
* Add supplemental tests for coverage
* Use dedicated helper for store access
* Handle quoted strings in imported module names
* Allow WASMWorld to hold multiple modules
Necessary for getting imports/exports to work, but still needs some changes before it passes the tests.
* Actually use export maps
* Fix most of the control flow issues with import tests
* Use available cores for test generation
* Escape module name
* Don't reinit modules that have already been initialized
* Use addresses for imports instead of copying
* Fix removed timeout argument
* Interleave registration and aliases
* Fix broken "None" check for address
* Blacken
* Make manual exports work with address syntax
* Import all registered modules
Only necessary for elem, but it's tricky to make changes just for one test, so *shrug emoji*
* Remove stringified annotations
Turns out those only work in the same file...
* Fix easy type issues in all but runtime structure
* Fix easy type issues in runtime_structure
* Fix* everything but imports
* Blacken
* Blacken 2
* Typo
* Merge module structure and runtime structure
* Fix instance vs type confusion
* Fix mypy and concrete tests
Still an issue with the symbolic tests
* Still generate symbolic tests though
I keep forgetting not to commit that file...
* Attempt to fix env for symbolic tests
* Blacken/mypy
* Fix missing supplemental environment variables arg
* Fix docstrings for new import style
* CC
* Bump timeout duration to 70 minutes
I don't like it, but adding more jobs to Travis isn't going to make our builds any faster
* Fix pycharm type, import, and docstring errors
* Explain type: ignore
* Missed two type:ignore's
* Fix CLI so it doesn't throw up on arbitrary modules
Manticore
Manticore is a symbolic execution tool for analysis of smart contracts and binaries.
Note: Beginning with version 0.2.0, Python 3.6+ is required.
Features
- Input Generation: Manticore automatically generates inputs that trigger unique code paths
- Error Discovery: Manticore discovers bugs and produces inputs required to trigger them
- Execution Tracing: Manticore records an instruction-level trace of execution for each generated input
- Programmatic Interface: Manticore exposes programmatic access to its analysis engine via a Python API
Manticore can analyze the following types of programs:
- Ethereum smart contracts (EVM bytecode)
- Linux ELF binaries (x86, x86_64 and ARMv7)
Usage
CLI
Manticore has a command line interface which can be used to easily symbolically execute a supported program or smart contract. Analysis results will be placed into a new directory beginning with mcore_.
Use the CLI to explore possible states in Ethereum smart contracts. Manticore includes detectors that flag potentially vulnerable code in discovered states; output from them will be written to stdout and the results directory.
Solidity smart contracts must have a .sol extension for analysis by Manticore. See a demo.
$ manticore ./path/to/contract.sol # runs, and creates a mcore_* directory with analysis results
The command line can also be used to simply explore a Linux binary:
$ manticore ./path/to/binary # runs, and creates a mcore_* directory with analysis results
$ manticore ./path/to/binary ab cd # use concrete strings "ab", "cd" as program arguments
$ manticore ./path/to/binary ++ ++ # use two symbolic strings of length two as program arguments
API
Manticore has a Python programming interface which can be used to implement custom analyses.
For Ethereum smart contracts, it can be used for detailed verification of arbitrary contract properties. Set starting conditions, execute symbolic transactions, then review discovered states to ensure invariants for your contract hold.
from manticore.ethereum import ManticoreEVM
contract_src="""
contract Adder {
function incremented(uint value) public returns (uint){
if (value == 1)
revert();
return value + 1;
}
}
"""
m = ManticoreEVM()
user_account = m.create_account(balance=1000)
contract_account = m.solidity_create_contract(contract_src,
owner=user_account,
balance=0)
value = m.make_symbolic_value()
contract_account.incremented(value)
for state in m.ready_states:
print("can value be 1? {}".format(state.can_be_true(value == 1)))
print("can value be 200? {}".format(state.can_be_true(value == 200)))
It is also possible to use the API to create custom analysis tools for Linux binaries.
# example Manticore script
from manticore.native import Manticore
hook_pc = 0x400ca0
m = Manticore.linux('./path/to/binary')
@m.hook(hook_pc)
def hook(state):
cpu = state.cpu
print('eax', cpu.EAX)
print(cpu.read_int(cpu.ESP))
m.kill() # tell Manticore to stop
m.run()
Requirements
- Manticore is supported on Linux and requires Python 3.6+.
- Ubuntu 18.04 is strongly recommended.
- Ethereum smart contract analysis requires the
solcprogram in your$PATH. - Increased stack size is recommended; this can be done by
ulimit -s 100000or by passing--ulimit stack=100000000:100000000todocker runif docker is used.
Quickstart
Install and try Manticore in a few shell commands:
# Install system dependencies
sudo apt-get update && sudo apt-get install python3 python3-pip -y
# Install Manticore and its dependencies
sudo pip3 install manticore[native]
# Download the examples
git clone https://github.com/trailofbits/manticore.git && cd manticore/examples/linux
# Build the examples
make
# Use the Manticore CLI
manticore basic
cat mcore_*/*0.stdin | ./basic
cat mcore_*/*1.stdin | ./basic
# Use the Manticore API
cd ../script
python3 count_instructions.py ../linux/helloworld
You can also use Docker to quickly install and try Manticore:
# Run container with a shared examples/ directory
# Note that `--rm` will make the container be deleted if you exit it
# (if you want to persist data from the container, use docker volumes)
# (we need to increase maximum stack size, so we use ulimit for that)
$ docker run --rm -it --ulimit stack=100000000:100000000 trailofbits/manticore bash
# Change to examples directory
manticore@8d456f662d0f:~$ cd manticore/examples/linux/
# Build the examples
manticore@8d456f662d0f:~/manticore/examples/linux$ make
# Use the Manticore CLI
manticore@8d456f662d0f:~/manticore/examples/linux$ manticore basic
manticore@8d456f662d0f:~/manticore/examples/linux$ cat mcore_*/*0.stdin | ./basic
manticore@8d456f662d0f:~/manticore/examples/linux$ cat mcore_*/*1.stdin | ./basic
# Use the Manticore API
manticore@8d456f662d0f:~/manticore/examples/linux$ cd ../script
manticore@8d456f662d0f:~/manticore/examples/script$ python3 count_instructions.py ../linux/helloworld
Installation
NOTE: For native binary analysis, Manticore requires additional dependencies that are not installed by default. To install these also, substitute
manticore[native]formanticorein anypipcommand.
Option 1: Perform a user install (requires ~/.local/bin in your PATH).
echo "PATH=\$PATH:~/.local/bin" >> ~/.profile
source ~/.profile
pip3 install --user manticore
Option 2: Use a virtual environment (requires virtualenvwrapper or similar).
sudo pip3 install virtualenvwrapper
echo "source /usr/local/bin/virtualenvwrapper.sh" >> ~/.profile
source ~/.profile
mkvirtualenv manticore
sudo ./manticore/bin/pip3 install manticore
Option 3: Perform a system install.
sudo pip3 install manticore
Option 4: Install via Docker.
docker pull trailofbits/manticore
Once installed, the manticore CLI tool and Python API will be available.
For installing a development version of Manticore, see our wiki.
If you use Mac OS X you may need to install dependencies manually:
brew install capstone
export MACOS_UNIVERSAL=no && pip install capstone
brew install unicorn
UNICORN_QEMU_FLAGS="--python=`whereis python`" pip install unicorn
Solidity Versions
Note that we're still in the process of implementing full support for the EVM Constantinople instruction semantics, so certain opcodes may not be supported.
You may want to consider using a version of solc that's less likely to generate these opcodes (eg pre-0.5.0).
Getting Help
Feel free to stop by our #manticore slack channel in Empire Hacking for help using or extending Manticore.
Documentation is available in several places:
-
The wiki contains some basic information about getting started with Manticore and contributing
-
The examples directory has some very minimal examples that showcase API features
-
The API reference has more thorough and in-depth documentation on our API
-
The manticore-examples repository has some more involved examples, for instance solving real CTF problems
License
Manticore is licensed and distributed under the AGPLv3 license. Contact us if you're looking for an exception to the terms.
