The WASM branch is ready to be included in the rest of Manticore. It doesn't include several advanced features, including a symbolic memory model, symbolic floating point support, or a standard library implementation (WASI/emscripten), but it does have full symbolic semantics for all of the integer instructions.
### Commit log:
* Empty WASM Module
Will run on a .wasm file and do nothing forever
* Read and parse sections
/shrug
* Can partially allocate modules
Just pushing for the weekend
* Blacken
* Can allocate with exports now
* Static initialization
Doesn't evaluate any of the starting expressions, but that's okay
* Make type hints a little more pycharm-friendly
* First part of invocation
* Blacken...
* Barebones execution poc
Can execute the following snippet:
```
main(){
return 42;
}
```
* blacken
* Add some loads and stores
* Add global operations
* Move float methods further down
* Support internal function calls
```
18:20:42 (mc) ehennenfent@nessie:~/wasm ➜ cat loop_check.c
#include <stdio.h>
#include <stdbool.h>
bool check(int arg){
if (((arg << 2) ^ 16) == 36) {
return true;
}
else{
return false;
}
}
int main(){
printf("Hello world!\n");
int g = getchar();
printf("Got: %d\n", g);
int y = 0;
for (int i = 0; i < 20; i++){
y++;
if (y > 18){
return check(g) ? 0 : -1;
}
}
return -1;
}
18:20:54 (mc) ehennenfent@nessie:~/wasm ➜ manticore loop_check.wasm
65: i32.const (16)
11: end
65: i32.const (32)
11: end
Initialization Complete
65: i32.const (0)
65: i32.const (0)
40: i32.load (Offset 4)
65: i32.const (16)
107: i32.sub
34: tee_local (Local 1)
54: i32.store (Offset 4)
65: i32.const (32)
16: call (Func Idx 2)
Called stub function with args: (32,)
HostFunc returned [13]
26: drop
32: get_local (Local 1)
16: call (Func Idx 0)
Called stub function with args: ()
HostFunc returned [13]
34: tee_local (Local 0)
54: i32.store (Offset 0)
65: i32.const (16)
32: get_local (Local 1)
16: call (Func Idx 1)
Called stub function with args: (16, 16)
HostFunc returned [13]
26: drop
65: i32.const (0)
32: get_local (Local 1)
65: i32.const (16)
106: i32.add
54: i32.store (Offset 4)
65: i32.const (-1)
65: i32.const (0)
32: get_local (Local 0)
65: i32.const (1073741823)
113: i32.and
65: i32.const (13)
71: i32.ne
27: select
11: end
[0]
2019-07-25 18:20:58,138: [13842] m.c.manticore:INFO: Generated testcase No. 0 - test
2019-07-25 18:20:58,143: [13842] m.c.manticore:INFO: Results in /home/ehennenfent/wasm/mcore_1evptpip
```
* Improve pickle-friendliness
* Convert wasm module instructions into internal type s
* Fix class naming conflict
Thanks Pickle
* Support bitvecs as an integer type
* Refactor execution to handle single instructions at a time
* Add Atomic Stacks
It's a very inefficient implementation, but we'll make it better
* Handle Concretize Exceptions
Needs a lot of fine tuning but it's a start
* Propagate symbols all the way to return
* Add test generation infrastructure
Still very basic but it's a start
* Attempt to make Travis happy
* Fix missing generation step
* Ignore stack exhaustsion and invalid modules
* Add basic shift instructions
* Add lots of arithmetic instructions
* Fix global initialization
* Add optimization for fixed size left shift
* Make Travis tests less verbose
* Blacken
* Properly handle modulo in shifting
* Add several control flow instructions
* Blacken
* Partially Fix return instruction
* Fix missing return value from look_forward
* Update look_forward to handle nested blocks
* Fix type sigs and executor copy/paste errors, deduplicate test names
* Fix an instruction exit issue
* Fix local ordering
* Blacken
* test_address now passes for wasm
Fixed issue with json2mc.py issue with testname uniqueness
Started some work on floating point
* i hate git
* i hate git
* Adding linenumber to testcase function name
minor float progress
* f32_cmp passed for wasm
* f64_cmp passes for wasm
* f32_bitwise passes except for the problem with assertEqual not working with nan
* f64_bitwise passes except for the problem with assertEqual not working with nan
* Fixed the tests to handle [nan] comparisons
* better handling of nans and infinites
* added integer backing value for f64
* Blacken
* Blacken
* f64.nearest
* Fix argument ordering and implement N-bit memory stores
* Use subtests
* Switch print to logger
* Modify control flow instructions to match spec
"Continuation" != the thing that goes inside the Label
* Teporarily disable call instructions to prevent infinite loop
Still need to figure out control flow for nested function calls
* Blacken
* Fixed infinite loop in _call via block depth tracking
The behavior of `end` needs to depend on how many labels and activations are on the stack. Every time we push or pop a label, we increment or decrement the counter for the current call frame.
* Blacken
* Initial `br` implementation
Probably still bugged
* Support if blocks without an else instruction
* Add call_indirect
* Add grow_memory
* i32_wrap_i64
* Add several arithmetic operators
* Copy i32.ctz impl from x86 TZCNT
* Fix off-by-one in br
I think this is the correct fix (it makes all the _br tests pass) but I'm leaving the TODO there just in case.
* Blacken...
* Fix lingering issue with if block splitting
* Fix bad args to sextend in i64_extend_s_i32
* Fix indirect call null check and Float binop return types
* Improve br_table indexing
See comment
* Remove vestigial argument popping in invoke
* Fix looking forward in _return and _block
* Blacken
* Generate trap and action tests
* Fix extraneous call to exec_for_test
* Fix missing traps in load instructions
* Fix c1=0 case in ctz/clz
Hopefully this also fixes the fact that the original CTZ implementations seems to have been intended for little-endian architectures
* Handle traps in-stride during test execution
* Treat zero division as trap
* Use ctypes to convert integers
* Blacken
* Apparently I was wrong about the CTZ endianness thing
* Better range checks for offsets
This really hammers home why checking whether ea + size is _greater than_ len(mem.data) is strange if mem.data is 0-indexed, but idk, that's what the spec says.
* Make stub functions return the correct number of results
* ctypes can't handle floats
* Add option to run the start function
* Correctly handle functions with unacceptable names
* Blacken
* Treat NotImplemented as a Skip
Also counts individual subtests directly instead of screwing around with trying to grep for them in the log.
* Fix unhandled negative indices
* Fixed some integer instruction bugs
i64/32:
div_s/u
shr_s/u
shl
clz
* Fix global and memory imports
* CC
* Make default loglevel debug
* Rework conversion from int
* Make float_store work
* _actually_ fix float_store
* Restore old float creation method
* Blacken
* Add range checks to float load/store
* Force cast to unsigned in _u instructions
* Fix yet another I32 -> I64 copy paste error
How many times do we have to teach you this lesson, ~old~ young man?
* Skip problematic float tests for now
* Handle floating point inaccuracies when dividing large ints
* Skip conversion and endianness tests
* Rotl, rotl, popcnt
* Blacken
* Add some conversion stubs
* Maybe fix conversions
* Improve reinterpret and float demotion
* Re-enable endianness tests
* Include WASM logging in verbosity controls
* Propagate constraints to executor
* Blacken
* Add prototype symbolic tests
* Add api for invocation to Manticore object
* Temporarily disable native/evm tests
There's no reason to tie up Travis for 40 minutes every time I push a commit.
* Fix wasmworld import
* Concretize br_if
* Pass arg generator to invoke
* Download updated WABT
It's fixed now!
* Generate Symbolic tests (partial)
* Follow m.run format
* Blacken
* Reinitialize Manticore every 50 tests
This should cut down on the performance issues we've been seeing
* Revert "Temporarily disable native/evm tests"
This reverts commit 7abd6c1333.
* I32 and I64 pass Symbolic
* Create new MC on reinit
* Blacken
* Fix return handling in test generation
* Concretize operand in `if_`
* Concretize memory addresses before access
In the long run, we should have a symbolic memory model
* Better memory access concretization (and br_table)
* Concretize float converions and indirect call operands
* Run start method
* Handle size errors in wrap+i64 and select
* Add timeout (and skip extra-problematic tests)
* Blacken
* Aggressively concretize floats
* Recast ints that should have been floats
* Better trap handling
* Unstash float concretization
* Fix log-lived 'trap' attr
* Blacken
* Concretize F32 when promoting to F64
_shouldn't_ strictly be necesssary, but the lazy conversion means that one of the tests fails. Until we get symbolic floating point, this will have to do.
* Docstrings for platforms/wasm.py
* Docstrings for ManticoreWASM
* Add docs for the module structure
* Add WASM to RTD
* missing docstring for Module itself
* Fix sphinx errors
* Bump RTD python version
I'm sorry and I swear I'll fix it later
* Reformat docstrings for sphinx
* Pacify RTD by adding newlines before params
* Missed a few
* Document the types
* Document executor
* A few more type docstrings
* Partial runtime_structure docs
* Finish documenting runtime structure
* Fix trailing whitespace
* Add collatz conjecture example
* Include undocumented members in RTD
* Add a bunch of events
* Appropriately connect event publishers
* Generate useful testcases
* Blacken
* Attempt at 3.6 backporting
* Optimize AtomicStack
* Sort solve_n output in test_solve_n
Very unclear why this is happening, but it's breaking a travis build
https://travis-ci.com/trailofbits/manticore/jobs/246451527
* Roll back to trusty?
Unsure if this is what's causing the EVM issues
* Create custom trap types
* Lazily evaluate overflows
May need to revert this if it slows down the Travis tests. It's 0.7% slower in the example I'm looking at, but that may not carry over.
* Blacken
* Address Dominik's feedback, remove stale TODOs
* Expand & Test Examples
* Fix relative path
* Fix relative path?
* capitalize Path
* Fix relative path for real
* Blacken...
* Test finalize
* Add supplemental tests for coverage
* Use dedicated helper for store access
* Handle quoted strings in imported module names
* Allow WASMWorld to hold multiple modules
Necessary for getting imports/exports to work, but still needs some changes before it passes the tests.
* Actually use export maps
* Fix most of the control flow issues with import tests
* Use available cores for test generation
* Escape module name
* Don't reinit modules that have already been initialized
* Use addresses for imports instead of copying
* Fix removed timeout argument
* Interleave registration and aliases
* Fix broken "None" check for address
* Blacken
* Make manual exports work with address syntax
* Import all registered modules
Only necessary for elem, but it's tricky to make changes just for one test, so *shrug emoji*
* Remove stringified annotations
Turns out those only work in the same file...
* Fix easy type issues in all but runtime structure
* Fix easy type issues in runtime_structure
* Fix* everything but imports
* Blacken
* Blacken 2
* Typo
* Merge module structure and runtime structure
* Fix instance vs type confusion
* Fix mypy and concrete tests
Still an issue with the symbolic tests
* Still generate symbolic tests though
I keep forgetting not to commit that file...
* Attempt to fix env for symbolic tests
* Blacken/mypy
* Fix missing supplemental environment variables arg
* Fix docstrings for new import style
* CC
* Bump timeout duration to 70 minutes
I don't like it, but adding more jobs to Travis isn't going to make our builds any faster
* Fix pycharm type, import, and docstring errors
* Explain type: ignore
* Missed two type:ignore's
* Fix CLI so it doesn't throw up on arbitrary modules
* linux: Add return 0 for success on sys_pipe2
* mypy: Initial config file to ignore modules without typing info
* mypy: types manticore/platforms/linux.py
* mypy: manticore/ethereum/manticore.py
* Ignore error because mypy doesn't support decorated @property's
* mypy: manticore/ethereum/solidity.py
* Ignore error because mypy doesn't support decorated @property's
* mypy: manticore/ethereum/detectors.py
* Use Python's Enum class for DetectorClassification
* mypy: manticore/platforms/cgcrandom.py
* Set as byte string
* mypy: manticore/binary/binary.py
* mypy: manticore/utils/event.py
* mypy: manticore/ethereum/abitypes
* Use actual ResourceWarning class instead of string name
* mypy: manticore/utils/config.py
* mypy: manticore/core/workspace.py
* Support both Python 3.6 and Python 3.7 with nullcontext. Ignore the
mypy error because it checks against Python 3.6 and Manticore does not
explicitly claim support for Python 3.7
* mypy: manticore/core/manticore
* Ignore mypy error on sync, at_running, at_not_running:
Self argument missing for a non-static method (or an invalid type for
self)
* Ignore type info for unsupported decorators on @property fields
* mypy: manticore/platforms/evm.py
* Ignore error of already defined variable when setting member variable
of class transaction
* Remove redundant get_nonce call
* Move property setter next to property definition
* mypy: manticore/core/smtlib/solver.py
* Fix instance where no match was found in regex
* mypy: Add mypy to travis and setup.py
* Lock to specific mypy and black tool versions
* Rename "format" stage to "lint"
* Add text about mypy to CONTRIBUTING.md
* Revert black tool upgrade to 19.3b0
* Fix typing issues introduced by SHA3 branch
* Add black to CI tests
I'd rather have blackening be an optional check, but this will work for now
* Yep, messed up Travis
Again, sorry about the commit spam
* Bump accepted line length to 120
Going to auto-format at 100 but will allow 20% overruns
* Blacken
* Cap line length at 100
rip my undisciplined coding habits
* update black on the fly if necessary
* Ignore CC errors
* Only check modified files
* Only python files
* Re-blacken post merge
* Re-Blacken
* Fix deleted files
* Explicitly invoke python
The +x must not have gotten commited
* Forgot to blacken again
Hoisted on my own petard
* Re-blacken
* Add assertions to auto test gen
* Add symbolic tests
* Make calldata symbolic
* EVM: Support exp aka pow (#1361)
* EVM: Add support for EXP with concrete/solvable exponent
Adds support for EXP aka POW, with concrete or solvable exponent, fixing https://github.com/trailofbits/manticore/issues/1005 effectively.
Not sure if we can do it any better - it seems z3 doesn't have exp/pow for bitvectors.
* Update operators.py
* Update evm.py
* Use concretized_args
* Move Operators.POW to EVM._exp
* Extend travis wait for output to 30m
* Extend travis build to 60m...
* Fix Operators.ITE -> Operators.ITEBV
* Split ethereum travis job to two jobs
* EVM.EXP: concretize base=SAMPLED
* Fix concrete tests: use to_constant
* Fix set storage in concrete tests
* Split ethereum_vm tests into concrete and symbolic
* Fix travis tests
* Split symbolic tests into two jobs
* Split VMTests even more
* More tests split
* [WIP][WIP][WIP] Moving executor functionality to ManticoreBase and refactor concurrency shared data
* Workspace locks
* Concurrency flavor configurable from commandline
* Asserts and refactorrrrrrrs
* Remove unused callback
* Some CC
* Some CC
* Some CC
* Fix solver vs Z3Solver
* Make solver a singleton based on tid/pid. REfactor m._save. Fix some tests
* typo and evm bugfix
* Fix some tests referecing global solver
* Fix concolic tests and more global solver refs
* Fix tests
* CC fixes
* Fix tests. Fix testcase id generation
* Move profiling to a plugin and fix tests
* Add solver intance ref to mem test
* Fix mem workspace tests
* Fix output checking tests
* Fix z3solver ref
* Relax verbosity/log tests
* Moved Workers to its own file
* Relax output tests
* Relax output tests
* Fix profiling test
* Fix more tests
* Default multiprocessing
* Try to clean mcore __del__
* Change Worker life span
* Fix Single mode
* CC
* revert verbosity travis
* CC and solver ref fix
* Relax ouput checking tests and some bugfixes
* running -> ready
* Fixing teeests
* add weak cache to _load
* del debug prints
* Adding config.py support for Enums
* Try/Remove generate_testcase event as it never occurs online. Fix tests
* Fix CC
* Kill the cache when start/stop run. Remove debugprints. clean tests
* Fix travis test _other_
* Fix native tests and timeout
* Fix state.must_be_true
* fix CC
* Changing fstat tets...:
~'
* LLLLLLLLinux tests
* Skip unicorn concrete test for now
* Try fix CodeClimate
* Try fix CodeClimate
* Update evm examples to newest solidity
* Complete transformation of consts.mprocessing to enum
* Add blank line (codeclimate)
* Using the enum instead of the string
* Using the enum instead of the string
* Merge and fix
* CC and debug print
* Move fee consumption to checkpoint so it is not done twice. Fix frontier test generator
* Fix Job Count (and force travis rebuild)
```
0.02s$ ./cc-test-reporter sum-coverage --output - --parts $JOB_COUNT coverage/codeclimate.*.json | ./cc-test-reporter upload-coverage --input -
Error: expected 3 parts, received 4 parts
```
* Add tx number to testcase log
* Attempt to do vmtests on the fly
* Del test verbosity
* Update .travis.yml
Co-Authored-By: Eric Hennenfent <[email protected]>
As noted, we should roll this back whenever Unicorn pushes to PyPi
* Remove write to Unicorn x86 FS register
In latest version of unicorn (778171fc9546c1fc3d1341ff1151eab379848ea0)
a write to FS will cause segfault or hang if we try to write to it.
Issue #1434
* Update unicorn dependency to latest commit on master
Latest Unicorn master commit is 778171fc9546c1fc3d1341ff1151eab379848ea0
Issue #1434
* Add assertions to auto test gen
* Add symbolic tests
* Make calldata symbolic
* EVM: Support exp aka pow (#1361)
* EVM: Add support for EXP with concrete/solvable exponent
Adds support for EXP aka POW, with concrete or solvable exponent, fixing https://github.com/trailofbits/manticore/issues/1005 effectively.
Not sure if we can do it any better - it seems z3 doesn't have exp/pow for bitvectors.
* Update operators.py
* Update evm.py
* Use concretized_args
* Move Operators.POW to EVM._exp
* Extend travis wait for output to 30m
* Extend travis build to 60m...
* Fix Operators.ITE -> Operators.ITEBV
* Split ethereum travis job to two jobs
* EVM.EXP: concretize base=SAMPLED
* Fix concrete tests: use to_constant
* Fix set storage in concrete tests
* Split ethereum_vm tests into concrete and symbolic
* Fix travis tests
* Split symbolic tests into two jobs
* Split VMTests even more
* More tests split
* [WIP][WIP][WIP] Moving executor functionality to ManticoreBase and refactor concurrency shared data
* Workspace locks
* Concurrency flavor configurable from commandline
* Asserts and refactorrrrrrrs
* Remove unused callback
* Some CC
* Some CC
* Some CC
* Fix solver vs Z3Solver
* Make solver a singleton based on tid/pid. REfactor m._save. Fix some tests
* typo and evm bugfix
* Fix some tests referecing global solver
* Fix concolic tests and more global solver refs
* Fix tests
* CC fixes
* Fix tests. Fix testcase id generation
* Move profiling to a plugin and fix tests
* Add solver intance ref to mem test
* Fix mem workspace tests
* Fix output checking tests
* Fix z3solver ref
* Relax verbosity/log tests
* Moved Workers to its own file
* Relax output tests
* Relax output tests
* Fix profiling test
* Fix more tests
* Default multiprocessing
* Try to clean mcore __del__
* Change Worker life span
* Fix Single mode
* CC
* revert verbosity travis
* CC and solver ref fix
* Relax ouput checking tests and some bugfixes
* running -> ready
* Fixing teeests
* add weak cache to _load
* del debug prints
* Adding config.py support for Enums
* Try/Remove generate_testcase event as it never occurs online. Fix tests
* Fix CC
* Kill the cache when start/stop run. Remove debugprints. clean tests
* Fix travis test _other_
* Fix native tests and timeout
* Fix state.must_be_true
* fix CC
* Changing fstat tets...:
~'
* LLLLLLLLinux tests
* Skip unicorn concrete test for now
* Try fix CodeClimate
* Try fix CodeClimate
* Update evm examples to newest solidity
* Complete transformation of consts.mprocessing to enum
* Add blank line (codeclimate)
* Using the enum instead of the string
* Using the enum instead of the string
* Merge and fix
* CC and debug print
* Move fee consumption to checkpoint so it is not done twice. Fix frontier test generator
* Fix Job Count (and force travis rebuild)
```
0.02s$ ./cc-test-reporter sum-coverage --output - --parts $JOB_COUNT coverage/codeclimate.*.json | ./cc-test-reporter upload-coverage --input -
Error: expected 3 parts, received 4 parts
```
* Add tx number to testcase log
* Del test verbosity
* Move unimplemented syscall logging to decorator
Adds [wrapt module](https://wrapt.readthedocs.io/en/latest/) from pypi
* Move unimplemented calls further down
* Add stubs for unimplemented system calls
* Move most stubs back to stub file
A few have been left in linux.py because they're necessary for other projects
* Codeclimate
* Fix discrepancies in syscall ABI
* Removed extraneous void argument for getpid and gettid
* Added missing sigsetsize parameter for rt_sigaction
* Removed incorrect 'flags' parameter on accept (should be on accept4
* Fixed 'SocketDesc' has no attribute 'close' error
* (Partially) Fixed duplicate constraint error with socket file descriptors
* Add gettimeofday and directory handling
* Hacky pipe implementation
Pretend that sockets are pipes and pipes are sockets
* Better support for MSR writing
* Default to success on directory operations
* Codeclimate
* More verbose unimplemented syscall printing
* Fix line overrun before CC calls me out
* Add ftruncate support
* Add link, unlink, and nanosleep
* Add getdents
* Add tests for mkdir and time functions
* Fix codeclimate and force rebuild
* Add pre and post hooks for syscalls
* Fix lingering single quotes
* Handle dumb invocations
* Type-o
* Make emulator follow logging behavior
* CC
* Fix pipe tests
* Fix shadowed name
* Add test for ftruncate
* Add link tests
* Add chmod test
* Rewrite docstrings
Best not to risk licensing issues
* Replace with simple returns
* Strip out unique implementations
Should bring the line count down a bit
* Check that unimplemented syscalls exhibit the proper behavior
* Fix missing keyword 'solver'
* Pass initial states to will_run callback
* Restore debug printouts after run
* Fix existing `init` calls
* Restore old exception behavior
* Move save_run_data to manticore native
* CC
* Refactor finalization call
* Fix save_run_data
I think it makes sense now
* Adjust expected output line counts
* CC [ci skip]
* Rename terminate state on exception to kill state
There's gotta be a better pun about 'overthrowing the state' in there somewhere
* Move time started to manticore_native
* [WIP][WIP][WIP] Moving executor functionality to ManticoreBase and refactor concurrency shared data
* Workspace locks
* Concurrency flavor configurable from commandline
* Asserts and refactorrrrrrrs
* Remove unused callback
* Some CC
* Some CC
* Some CC
* Fix solver vs Z3Solver
* Make solver a singleton based on tid/pid. REfactor m._save. Fix some tests
* typo and evm bugfix
* Fix some tests referecing global solver
* Fix concolic tests and more global solver refs
* Fix tests
* CC fixes
* Fix tests. Fix testcase id generation
* Move profiling to a plugin and fix tests
* Add solver intance ref to mem test
* Fix mem workspace tests
* Fix output checking tests
* Fix z3solver ref
* Relax verbosity/log tests
* Moved Workers to its own file
* Relax output tests
* Relax output tests
* Fix profiling test
* merge
* Fix more tests
* Default multiprocessing
* Try to clean mcore __del__
* Change Worker life span
* Fix Single mode
* CC
* Merge branch 'master' into dev-executor-refactor
* revert verbosity travis
* CC and solver ref fix
* Relax ouput checking tests and some bugfixes
* running -> ready
* Fixing teeests
* add weak cache to _load
* del debug prints
* Adding config.py support for Enums
* Try/Remove generate_testcase event as it never occurs online. Fix tests
* Fix CC
* Kill the cache when start/stop run. Remove debugprints. clean tests
* Fix travis test _other_
* Fix native tests and timeout
* Fix state.must_be_true
* fix CC
* Merge and fix more tests
* Changing fstat tets...:
~'
* LLLLLLLLinux tests
* Skip unicorn concrete test for now
* Try fix CodeClimate
* Try fix CodeClimate
* Update evm examples to newest solidity
* Complete transformation of consts.mprocessing to enum
* Add blank line (codeclimate)
* Using the enum instead of the string
* Merge branch 'dev-executor-refactor' of github.com:trailofbits/manticore into dev-executor-refactor
* Using the enum instead of the string
* Merge branch 'master' into dev-executor-refactor
* Fix solver instance reference
* aarch64: add everything except instructions
Based on disconnect3d/aarch64/skeleton.
* fixup: do not use relative imports
Otherwise, 'scripts/travis_test.sh' fails to work.
* fixup: remove x86-specific code from '_reg_name'
* review: rename the file
* review: avoid wildcard imports
* Revert "fixup: remove x86-specific code from '_reg_name'"
* fixup: support ARM64 in '_reg_name'
* WIP
* Add test
* Add missing test data file
* Fix missing testcase msg and assert it in test
* Remove Executor.generate_testcase - just publish event
* Remove redundant import
* Add ManticoreBase._last_run_stats
* Review fixes
* Remove redundant assertions in test_basic_arm
* Elapsed time calc fix
* New auto evm test generator [WIP]
* New auto test generator for EVM
* New VMTest mcore translator
* Allow to auto-make single tests
* New concrete auto test for vmTests snad tests
* MAke bytecode bytes or array in tests too
* Make bytecode bytes or array in tests too more
* initial timestamp fix
* Fix jmpdest gas test and evm
* New tests style and some fixes
* __init__ in test folders
* Fix VMTest to unitests. Fix Array/ArraySlice default values
* Added rlp to setup.py
* Remove old deprecated tests
* Debugging storage/memory default values
* avoid gas consumption of empty reads
* CC
* Fix .written list for sliced arrays
* Move blocknumber/timestamp out of evmworld
* CC
* Several gas related fix. Frontier concrete pass
* Fix ArrayProxy tests
* Debug travis
* Split the regresion test so it prints . (dots)
* If value is in the cache then it is a known index. optimization
* add main to some unittest
* FIx test maker
* CC and skip slow test
* Skipp other slow test for now
* initial_timestamp -> timestamp
* more skipping
* Fix/mitigate expression copying issue
* Try fix concolic
* Fix simplification so it does not loose taint
* Fix no-testcases and verbosity in tests
* Extra visitor tests
* Extra visitor tests
* Better array slicing test. Code review. Basic fork support. VMTest concrete added
* bump pyevmasm version
* Remove pyevmasm version pinning
* Pyevmasm from github for now
* Setup.py crazyness
* Pyevmasm version bump - again
* Update manticore/platforms/evm.py
Co-Authored-By: feliam <[email protected]>
* Update manticore/platforms/evm.py
Co-Authored-By: feliam <[email protected]>
* Update manticore/platforms/evm.py
Co-Authored-By: feliam <[email protected]>
* Update manticore/platforms/evm.py
Co-Authored-By: feliam <[email protected]>
* Update manticore/platforms/evm.py
Co-Authored-By: feliam <[email protected]>
* Remove cache oddity and fix auto tests
* Add some type checks
* A fix
* Prevent ResourceWarnings in Z3Solver._stop_proc
* Wrap temporary uses of open in with statement
* Close program file in platforms.linux.Linux destructor
* Close files opened in tests
This commit also ensures that the tempdir created in _create_openat_state in test_linux.py is deleted at the end of the test.
* Fix binary file open modes
* Enable ResourceWarnings in Travis build
* Fix codeclimate issue: bad indent
* Suppress harmless ResourceWarning emitted during the initial abitypes parser table generation
* Convert format strings to literal string interpolation
This commit converts most (only a couple of exceptions) of the formatted strings in Manticore to follow PEP-0498, literal string interpolation.
* Converted most format strings to f-strings, except for some in tests
* Fixing codeclimate issues
* Updated the new format strings that appeared after merging, fixed a flipped set of parameters
* Fixed another incorrect quote typo
* fixed extra quote typo
* DAO detector + bugfixes
* The actual benchmark tests
* The actual benchmark tests
* CC
* Experiment reporting the finding at a JUMPI
* Fix taint. Detect returned overflowded data
* DAO -> Reentrancy
* DAO -> reentrancy, C -> Benchmark
* DAO -> reentrancy, C -> Benchmark
* Allow function names to have numbers
* Fix contract names in benchmark
* Fix contract names in benchmark
* Move default plugin registration
* Better regexp
* Fix minimal_bytecode example
* Fix Array Slice and test
* add tests
* correct other bug
* implement bytesM
* BROKEN partial progress
* need bytearray here
* rm cmt
* add basic tests for bytesM and bytes symbolic
* correct bytes symbolic test
* Refactor, clean bytesM handling
* Add initial symbolic 'bytes' handling
* refactor tests
* Unify symbolic/concrete bytes handling in bytesM/bytes
* Rm import
* Rm debug assert
* cc
* Visitor/migrate/simplify fixes to make the seth refactor pass
* Fix concolic?
* Fix concolic?
* CC
* bytesM fix
* Fix address and caller concretization on symb tx
* Fix/refactor symbolic address/caller concretization
* Fix caller concretization
* Fix expression visiting
* Fix account policy refactor
* Accept numbers in function names abitypes
* Simplify installation instructions to recommend install manticore only for the current user
* Run some tests in parallel (#970)
This PR splits the current test runner into three environments:
1. Linux examples
2. Ethereum tests
3. Remaining tests
to faster complete each testing run. Ethereum tests include a number of integration tests that execute scripts to completion, which takes a while. We run them concurrently with other tests to save on execution time. The split is done by naming Ethereum tests differently (`eth_*.py` vs `test_*.py`) and updating what pattern unittest's `discover` uses.
This change also updates the installation script and chooses to forego installing Keystone for EVM tests as it takes a while, and it adds a `setup.cfg` config file so that Nose finds the eth tests as well by default.
* Be less verbose when testing
* Fix slicing wrongly reference to proxyArray. Fix#912
* Only export human/external tx in the testcase (#972)
* Make ManticoreEVM.make_symbolic_value size adjustable (#974)
* Make size adjustable
* Default to 256
* Dev evm yolo fix gas (#975)
* Fix gas stipend on CALL and check dao
* Add order dependence 1
* Going linter. Report/Detect that thing when code does not check returned value
* cleaner example of fail
* Update retval_crazy.sol
* new solc for travis
* CC
* Remove duplicated ReentrancyDetector
* POrt to py3
* POrt to py3
* P0rt to py3
* CC
* Tests doc
* CC
This PR enables the reporting of test coverage of all the test jobs (`eth` and `tests`) to CodeClimate. This uses S3 to temporarily store results between jobs and later upload them to CC.
Fixes#1000
* DAO detector + bugfixes
* The actual benchmark tests
* The actual benchmark tests
* CC
* Experiment reporting the finding at a JUMPI
* Fix taint. Detect returned overflowded data
* DAO -> Reentrancy
* DAO -> reentrancy, C -> Benchmark
* DAO -> reentrancy, C -> Benchmark
* Allow function names to have numbers
* Fix contract names in benchmark
* Fix contract names in benchmark
* Move default plugin registration
* Better regexp
* Fix minimal_bytecode example
* Fix Array Slice and test
* add tests
* correct other bug
* implement bytesM
* BROKEN partial progress
* need bytearray here
* rm cmt
* add basic tests for bytesM and bytes symbolic
* correct bytes symbolic test
* Refactor, clean bytesM handling
* Add initial symbolic 'bytes' handling
* refactor tests
* Unify symbolic/concrete bytes handling in bytesM/bytes
* Rm import
* Rm debug assert
* cc
* Visitor/migrate/simplify fixes to make the seth refactor pass
* Fix concolic?
* Fix concolic?
* CC
* bytesM fix
* Fix address and caller concretization on symb tx
* Fix/refactor symbolic address/caller concretization
* Fix caller concretization
* Fix expression visiting
* Fix account policy refactor
* Accept numbers in function names abitypes
* Simplify installation instructions to recommend install manticore only for the current user
* Run some tests in parallel (#970)
This PR splits the current test runner into three environments:
1. Linux examples
2. Ethereum tests
3. Remaining tests
to faster complete each testing run. Ethereum tests include a number of integration tests that execute scripts to completion, which takes a while. We run them concurrently with other tests to save on execution time. The split is done by naming Ethereum tests differently (`eth_*.py` vs `test_*.py`) and updating what pattern unittest's `discover` uses.
This change also updates the installation script and chooses to forego installing Keystone for EVM tests as it takes a while, and it adds a `setup.cfg` config file so that Nose finds the eth tests as well by default.
* Be less verbose when testing
* Fix slicing wrongly reference to proxyArray. Fix#912
* Only export human/external tx in the testcase (#972)
* Make ManticoreEVM.make_symbolic_value size adjustable (#974)
* Make size adjustable
* Default to 256
* Dev evm yolo fix gas (#975)
* Fix gas stipend on CALL and check dao
* Add order dependence 1
* missing files
* 985
* formatting fixes; codeclimate
* review changes
* incremental py3: automated conversion work
print statements, octal literals, and various small changes to support
identical calling between py2 and py3. Plus a few missed exception
syntax, etc.
* more print function imports, fix a syntax issue
* revert an import change for the moment
* different absolute import
* Update makefile; add a list target for testing
* simplify nostdlib example
* Make sendmail example return success
* Add tests to run all examples
* Add some targets to exclude
* Run example scripts; temporarily add a workspace accsesor to mcore
* Optionally read end of main from argv
* Make concolic test more robust
* Clean up Makefile
* Be better with phony targets
* Add run_simple and state_control tests
* verbosity++
* Make sure we fail when we intend to
* Simplify travis_test.sh
* Remove multi_arch_sym
* Move algorithm into ManticoreEVM
* Rm buggy unused line
* Initial eth testing setup
* Check for overflow in mul also
* clarifying comment
* Install solc for travis
* Add sudo
* x
* Rm travis_retry, since we rarely actually use it, and it's not available in this script for some reason
* Check for specific findings
* Fix test
* Clean up transaction firing logic
* Use less strict check, better for floating point
* WIP New Policy class
* WIP pubsub
* Update Signal tests
* small fixes from github comments
* Fix event decode_instruction signature
* Good merge
* Good good merge
* WIP manticore refactor
* Fix default old-style initial state
* add -> enqueue
* @m.init
* Fix workspace url
* Some test skipped
* Ad Fixme to platform specific stuff in State
* add -> enqueue
* Enqueue created state
* Fix m.init
Use a messy hack to adhere to the spec (callback func receive 1 state argument)
* Add _coverage_file ivar to Manticore
* Fix symbolic files
* remove extra enqueue
* Fixing __main__
* comments
* Experimental plugin system
* tests fixed
* Fix plugins
* Some reporting moved to plugin
* Fix assertions test
* Add published events to classes that publish them
* Update how we verify callbacks
* Update Eventful._publish
* Yet another flavor for event name checking
* really it's a bunch of minimal bugfixes
* Remove get_all_event_names from Plugin
* Update where we get all events
* Use new metaclass-based event registry
* Initial concrete trace follower
* Add extended (json) trace support
* More stubs for condition gather
* Update trace saving to new format
* Produce trace regardless of contents
* Record register deltas in trace
* Move initialization to _start_run so we can call run() multiple times
* Re-fix multiple workspace bug
* Fix it correctly
* Add extended trace and accessors
* make sure did_execute_instruction is always published
* Produce a consistent version
* move extended tracing to separate class
* Reorg of extended tracing plugins
* Add concolic tracing script
* Clean up old hooks
* Sync memory as well; cleanups
* Revert binary tests
* simplify concolic follower
* Move trace follower to plugin.py
* Simplify follower
* Add skip ranges to Follower
* Update concolic.py
* Remove redundant state init
* Clean up driver script
* Update header line
* Move trace follower
* Move Follower to follow_trace.py
* rm unused import
* Remove unnecessary property
* rm ConcreteTraceFollower
* Revert start_run behavior
* WIP New Policy class
* WIP pubsub
* Update Signal tests
* EVM support - Wip
* EVM support - dependencies fixed
* EVM support - fix decree merge
* fix decode instrucion event
* Fix small bugs in evm opcodes (too many arguments + wrong LOG name) (#380)
Fix wrong call parameters + typo
* Fix Create/Call
* Fix depth
* Default fixed point in arithmetic simplifier
* small fixes from github comments
* Fix event decode_instruction signature
* wip wip
* Auto tests for evm
* New EVM tests
* Ran 9556 FAILED (failures=166, errors=8, skipped=62)
* Fix some arithmetic instructions
* Ran 9556 FAILED (failures=136, errors=8, skipped=62)
* More instructions - Optimizing symbolic memory
* Added gas to opcodes description - FIX DELEGATECALL POPS
* Add wip wallet example
* The tests
* Solidity constructors need argument after bytecode
* Simple integer overflow working
* Good merge
* Good good merge
* WIP manticore refactor
* Fix default old-style initial state
* context now working
* Fix context serialization
* Fix test models. Can not set a state constraints
* typo
* A few typos (constraints setter) and use of public properties in internal methods
* Fix init wallet example
* State __init__ needs to initialize platform constraints
* Internal methods use internal properties
* Better attack modeling
* Better example layout
* Storage backup on CALL is now faster .. and correct
* Add LOG support
* Minimal SE test
* Added examples
* Send ether bugfix
* EVM: Fix wrong balance destination on CALL + decrease caller balance on CREATE
* New balance management
* Trying to maintain known hashes
* Known hash concretization policy
* CALLDATA max size bugfix
* Minimal SE example
* Remove evm tests
* add -> enqueue
* @m.init
* Fix workspace url
* Some test skipped
* Ad Fixme to platform specific stuff in State
* add -> enqueue
* Enqueue created state
* Fix m.init
Use a messy hack to adhere to the spec (callback func receive 1 state argument)
* Add _coverage_file ivar to Manticore
* Fix symbolic files
* remove extra enqueue
* Fixing __main__
* comments
* Fix visitors oddity
* setup merged
* remove duplicates and add pysha3
* Remove EVMTests import
* Refactor platform specific code out of ManticoreOutput (#505)
* Initial moving work
* Clean
* Make linux.generate_workspace_files work
* Fix
* clean
* Add test
* Test workspace for platform files
* Skip EVM cpu pretty print
* Remove bad import
* Fix coverage.py for testing
* Clean comment
* Comment hack
* Print evm cpu
* pretty print evm world instead of platform
* delet old scripts/examples
* delet old tests
* Remove z3 install script
* Array.max_size can be None, include check for that
* Rm unused _symbolic_files
add_symbolic_files was moved to linux, so this is not needed
* Rm unused args
* Import evm
* Rm dup function
* Rm stray prints
* Add docs for new classmethod apis
* minimal
* minimal example
* fix minimal
* Fair symbolic SHA3 handling
* Simple mapping example
* coverage example
* fix tests
* fix minimal
* Some eko fixes
* New SETH
* integer_overflow refactored
* Fixing the examples
* init_bytecode -> init
'
* Concrete reentrancy exampole
* concrete reentrancy selfdestruct
* Update minimal.py
* It's a new Minimal
* Integer overflow example
* New minimal
* minimal fix
* Examples last minute fixes
* Remove debug print
* add plugin.py
* Fixing event subscription
* remove temp params
* Remove param
* Update uncovered will_exec callback prototype
* Clean up debug output
* Automatically generated intruction tests
* Uninplemented instruction test removed
* Unused concretization policy removed
* Fixes enabling default bplugins
* solc from PATH
* Removed unused import
* Logger name updated
* plugin rearrangement and UI options
* Live update and clear options
* coverage stats
* added check for xref address
* coverage in separate option as BackgroundTaskThread
* license cleanup
Paul Kehrer