13 Commits
Author SHA1 Message Date
Eric Hennenfent 451965f03a Blacken Manticore (#1438)
* Add black to CI tests

I'd rather have blackening be an optional check, but this will work for now

* Yep, messed up Travis

Again, sorry about the commit spam

* Bump accepted line length to 120

Going to auto-format at 100 but will allow 20% overruns

* Blacken

* Cap line length at 100

rip my undisciplined coding habits

* update black on the fly if necessary

* Ignore CC errors

* Only check modified files

* Only python files

* Re-blacken post merge

* Re-Blacken

* Fix deleted files

* Explicitly invoke python

The +x must not have gotten commited

* Forgot to blacken again

Hoisted on my own petard

* Re-blacken
2019-05-23 15:19:24 -05:00
feliam a527e81830 ManticoreBase refactor (#1385)
* [WIP][WIP][WIP] Moving executor functionality to ManticoreBase and refactor concurrency shared data

* Workspace locks

* Concurrency flavor configurable from commandline

* Asserts and refactorrrrrrrs

* Remove unused callback

* Some CC

* Some CC

* Some CC

* Fix solver vs Z3Solver

* Make solver a singleton based on tid/pid. REfactor m._save. Fix some tests

* typo and evm bugfix

* Fix some tests referecing global solver

* Fix concolic tests and more global solver refs

* Fix tests

* CC fixes

* Fix tests. Fix testcase id generation

* Move profiling to a plugin and fix tests

* Add solver intance ref to mem test

* Fix mem workspace tests

* Fix output checking tests

* Fix z3solver ref

* Relax verbosity/log tests

* Moved Workers to its own file

* Relax output tests

* Relax output tests

* Fix profiling test

* merge

* Fix more tests

* Default multiprocessing

* Try to clean mcore __del__

* Change Worker life span

* Fix Single mode

* CC

* Merge branch 'master' into dev-executor-refactor

* revert verbosity travis

* CC and solver ref fix

* Relax ouput checking tests and some bugfixes

* running -> ready

* Fixing teeests

* add weak cache to _load

* del debug prints

* Adding config.py support for Enums

* Try/Remove generate_testcase event as it never occurs online. Fix tests

* Fix CC

* Kill the cache when start/stop run. Remove debugprints. clean tests

* Fix travis test _other_

* Fix native tests and timeout

* Fix state.must_be_true

* fix CC

* Merge and fix more tests

* Changing fstat tets...:
~'

* LLLLLLLLinux tests

* Skip unicorn concrete test for now

* Try fix CodeClimate

* Try fix CodeClimate

* Update evm examples to newest solidity

* Complete transformation of consts.mprocessing to enum

* Add blank line (codeclimate)

* Using the enum instead of the string

* Merge branch 'dev-executor-refactor' of github.com:trailofbits/manticore into dev-executor-refactor

* Using the enum instead of the string

* Merge branch 'master' into dev-executor-refactor

* Fix solver instance reference
2019-05-09 20:35:13 -03:00
Eric Hennenfent 3fc2faa588 Fix CodeClimate Master (#1408)
Fixes a few CC bugs that made it into master, and marks the "too many functions" issues that we always ignore as wontfix.

Travis is green so I'm just going to merge this.
2019-04-19 13:37:08 -05:00
Disconnect3d 10707f6d68 Fix #1380 - add parsetab to repo (#1383)
* Fix #1380 - add parsetab to repo

* Ignore parsetab in codeclimate

We don't need to  check autogenerated files since they shouldn't be modified by humans anyway

* Yaml paths are top-level
2019-03-06 20:48:43 +01:00
Yan Ivnitskiy 74d620f076 Implement FPSE-like memory model (#1127)
Implements am fpse-like memory model.
2018-12-03 16:38:44 -05:00
defunct 1f74f0fa84 fix sys_write logger output (#1024)
* fix sys_write logger output - resolves #1020
* write/writev/read fixes
* openat((int32)dirfd, ...) resolves #940, syscall logging
* disable E701, interferes with PEP484/526
2018-08-07 19:54:44 -04:00
defunct 4e73bc3b0c codeclimate - bump similar-code thresh; false positive 2018-07-31 13:44:03 +00:00
defunct b0e0a1b456 codeclimate 2018-07-31 13:42:28 +00:00
defunct e18016aa4a py3 yolo_master (feature / bugfixes) (#994)
* DAO detector + bugfixes

* The actual benchmark tests

* The actual benchmark tests

* CC

* Experiment reporting the finding at a JUMPI

* Fix taint. Detect returned overflowded data

* DAO -> Reentrancy

* DAO -> reentrancy, C -> Benchmark

* DAO -> reentrancy, C -> Benchmark

* Allow function names to have numbers

* Fix contract names in benchmark

* Fix contract names in benchmark

* Move default plugin registration

* Better regexp

* Fix minimal_bytecode example

* Fix Array Slice and test

* add tests

* correct other bug

* implement bytesM

* BROKEN partial progress

* need bytearray here

* rm cmt

* add basic tests for bytesM and bytes symbolic

* correct bytes symbolic test

* Refactor, clean bytesM handling

* Add initial symbolic 'bytes' handling

* refactor tests

* Unify symbolic/concrete bytes handling in bytesM/bytes

* Rm import

* Rm debug assert

* cc

* Visitor/migrate/simplify fixes to make the seth refactor pass

* Fix concolic?

* Fix concolic?

* CC

* bytesM fix

* Fix address and caller concretization on symb tx

* Fix/refactor symbolic address/caller concretization

* Fix caller concretization

* Fix expression visiting

* Fix account policy refactor

* Accept numbers in function names abitypes

* Simplify installation instructions to recommend install manticore only for the current user

* Run some tests in parallel (#970)

This PR splits the current test runner into three environments: 

1. Linux examples
2. Ethereum tests
3. Remaining tests

to faster complete each testing run. Ethereum tests include a number of integration tests that execute scripts to completion, which takes a while. We run them concurrently with other tests to save on execution time. The split is done by naming Ethereum tests differently (`eth_*.py` vs `test_*.py`) and updating what pattern unittest's `discover` uses.

This change also updates the installation script and chooses to forego installing Keystone for EVM tests as it takes a while, and it adds a `setup.cfg` config file so that Nose finds the eth tests as well by default.

* Be less verbose when testing

* Fix slicing wrongly reference to proxyArray. Fix #912

* Only export human/external tx in the testcase (#972)

* Make ManticoreEVM.make_symbolic_value size adjustable (#974)

* Make size adjustable

* Default to 256

* Dev evm yolo fix gas (#975)

* Fix gas stipend on CALL and check dao

* Add order dependence 1

* missing files

* 985

* formatting fixes; codeclimate

* review changes
2018-07-27 14:12:09 -04:00
defunct 664e53bff0 Python 3 (#968)
* Initial commit for Python3 branch that targets porting the following;
* Core manticore functionality
* x86/x64/arm architectures
* linux platform
* functioning unit tests for the above

* WIP ethereum support
notes:
* eth unit tests passing
* decree disabled
* ManticoreEVM.__init__ procs=1
* no iterpickle; see eth tests for setting stack size/recursion

* EVMAccount;
  add __hash__
  remove redefinition of __eq__

port/update EVM examples

* update stacksize / recursion limit

* pyevmasm dep updates, port linux examples

* update docs, tox/travis configs, port missing x86 unit test

* disable decree tests

* update example, update pyevmasm dep_links

* disable testArgumentsAssertions to speed up tests

* py35/36 json.loads compat fix

* fix summary output encoding

* disable eth binary test 808

* PEP479 - Python 3.7 support

* travisci; fix code coverage reporting

* port decree and tests

* codeclimate

* review changes;
	* cleanup
	* reset version
	* remove EVMAccount.__hash__, fix EVMWorld.get_balance()

* switch to official pyevmasm

* z3 version check - py3.5 fix

* pyevmasm; remove version requirement

* review changes;
	* remove redundant locking
	* shutdown executor manager on deletion

* review changes;
  * cleanup PY3FIX comments
  * _run_solc version ascii to utf encoding
  * save_summary - use filter instead of comprehension
  * comments

* __get_related - set literal mistake

* review changes;
  * remove redundant []
  * cleanup unused code, params, vars, imports
  * additional comments

* review changes;
  * fix symbolic const instruction decoding and add unit test
  * misc cleanup

* fix verbose logging output

* Update manticore defaults / requirements;
  * Changes docs for 18.04
  * Python >= 3.6

* minor review changes

* minor review changes, codeclimate
2018-07-25 10:10:44 -04:00
feliam c29c3cc7e8 [WIP] Large EVM refactor (#843)
* Array value configurable. 8,16,32..256 bits (not prefixed to 8)

* Test Array256->256

* Remove old 8 bit assumptions

* Add arrayProxy copy constructors and remove old 8 bit assumptions

* remove old 8 bit assumptions

* remove old 8 bit assumptions

* Better ArrayProxy special case handling

* WiP-research Use raw no-optimized smtlib generic arrays as memory and storage

* remove pdebug print:

* Fix tests

* Review fixes

* review fix

* fix gas variable

* Review fixes

* git hell

* fix asm_to_smtlib

* Fix vm.gas access

* getvalue hack

* Improve internal API

* solver.can_be_true(cs, True) must check that cs is feasible

* Simpler callstack code

* Fix merge and tests

* Fix copy.copy

* Move state ids to private

* New internal add_transaction api

* Fix tx.return_data

* Remove comments

* Do not rely on .last_exception

* Fix evm tests

* peping

* peping

* Peping +test fixes

* Peping +test fixes

* Peping +test fixes

* Fix endTx pop vm

* Improve speed and simplifications. Remove internal_transaction mess

* Change test_eth.EthTests test a bit

* Fix some tests and CALLDATALOAD

* Add CALLCODE and reachability test frame

* Fix calldataload, msize. Add reachability test frame

* Fixlog

* Clean reachability test

* Fixe CODECOPY concrete

* Some style sdded

* peping

* Remove origin and price from EVM as it can be retrieved from human tx

* Added current_human_transaction and last_human_transaction

* peping

* Refactor tests

* Increased argument count  allowed CC

* Initial automatic library loading (no toposort)

* pep8

* Accepts any number of libaries on any ordering

* pep8 and relaxed codeclimate

* Simple library test

* merge libraries in

* Solve https://capturetheether.com/challenges/math/mapping/

* Remove debug print

* More robust path creation for test files

* DELEGATECALL support

* generate testcases only at finalize()

* Add functools32 as dependency for lru in py2

* Manticore Library PoC

* Favor bytearray

* Favor bytearray

* config

* Fix concolic and sha3 arraybyte-> str

* Fix concolic

* Fix typo

* remove absolute path from tests so it fits in link placeholders

* Shrink path spec for solc

* Hopefuly Fix solc invocation bug

* FFFFFFFFFIIIIIIIIIIIIX ITTTTTTTTTTTTTTT

* Better performance on visitors cache, and new ArraySlice

* better os joining, evm library link, and tx based events

* Fitering solidity functions plugin

* add itereator capabilities to ConstarintSet

* Some solver functions exported to state

* bye bye metaclass dynamic check

* extra simplification

* Improved symbolic gas and other bugfixes

* Better visit_BitVecExtract

* Default initial timestamp and block number

* 99 typos

* Re-enable account.named_func() trick

* typo

* Better stronger

* Added default symbolic arguments to constructor..

* Remove constant marked functions from multi_tx

* Add default 16G memory cap to z3

* concretized_args now supports concretizing address arguments to all possible accounts

* Fix coverage calc

* reset fails sometimes

* PEP8 ROA (#900)

* pep8 formatting

* Code Climate updates

* cyclocomplexity--

* Refunds moved from the instruction to the the exeption catching

* merge

* merge

* accept both bytearray and str

* doc

* blockhash changed and returns a symbolic value now, so these tests are wrong

* A bugfix

* pep8/complexity updates

* remove more questionable tests

same thing as with the blockhash thing. we're doing things a little more symbolically now so comparing to concrete will not work

* Accomodate our chubby State class

* Keep moving the refunds

* Disble reachability test for now

* Remove double declaring log for now

* Fix coverage calculation

was prev creating a generator, which you can't do `in` on??

* require 1 tx for int overflow test

* Fix path tests

* Remove debug print in tests

* Only enable gas when necessary

* codeclimate fixes

* Fix concretize_args

* only 1 tx

* Disable max/min support in solver
2018-05-25 19:22:49 -04:00
Mark Mossberg 6616b9e05c Ethereum ABI and misc fixes (#855)
* Port most of the less invasive/controversial changes from dev-evm-dynamicarguments

* Port section of fixes from ethereum.py

* Port tests unrelated to concretizing dyn args

* Add remaining changes from evm.py

* clean, fmt, code climate

* rm unnecessary setup

* disable??

* fmt

* remove all the returns

* lol cc

* Rm unused code

* port to .format

* don't use map (for py2/3 compat)

* simplify slicing

* Rm dyn arg code, since it doesn't really even work

and will be included when dev-evm-dynamicarguments is totally resolved

* add address/get_uint test

* Revert "Rm dyn arg code, since it doesn't really even work"

69188da4ca
2018-04-05 13:31:16 -06:00
Dan Guido bb4a7966a8 Add codeclimate, tox, and pycodestyle configs (#772)
* pep8

autopep8 -r -i -a -a manticore

* tox for testing py3k

* minimal codeclimate

* Add correct Code Climate ID

* unignore max line endings since it's set to 160

* more badges

* turn off pep8 in code climate

* need to remove for codeclimate to work right

* re-enable most checks with looser thresholds

* remove unused imports

autoflake --in-place -r manticore

* revert the autoflakes changes

* set tox to only py27 for now

* cache is hardcoded to python 2.7.13

* Comment cleanup

* Try autoflake again

* Add dev environment dependency management to tox.ini

* Remove count/statistics options from tox.ini

Per Dan, these crash pep8 analysis on codeclimate at present

* Revert setstate initialization

* Re-introduce explicit compare for flags

* add back structural comment

* rm unneeeded ` as e`

* bump py tox
2018-03-12 14:40:06 -04:00