That fixes a really annoying bug of aar ignoring any vmap.
If core_search_for_xrefs_in_boundaries() gets an interval which lies within a vmap,
rz_io_size() returns the size equivalent mmap.
Which can be smaller and make the if case fails.
This seems to be a bug for all virtual files mapped in memory.
If a virtual file of size 0x100 is mapped to a memory map of
size 0x180, it creates a vmap of 0x100 for the file content and
a 0x80 mmap for the empty bytes.
These maps are names vmap.filename, mmap.filename.
rz_io_size() now seems to only ever return the size of the mmap.
I don't know why it does that.
And because it is one of the stone age functions I am not going down there to look.
This bug breaks aar for Qualcomm split firmware images (.mdt).
Removing this check should not have an impact, because I added
a check for the memory read.
* Allow to set dotall and multiline flag also for normal regex patterns.
* String search sees NUL as newline by default.
That is the most intuitive option for binary searches.
The behavior can still be changed. An example us added.
The jump addresses of call and jump instructions are now written to their own
and unique local variables. Before this, all jump instructions wrote to the
same local variable.
This was a problem for abstract interpretation: Because if multiple writes to
the same local var happen due to a previous TOP condition, the local variable
content is also TOP. If the jump target is TOP, the interpreter can't follow it
anymore.
Added tests for all the funny packet configurations with 0-2 jumps in it.
* Don't print meta items which are not at the current seek.
The old code tried (unsuccessfully) to print _any_ meta item _covering_ the seek (ds->at).
There seems to be several bugs getting triggered with that.
One of them giving the behavior of https://github.com/rizinorg/rizin/issues/6556.
If the current seek is in a _data_ region, the disassembler logic doesn't care.
It just assumes that RzAsmOp.size is equivalent to the size of the objects there.
Even though there are only Meta items.
But since some meta items are like 4K bytes, RzAsmOp.size gets
trimmed down.
Anyways, that completely messes up the size calculation (as can be seen in the issue),
and the navigation.
I couldn't figure out where stuff broke.
But the library closes and I have to leave, so I push that.
That "fix" makes it at least behave somewhat consistently.
* Fix leaks
* Fix and add interactive test
* Cast operands for AND and OR instructions to the correct width
* Add missing operand casts for SBB and ADC.
* Add flawed instructions to asm tests
---------
Co-authored-by: Dhruv Maroo <[email protected]>
* Fix revert if adding a node failed.
* Reduce allocations by keeping only a single RzGraphEdge object per edge around.
* Add benchmark for graph deletion and addition of nodes/edges
* Use rz_pvector_remove_at_unsorted to save some runtime.
* Use realloc and memmove for matrix graphs on capacity increase.
* Add helper to determine memory usage.
* Add benchmark
* Revert matrix capacity extension to simple and jsut as fast loop.
* Missing type annotations
* Add option to replace (geometric) invalid values with another value.
* Add geometric Mean and Std Dev to benchmarks.
* Add Doxygen documentation
* Add a README to the bench dir as intro.
* Add Welfords square of sums algorithm for variance and std deviation calculations.
* Add standard deviation to benchmarks
* Simplify Welford
* Add geometric mean and standard deviation to Welford Sums
* Add order ignoring remove_at version with better performance.
* Optimize rz_vector_swap by using stack memory for small elements.
* Add benchmark for rz_vector_swap
* Refactor del_edges to use RzGraphStatus.
* Refactor del_edge() to use RzGraphStatus.
* Refactor update_edge() to use RzGraphStatus.
* Refactor has_edge() to use RzGraphStatus.
* Refactor add_edge() to use RzGraphStatus.
* Fix leak of b
* Fix leak of xref list
* Fix leaks of analysis ops
* Address review comments
* Inlcude clean up
* Fix invalid free
* Add tests with node and edge data.
* Extend tests
* GRAPH: Add rz_graph_add_get_node() and refine API.
* GRAPH: Remove unused, and rename parameters.
* Improve doxygen.
* Implement rz_graph_del_edges()
* Rename rz_graph_node_get_hash_id() to rz_graph_node_get_hash_id() to make clear what identifier is returned.
* Simplify node identification.
Removes the option to have two sources of identifiers (node data or other identifier data).
Changes the API to use the hash id instead of a pointer to data.
* Fix type annotations.
* Remove duplicate function.
* Fix invalid asserts.
* Set flag if node was present.
* Grow by a factor of 1.25. Exponential growth quickly leads to OOM.
* Remove the edge index again to not remove reduce the main advantage of an adjacency matrix
* Refactor list based graph to use vectors instead of hash maps for edges.
* Fix heap-use-after-free
* Add an rz_graph_update_edge function.
* Add function to print graph as dot graph.
* Add warning about del_edges runtime.
* Refactor add_node to use RzGraphStatus.
* Refactor del_node to use RzGraphStatus.
* Use cast-macro to prevent ASAN issues.
* Add getter for vector capacity.
* Make rz_(p)vector_assign_at consistent in behavior.
* Document rz_vector_clear
* Move doxygen for rz_vector_flush to definition.
* Add rz_vector_purge for clearing, but capacity keeping.
* Rename flush -> take_array to signal ownership transfer.
* Add first ring buffer implementation.
* Add likely and unlikely macros.
* Implement take and put
* Block writes to buffer if it is full.
* Make Is_open return bool and add unsafe fcn for is_empty
* Add ring buffer tests.
* Add blocking take() function to ring buffer.
* Add open() function to ring buffer
- It uses now the changed lines provided by the get_real_shas workflow,
which prevents failing if PR gets out of sync with base branch.
- Fixes a bug in lsan_check.py which used the wrong line number as reference.
- Moves the whole leak detection workflow out of ci.yml to simplify it.
- Uploads the leak stack traces as artifact for local analysis.
* Basic implement of list based and matrix based graph
Add comments
Basic support for list and matrix based graph refactor
Add rz_graph_*_new node and edge API
Modify calling for rz graph edge data
Add dfs and visitor mode
Add new get nth neighbours
Add unit test and wrapper for get edges
Bug fixed and unit test
Solve TODO about better semantic of get nodes
Doc RZ_API and other functions, split impl to new files
[cannot build] remove old graph impl and rename new graph API
* Rewrite and replace agrach, cgraph, drawable_graph, il_graph with new API
* Refactor graph API and fix unit test and multiple leaks
Update Wrapper for graph identifier
Ordered Nodes in graph drawable
Fix icfg and several agraph refactor issues
Use kahn to get topo sort in assign_layers
Introduce khan for assign_layer and find DAG cycle and backedges in algorithm
* Fix self loop and update db test
* Solve request changes
* Fix cbpf db
* Fix test graph warning in testing NULL deletion
* Fix mismatched rz_agraph_compute_layout nonnull
* Fix memleak
* Clean and fix code
* Fix memleak in graph free
* Fix memleak in matrix implement
* Add bindgen doc
* Fix memleak in early exit
* Fix rz-bindgen warning comments
* Make graph structures private.
* Fix Windows C2036
* Fix typo in type annotation.
* Add new type annotations for RzGraph and HtPP
---------
Co-authored-by: Heersin <[email protected]>
I tried several ways to fix this issue more elegantly.
But I either was too stupid, too tired to do it right, or it got too complex.
The script fails because the PR "base" branch and the PR HEAD
have no merge-base commit in the shallow clone Github does.
If a PR has more than 10 commits, the workflow fails because
the fetching didn't get enough commits to reach the merge-base.
Anyways, this is the simple solution.
Fetch 200 commits at once and done.
The workflow will fail if a PR has more than 200 commits
or the base branch has 200+ commits since creation of the
PR. But this rare and almost always requires a rebase
anyways. So this is the simplest fix.
* Add AGENTS.md with requirement to disclose agent authorship.
* Check for edits to AGENTS.md
* Add PR label on detected AI authorship.
This PR adds an automated check if a contributor used an AI Agent to generate code.
It does this by:
- Adding an AGENT.md requesting the Agent to add a "co-authored by" line to each commit message it generates.
- An added workflow (`llm_check.yml`) runs to check for any commit having such a matched line in it.
- If it finds it, it adds the `AI/LLM` label. The label IS NOT removed automatically if a force push changes the commit message.
The additional `get_real_pr_shas.yml` is a requirement sadly.
It is currently impossible to get the commit the PR is based on via Github APIs.
The variables `github.base_ref` or `github.pull_request.base.sha` either return the commit hash when the PR was created (breaks after rebase) or the new HEAD of the base branch after something unrelated was merged (see: https://github.com/actions/runner/issues/1689).
This makes it effectively impossible to get the output of `git log actual_pr_base..HEAD` via the Github API.
The previous assumption of RzILMem was that the buffer is not owned by it.
This made sense, because in the beginning it only held an RzIO buffer.
But the Xtensa and Sparc RzAnalysisILInitCallback definition initializes
an additional RzILMem object. This additional RzILMem has a sparse buffer in both cases.
The Xtensa and Sparc plugin pass the sparse buffer to RzILMem with the assumption that it takes ownership.
They have to, because there is no fini() version of RzAnalysisILInitCallback in which they could free the buffer.
There are also no API functions to free the buffer in RzILMem.
So the plugins don't have a nice way to clean it up in a theoretical fini() callback.
This commit fixes the leak by splitting rz_il_mem_new() into two version.
One to taking the ownership of the buffer, one borrowing it.
This seems the most natural solution, because buffers can abstract from
all kind of backends (RzIO, a file, some memory etc.).
Each time the ownership of the buffer is different.
So the creator of rz_il_mem_new_*() can decide what buffer with what ownership they pass.
The getter is useful if a module outputs debug logs buts also some updated status update in a single line with rz_cons_printf("\r...").
The status update should not happen if the log level is below warning.
The command was untested. And while it still could be useful less code is probably better.
Additionally, finding collisions is a rather expensive problem,
better left to optimized algorithms and implementations.
* Add an in-place addition of bitvectors.
* Decouple _elem_len from assumption it has just enough bytes to hold len in bits.
* Add in-place copy of bits.
* Use default copy bits bitvector function.
* Mark stack allocated bit vector.
* Add inplace variant of complement_1
* Implement inplace bitvecotr not.
* Add inplace variant of rz_bv_and
* Implement inplace variant of rz_bv_or
* Implement inplace variant of rz_bv_xor
* Constify rz_bv_lsb/msb
* Implement inplace variant of rz_bv_neg
* Fix inplace add. Add inplace SUB
* Remove prefix
* Remove invalid const
* Implement in-place casting of bit vectors.
* Add a hash test
* Fix unnecessary &
* Prevent OOB reads & writes by copying only the minimum of bytes.
* Add inplace MUL
* Add ble version for bitvector set/get with bytes.
* Add documentation for inplace bitvector functions.
* Add bench of add/sub() and sub/add_inplace().
* Remove rz_bv_copy_nbits_inplace because it was the same as rz_bv_copy_nbits.
If a packet was pushed out of the cache, it can be decoded as invalid, although it is valid.
Then it is marked as such according to some heuristic.
But the get_pkt_op parameter was not set in this case, which further down let
the plugin return an EMPTY() effect.
The cache read must read within the mapped region and not beyond.
Otherwise it would go against the documentation of rz_io_nread_at().
rz_io_cache_read() internally doesn't know about map boundaries so
the length parameter passed to it in rz_io_nread_at()
should only read 'ret' number of bytes.
To give somewhat more introspective what rz_io_cache_read() does,
the return value meaning was changed to success/failure
and an out parameter was added to get the number of bytes read.
Rot127