* rich_internal function now returns the sum of counts
of the matching rich entries instead of a boolean
It remains backwards compatible thanks to non-null values being
considered as true and when not found it returns 0 (false).
* Update documentation of the `toolid` and `version` in PE module
to reflect changes of behavior: it now returns the sum of counts matching
to given `toolid` and `version`.
* Added test for the updated rich_signature function
* Implement improved xor modifier.
* Fix breakage when xor(...) came before another modifier.
* Add documentation for xor improvements.
* Fixes from review by Victor.
- No more changes to lexer.
- Duplicate modifiers are now errors.
- Do not rely on undocumented behavior in bison.
* Apply fix to deal with undocumented bison feature we shouldn't rely on.
* Make xor work properly with other modifiers.
This commit now makes it so that "xor(1-255) wide" will no longer search for the
plaintext wide version of the string, but "xor wide" will because it implies
xor(0-255).
* Add support for xor(X) and tests.
* Make YR_STOPWATCH to have the same size in Linux, independently of HAVE_CLOCK_GETTIME being true or not.
No matter the value for HAVE_CLOCK_GETTIME the YR_STOPWATCH structure now have the same size, and users of the library don't need to provide the right value for HAVE_CLOCK_GETTIME while building their programs.
* Change #elif to #else.
* Build with profiling enabled in Travis-CI.
* Try to fix issue with certificates in Travis CI.
* Fix typo.
* Fixes for xor modifier.
This commit makes the xor modifier no longer include "plaintext" versions of the
strings. In particular if you have:
$a = "This program cannot" xor
Then it will no longer match on the plaintext version of the string. It will
only match on the xor versions. If you want it to match on plaintext or xor then
you can use:
$a = "This program cannot" xor ascii
The same is true for wide.
* Make 'xor nocase' a compiler error.
* Add newline between ascii and wide strings.
Due to an unsigned to signed conversion _yr_re_is_char_in_class was returning a wrong result for some non-ascii inputs when called in case-insensitive mode.
Add some tests for the dotnet module, which uses the smallest .NET binary I had
laying around that exercises a fair bit of the code. In particular it has a
resource, guids, assembly information and a couple of other things that make for
good tests.
* Add test case.
* Fix issue with regexps and hex strings that were not splitted correctly.
The function yr_re_ast_split_at_chaining_point was not splitting correctly regexps and hex strings that contained more than one large jump (a.k.a gap). For example, { 01 02 03 [0-1000] 04 05 06 [500-2000] 07 08 09 } was not splitted in three parts as expected, but only in two parts: { 01 02 03 } and { 04 05 06 [500-2000] 07 08 09 } . This didn't affected the correctness of the matches, but it caused ERROR_TOO_MANY_RE_FIBERS in some cases while trying to match { 04 05 06 [500-2000] 07 08 09 } because of the large jump in the pattern. The occurrence of the error depended on the data being scanned.
This commit fixes the aforementioned issue, improves the documentation of related functions and simplifies some portions of the code.
* Avoid putting the same string multiple times in matching_strings_arena.
This fixes a small issue introduced in #1096. The same string could be put more than once in the matching_strings_arena, this doesn't causes any major error nor affect the correctness of the matching, but it could have a negative impact on performance.
* Add more comments.
* Improve documentation for _yr_scan_verify_chained_string_match.
A heap overrun can occur in call to set_sized_string when called with blob_offset and blob_result.length values that are not validated to be within the boundaries of the buffer containing the PE file. The validation was being done before adding blob_result.length to blob_offset.
Credit to OSS-Fuzz for finding this bug.
These two strings are logically equivalent, but one throws a warning and one
does not:
$a = { 13 [1] 20 00 [1-30] 13 [1] 10 00 }
$b = { 13 ?? 20 00 [1-30] 13 ?? 10 00 }
Currently $a will throw a warning and $b will not. Fix this by treating single
byte jumps as a masked literal, so atom selection will be able to properly work.
* Add private modifier for strings.
This commit adds a private modifier to the grammar, which can be used on any
string. When this modifier is used the string is treated as normal except it is
never reported back in match objects. It is still searched for normally and
still used normally in condition evaluation but it will never be seen in the
match callback.
This is a useful feature when you have strings that you want to use but they
match a lot of times and you don't want them polluting your log results. The
best example that demonstrates this is the technique described at [1]. In this
example the local file header magic is going to match a lot of times, and it is
only needed to look through the local file header for filenames at offsets
relative to the matches. We don't really care where the offsets of the local
file header are, just where the matching filenames are.
If you are logging where all the matches are for every string and there are
10,000 local file headers that's a lot of log data being generated which is not
helpful at all.
[1]: https://blog.korelogic.com/blog/2015/08/19/yara-ctf-2
* Optimize VM execution code for private strings.
Rather than checking the matches list and only if none are found then check for
private matches, it's better to check only the appropriate list after using
STRING_IS_PRIVATE.
* Add private strings in various tests.
This should cover all the cases of operators on strings that also need to check
for private strings.
* Add docs for private strings.
* Fixup documentation.
Private strings apply to all string types, not just text strings.
* Fix private modifier for chained strings. Add a test case for it too.
The library built by the Bazel target //sandbox:yara_sapi can be used
directly to perform sandboxed YARA scans.
The API is intentionally kept very simple, only three methods are
exposed by the YaraTransaction C++ class:
* YaraTransaction::Create() to create with the specified timeout and
number of worker threads
* YaraTransaction::LoadRules() to load YARA rules from string, and
* YaraTransaction::ScanFd() to scan the contents of a file descriptor.
Internally, this creates a new sandboxed sub-process which maintains a
dispatch queue and multiple worker threads. The transaction class then
multiplexes scan tasks to the sandboxee where they will be executed in
parallel. This allows many scan threads to share the same set of YARA
rules, saving memory.
See sandbox/sandboxed_yara.cc and the unit test in
sandbox/yara_transaction_test.cc for examples on how to use this API.
Signed-off-by: Christian Blichmann <[email protected]>
Add crc32 functions to the hash module. They work in the normal "offset and
size" way and on arbitrary strings.
While here, add tests for it and also documentation.
* Add Bazel build file and workspace
This will build Yara on Linux using BoringSSL, other platforms are
currently untested.
A few of the modules are currently disabled in the build, like Cuckoo
and Magic. This is due to additional external dependencies.
Signed-off-by: Christian Blichmann <[email protected]>
* Add Bazel build file for the tests.
This omits the version test, as it is not very meaningful for Bazel.
Signed-off-by: Christian Blichmann <[email protected]>
Ԝеѕ