* cli/yara: Replace printf with _tprintf in console output callback
This unbreaks output on MSVC-based builds as discussed in #1594
* cli/yara: Fix MSVC warning about incompatible pointers
There is an issue with argument handling that is causing issues when scanning
using certain argument combinations. I believe this is similar to
https://github.com/VirusTotal/yara/pull/927. Here's an example of it broken in
master:
wxs@wxs-mbp yara % cat rules/always_true.yara
rule a { condition: true }
wxs@wxs-mbp yara % ./yara -p 1 rules/always_true.yara /bin
wxs@wxs-mbp yara %
And the same thing with this fix:
wxs@wxs-mbp yara % ./yara -p 1 rules/always_true.yara /bin
a /bin/cat
a /bin/echo
a /bin/launchctl
a /bin/df
<snip>
Issue spotted by: Conor Quigley (https://github.com/schrodyn)
Add a console module which can be used to print things during condition
execution. This is especially useful for debugging complex conditions where
you're following multiple offsets. For example, people sometimes want to express
a PE by doing this:
uint32(uint32(0x3c)) == 0x50450000
What they really want is:
uint32be(uint32(0x3c)) == 0x50450000
Or they want:
uint32(uint32(0x3c)) == 0x00004550
In either case it would be helpful if they could do:
console.hex(uint32(uint32(0x3c)))
Another example of where this could be useful is being able to know what YARA
thinks an imphash is just by doing:
console.log(pe.imphash())
* Make console module a default module.
* Add support for optional string messages.
Override each function so that it can also take an optional string that is
printed at the start of the message. Tested with the following:
import "pe"
import "math"
import "console"
rule a {
condition:
console.hex("0x3c: ", uint32(uint32(0x3c))) and
console.hex(uint32(uint32(0x3c))) and
console.log("String: ", "foo\x00bar\x01baz") and
console.log("foo\x00bar\x01baz") and
console.log("Some integer: ", 987654321) and
console.log(987654321) and
console.log("Some float: ", math.MEAN_BYTES) and
console.log(math.MEAN_BYTES) and
console.log("Imphash: ", pe.imphash()) and
console.log(pe.imphash())
}
* Remove duplicate macro definitions.
* Make console functions use callback.
Add a CALLBACK_MSG_CONSOLE_LOG which the console module uses to send a message
to the main callback. Make the console module functions all generate a single
string which is then passed to the callback. This keeps the callback handler
simple: it just prints a string.
* Fix newlines.
* Add docs for console functionality.
* Add docs for console_callback to yara-python.
* Address comments from Victor.
This is a more comprehensive fix than #1617, it adds new functions to the API for getting/setting uint32 and uint64 settings. Using these functions is preferable over calling `yr_(get|set)_configuration` directly.
A common use case (I've run into this many times, as have some friends) is to
run yara against a file named after the hash of the content. If the hash happens
to start with a number yara.c treats it as a PID, and will fail. The workaround
is to prefix the file with "./" (or whatever the equivalent is on Windows) so
that the strol() check fails.
This diff switches the behavior so it treats the argument as a file first, and
only if that can be opened is it then checked for a PID. Further, to ensure the
PID check is accurate I extended it so that the entire argument must be
converted. If there are any characters left after the conversion then it is not
a valid PID and we will not attempt to scan it as it is (by definition) not a
valid file nor a valid PID.
The warning was caused by the use of strcmp with a string returned by readlink, which doesn't guarantee that the string is null-terminated. In this case the use of strcmp is not required as we are checking only one character.
1. skip all symlinks pointing to . so you do "yara -r test.yar /usr/bin" on debian without -N (/usr/bin/X11 points to . )
2. Show 64 instead of 32 hex chars (regex are shown in full so there's enough clutter anyway)
Includes the following changes:
* Print the rule identifier in warnings due to strings matching too much. The string identifier is not enough, common identifiers like $a are not very informative.
* Use the same format for compile time warnings and scan time warnings.
* --no-warnings disables the "too many matches" warning.
* --fail-on-warnings makes yara fail with the "too many matches" warning.
* Use the same message format in yara and yarac.
When we hit the maximum match limit on a string we currently abort all scanning,
which can be problematic since it isn't a good reason to stop all scanning, and
is better off as a warning to the user.
This commit makes it so that when the string match limit is hit a warning is
printed and scanning will continue. The warning message indicates that the
results may be invalid. It is hard to know for certain if they are invalid
because it depends entirely upon the condition. If you use the # or @ operator
then your condition _MAY_ be invalid since the total number of matches and
locations is not known anymore. Just checking that the string exists won't be an
invalid state since we have confirmed the existence of _AT LEAST_ the maximum
number of occurrences, which clearly satisfies the existence of a single
instance of your string.
I decided to implement this as a callback so the user of libyara has some choice
on the default behavior. If they want to consider this a "stop all scanning"
error they are free to return CALLBACK_ERROR in their callback.
Apparently, the C standard library in older Windows versions does not
contain strtok_s (see discussiion for PR #1381). Since strtok_s is
only used in the single-threaded code of YARA itself, reentrancy is
not really needed.
Victor M. Alvarez