* Import authenticode-parser into Yara. Rewrite PE signature processing using the authenticode-parser. Update bazel OpenSSL dependency to 1.1.1 from 1.1.0
* Forgot to include structs.c
* Add Avast to authors and myself to contributors
* Set forgotten algorithm_oid
* Fix incorrect indentation
* Fix race condition when initializing openssl objects
* Don't malloc ByteArray with length 0
* Auth. parser update - bug fixes, openssl3.0 support
* Update authenticode parser
Co-authored-by: Victor M. Alvarez <[email protected]>
* Fix some compiler edge cases.
Add a warning when "0 of them" is used. Point to some new documentation that
explains why "none of them" is preferred.
Handle some edge cases, which are all now errors:
* -1 of them
* "foo" of them
* /foo/ of them
And this one is also an error if the external variable "x" is a negative integer
or a string:
* x of them
I've also made sure this works with a construct like "0 + x of them" where x is
defined as a negative integer.
Add test cases for as many of these as I can. I don't have test cases for the
external variable case, but it does work as expected. Here is the output of a
rule using external variables where the variable is defined to be 1, 0, -1 and
foo:
```
wxs@mbp yara % cat rules/test.yara
rule a {
strings:
$a = "FreeBSD"
condition:
x of them
}
wxs@mbp yara % ./yara -d x=1 rules/test.yara /bin/ls
a /bin/ls
wxs@mbp yara % ./yara -d x=0 rules/test.yara /bin/ls
warning: rule "a" in rules/test.yara(5): Consider using "none" keyword, it is less ambiguous. Please see https://yara.readthedocs.io/en/stable/writingrules.html#sets-of-strings-1 for an explanation.
wxs@mbp yara % ./yara -d x=-1 rules/test.yara /bin/ls
error: rule "a" in rules/test.yara(5): invalid value in condition: "-1"
wxs@mbp yara % ./yara -d x=foo rules/test.yara /bin/ls
error: rule "a" in rules/test.yara(5): invalid value in condition: "string in for_expression is invalid"
wxs@mbp yara %
```
* More compiler warnings and errors.
If you specify "2 of ($a)" it is now a warning because it will always evaluate
to false, and is quite possibly not what you wanted to write. This is true for
string sets, rule sets and string sets in a range like "2 of ($a) in (0..10)".
If you are using an integer range it is now an error if the lower bound is
greater than the upper bound.
To support rule sets I needed to modify yr_parser_emit_pushes_for_rules() to
use a count parameter, which it will update with the number of pushed rules.
This is identical to how yr_parser_emit_pushes_for_strings() works.
While I'm here, remove the link from the warning message for the "none" keyword.
* Add not operator (~) for bytes in hex strings.
* Describe not hexstring not operator in documentation.
* Basic tests for the hextring NOT operator
* Fix not operator documentation ordering
This is done for consistency with pe.is_pe, which is a field instead of a function. The initial idea was adding the is_pe() function to the pe module, but that's not possible because a function and a field can not have the same name, and is too late to remove the is_pe field.
Old documentation referenced ".c" files that were all in the same folder. However, later versions of YARA have them each in their own folder. Updating the docs to reflect this will mean there is no confusion when trying to create new modules.
Currently adding in `MODULES += modules/demo.c` won't work either, so this will make it easier for people to test the demo module.
This adds the ability to use rule sets just like we do with string sets.
Specifically, it enables conditions like the following:
5 of (a, b)
50% of (a*)
where "a" is an existing rule and "b*" is a wildcard rule identifier, which is
expanded out precisely as wildcard string identifiers are.
I chose to make OP_OF and OP_OF_PERCENT emit an argument which is the "type" of
set it is expected to match on. The two acceptable values are OF_STRING_SET and
OF_RULE_SET. When OF_STRING_SET is used then the references on the stack are
expected to be strings and OF_RULE_SET will check if the rules on the stack
match.
I added a yr_parser_emit_pushes_for_rules() which behaves similarly to the
string counterpart. The one big exception is that I can't use yr_rules_foreach
macro since that needs the terminating "null" rule which is not yet added, so
instead I have to iterate the rules manually using yr_arena_get_ptr() to walk
all the currently compiled rules.
* Stricter ordering of rule set in compiler.
Make sure that when we compile a new rule that any existing rule sets do not
match the new rule. Specifically we are making the following a compiler error:
rule a1 { condition: true }
rule b { condition: 1 of (a*) }
rule a2 { condition: true }
This is a compiler error because "a2" is defined after the usage of (a*) in rule
b.
There is still an edge case here. These rules compile:
rule a { condition 1 of (a) }
rule b { condition: 1 of (b*) }
In both cases those rules evaluate to false, which is in line with the behavior
of this rule:
rule c { condition: c }
Given that the behavior around rule sets matches the current behavior I think
this is acceptable.
* Add docs for rule sets.
* Implement hash table iterators.
* Remove old comment about linked list.
* Move variable declaration.
* Iterate over rules better.
Add a console module which can be used to print things during condition
execution. This is especially useful for debugging complex conditions where
you're following multiple offsets. For example, people sometimes want to express
a PE by doing this:
uint32(uint32(0x3c)) == 0x50450000
What they really want is:
uint32be(uint32(0x3c)) == 0x50450000
Or they want:
uint32(uint32(0x3c)) == 0x00004550
In either case it would be helpful if they could do:
console.hex(uint32(uint32(0x3c)))
Another example of where this could be useful is being able to know what YARA
thinks an imphash is just by doing:
console.log(pe.imphash())
* Make console module a default module.
* Add support for optional string messages.
Override each function so that it can also take an optional string that is
printed at the start of the message. Tested with the following:
import "pe"
import "math"
import "console"
rule a {
condition:
console.hex("0x3c: ", uint32(uint32(0x3c))) and
console.hex(uint32(uint32(0x3c))) and
console.log("String: ", "foo\x00bar\x01baz") and
console.log("foo\x00bar\x01baz") and
console.log("Some integer: ", 987654321) and
console.log(987654321) and
console.log("Some float: ", math.MEAN_BYTES) and
console.log(math.MEAN_BYTES) and
console.log("Imphash: ", pe.imphash()) and
console.log(pe.imphash())
}
* Remove duplicate macro definitions.
* Make console functions use callback.
Add a CALLBACK_MSG_CONSOLE_LOG which the console module uses to send a message
to the main callback. Make the console module functions all generate a single
string which is then passed to the callback. This keeps the callback handler
simple: it just prints a string.
* Fix newlines.
* Add docs for console functionality.
* Add docs for console_callback to yara-python.
* Address comments from Victor.
This is a more comprehensive fix than #1617, it adds new functions to the API for getting/setting uint32 and uint64 settings. Using these functions is preferable over calling `yr_(get|set)_configuration` directly.
YARA didn't support any expression which would check value against YR_UNDEFINED constant so checking for non-existence of something can be very problematic. But we run multiple times into issue of people asking us for checking of absence of something.
Because of this we propose operator defined which would be unary operator (like not) which works like this:
defined x evaluates to
true - whenever x != YR_UNDEFINED
false - whenever x == YR_UNDEFINED
Related to issue #835
* Implement OP_COUNT_IN.
Add a new instruction (OP_COUNT_IN) which is used to check the number of
occurrences of a string in a range. You can use this with the following syntax:
#a in (0..100) == 2
This will check that the $a string matches exactly 2 times in the range
provided.
* Add docs for counting strings in a range.
* feat: Implement stats module
Add a new statistics module. For now, the module supports counting
the occurrences of specific bytes (or returning the occurrence rate
as a percentage) and calculating the most common byte for an area.
* Another example
* feat: Implement stats module
Add a new statistics module. For now, the module supports counting
the occurrences of specific bytes (or returning the occurrence rate
as a percentage) and calculating the most common byte for an area.
* refactor: Integrate stats module into math module
Integrate the new functionality into the math module instead
of creating a new module.
* chore: Adjust module name in documentation
* chore: Fix memory leak issue
Fixes a memory leak where a temporary array (used for match counting)
wasn't released properly.
* fix: Remove global char count function, optimize char count
Remove a redundant (and slow) function to count a char globally.
Rework the remaining char count function to be more efficient.
It still requires O(n*m) in worst case and O(n) in best case.
* chore: Update documentation
* chore: Remove string counting function
Due to the new "x of them in ..." syntax, string counting is no
longer required (it already is supported and this function is
slower in most circumstances).
* chore: remove test functions for string count
Co-authored-by: Florian Roth <[email protected]>
This is a combination of "OP_OF" and "OP_FOUND_IN" that allows you to express logic like "all of them in (0..100)". This came out of the discussion in #1483 and likely makes part of that pull request redundant.
I fixed syntax of table in writingrules.rst
・“YARA keywords” table has syntax error, and it table couldn't see on the document page.
・I changed that some '*' positions.
