Add numerous null and sequence type checks to prevent crashes when
processing corrupted or unexpected Authenticode signatures and
their nested attributes.
Closes#2242.
Ensure `YR_ARENA` objects are created, loaded, and accessed with a valid number of buffers. This prevents out-of-bounds memory access and improves robustness against corrupted arena data or incorrect API usage.
Closes#2238.
A heap overflow could occur when loading a hand-crafted compiled rules file. If the used field is less than sizeof(void) an integer overflow occurs that bypasses the file validation and causes heap overflow later on.
Bazel was the build system used by VirusTotal. Now that VirusTotal has moved to YARA-X, maintaining the Bazel infrastructure working is painful, specially with the backward incompatible changes introduced in recent versions of Bazel.
After #2119 `yara` is consuming a large amount of memory while parsing 9bddb45c44d9c25a4f97ef800cb110de5e6a15349bac05d389c8bda37902f25a. That's because after the change it doesn't limit the total number of imported functions, only the total number of parsing attempts, but the count is reset with each import entry. This file has a large number of entries and a large number of functions per entry, the total number of functions is very high.
It turns out that we must limit both the total number of correctly parsed functions (for cases like this one), and the total number of parsing attempts (for cases like the one #2119 was aiming to solve).
One section entry was being ignored while parsing file b200f89fe313b6311e444e37725cae95127797ddb430e711c788c264acf92eb0 because we were forcing every resource entry to be shorter than the PE itself.
Although this sanity check makes sense, with truncated files this can lead to resource entries that are ignored. Here we make the sanity check more permissive, allowing entries that are larger than the PE file, but rejecting entries with ridiculously large sizes.
For some reason in MacOS 14 the build fails because the `configure` script is unable to find the Jansson library, even thought it is correctly installed by `brew`.
File e77b007c9a964411c5e33afeec18be32c86963b78f3c3e906b28fcf1382f46c3 has a Rich header of only 8 bytes, which is smaller than the RICH_SIGNATURE structure. This was causing a crash when some of the `rich_xxx` functions were used with this file.
In regular expressions, repetition operators`{0,0}` and `{,0}` are valid, but they always match the zero-length string. For instance, `a{0,0}` and `[a-z]{0,0}` both match the zero-length string.
When the whole regular expression consists in one of these repetitions it caused a segfault during the evaluation of the regular expression because the `forward_code_ref` field for the root atom in the atom's tree was null.
Closes#2084.
When an included file contains an error that makes the parsing fail, the compiler must pop the file name from the stack before continuing parsing the original file, otherwise it will cause the confusing errors described in https://github.com/VirusTotal/yara-python/issues/260
For instance, `x{n, m}`, `x{n ,m}` and `x{n , m}` are all equivalent to `x{n,m}`. This is done for compatibility with YARA-X, but I've seen real-life regular expressions in that contain such spaces, for example:
```
/C:\\Users\\qqq\\.{30, 50}\.pdb/
```
The `pe_parse_delayed_imports` function didn't take into account that `pe_rva_to_offset` could return -1 when the RVA can't be translated into a file offset.
Until now, these two fields were populated only when the offset and offset + length were within the file boundaries. This meant that the fields were undefined for some files that were truncated. For instance, 3323480fc4e35f73a8e93c4a0658c818d7e517df7ba15ba4cbb5cd2b6058cb0f.
The `matches` operator was returning false if the left operand was an empty strings. So, `"" matches <some regexp>` was returning false regardless of the regexp used. However, some regexp *do* match the empty strings, like for example `/foo|/`.
File e64e806c6df76f245c9b1288f59198bc9478329cfdd2d7ddcb9490a57d0b3cf4 was not properly parsed because `NumberOfRvaAndSizes` overflowed and became negative.
Some PE files have a NumberOfRvaAndSizes equal to zero. Even if these files are not accepted by the Windows loader, they can be parsed, and useful information can be extracted from them. By relaxing the sanity check in pe_get_directory_entry we are no able to parse such files (like 0aa527fd26c8e7fa7f3476b6d7eb8f22c071ff4c0fd3ed3797fa90ceb80f24ce).
.NET resources can be found outside the current file. Before this change those resources were being ignored, but now they are included in the `resources` array. The `name` and `length` fields are kept undefined though. So, external resources have only the `name` field set.
This function was checking the first two bytes at the entrypoint of .NET files, if the bytes were not `FF 25` the file was not considered a .NET file. This check was overly restrictive, as some files like `8fa7fe73a65296e9ca8301734a0deaa298cda0b7a65f6b7d2ef6d1c8bbb8cd7a` don't have such files at the entrypoint.
This function was returning a wrong RVA for files like `67508ec9cbd9f82c6aea8a93ca27fd3684d368664470397755f8d70934d481fb`, when a section's virtual size is smaller than its physical size.
Failing to parse some resource directory does not mean that we can't keep parsing other directories. For instance, with file `0a65e22cdff348be11739122906355af728f6247fb849f98d69a6f6d51103071` YARA was aborting the parsing too early and extracted a single resource, while the file has more resources that could be parsed correctly.
YARA was making the assumption that the 12 bytes following the `DanS` header contained 3 copies of the 32-bits XOR key, but these bytes are actually padding. As these bytes were usually filled with zeroes before applying the XOR key, they appear to contain the key itself after the XOR operation. However, files like `043066108b68b30fc2c475eae8edfafc080be7d451600eaa283d2c750bddbceb` don't contain zeroes in those bytes, and therefore were not satisfying YARA's assumption.
YARA-X was right: https://github.com/VirusTotal/yara-x/blob/6ada059b9e4b328e2341b42304765c5614fc89af/lib/src/modules/pe/parser.rs#L687-L693
As discussed in https://github.com/VirusTotal/yara-python/issues/247, the example using `StringIO` worked fine in Python 2, but ti doesn't work in Python 3. Now that Python 2 is mostly dead, update the example.
This is backward compatible change that affects the `magic` field. After this change the value in the `magic` field looks exactly as it looks in the file regardless of the endianness of the current platform, if the file starts with `CA FE BA BE` the value in magic is `0xCAFEBABE`, not `0xBEBAFECA` as it used to be in little-endian architectures.
Regexps with quantifiers `{0}` and `{0,0}` were considered syntactically invalid. These quantifiers are not actually useful, but they are now allowed for compatibility with other regexp engines.
Closes#2037.
Until now, `pe.is_signed` was undefined for every PE file that was not signed, making impossible to use it in negated conditions, like `pe.is_pe and not pe.is_signed`. With this change, `pe.is_signed` is false for every PE file that is not signed.
Closes#2040
The pointer returned from `libmagic` was passed directly to `yr_strdup` (a wrapper of `strdup`), but this function has undefined behavior when the argument is null.
Instead of downloading the latest master branch, use the code in the current branch. Until know, this workflow was always operating wit the latest commit in the master branch.
Some regexps like `/(abc|)/` where not matching the empty string even if they should. Matching an empty string is not very useful, as such regexp will match on every offset of every file, but this is fixed for correctness.
Also fixes another issue with the `\b` metacharacter in regexps, which was exposed after fixing the main issue.
While porting the `dotnet` module to YARA-X I discovered the the implementation of `read_blob_signed` had a subtle bug. As `first_byte` was declared as `uint8_t` instead of `int8_t`, sign extension when casting the value to a `int32_t` was not done, leading to incorrect values.
Victor M. Alvarez