Commit Graph
140 Commits
Author SHA1 Message Date
Victor M. Alvarez fa5682a98d Homogenize function comments and remove empty lines. 2020-10-19 12:00:53 +02:00
Victor M. Alvarez 0bce4bd74d Start using clang-format for formatting the code automatically. 2020-10-07 16:27:30 +02:00
Victor M. Alvarez 1822d363bc More adjustments to the size of hash tables. 2020-07-13 14:02:30 +02:00
Victor M. Alvarez 1dd48ba24e Adjust hash table sizes. 2020-07-13 14:02:30 +02:00
Victor M. Alvarez ea6f24ce3b Make compiled rules smaller by not saving the same string twice.
Before this change if the identifier $a was used in 100 rules, the text string  "$a" would appear 100 times in the compiled rules. After this change it is saved once and referenced from multiple places. The same happens with module names, literal strings used in the rules and so on.
2020-07-13 14:02:30 +02:00
Victor M. Alvarez 9fe5640d9e Increase initial buffer size for compiler's arena. 2020-03-18 11:27:50 +01:00
Victor M. Alvarez 54eb7c35db Change the way in which the last meta for each rule is marked.
Use a flag for marking the last meta, instead of inserting an empty meta at the end of the list.
2020-03-17 16:44:20 +01:00
Victor M. Alvarez 4f716aafa0 Add "arena" field to YR_AC_AUTOMATON. 2020-03-16 10:52:17 +01:00
plusvic cc72b1aff2 Merge branch 'v4' 2020-03-13 10:06:27 +01:00
Victor M. Alvarez 36f3d6a2fe Rename arena2 to arena. 2020-03-05 10:56:45 +01:00
Victor M. Alvarez 62be88c654 Change the way in which lists of Aho-Corasick matches are encoded. 2020-03-02 21:15:47 +01:00
Victor M. Alvarez 6b8111db71 Add some explicit typecasts and fix issues with features not supported in Microsoft compilers. 2020-02-28 19:52:45 +01:00
Victor M. Alvarez 73aa6097f4 More refactoring. 2020-02-28 17:29:32 +01:00
Victor M. Alvarez b68deed0c3 Remove all remaining dependencies from tidx (except for the "magic" module). 2020-02-27 19:54:11 +01:00
Victor M. Alvarez 2b121b166d Track string matches using YR_SCAN_CONTEXT. 2020-02-27 13:51:47 +01:00
Victor M. Alvarez 34ec34fbe1 Start using bitmap for tracking rule matches. 2020-02-27 09:34:29 +01:00
Victor M. Alvarez 8ce23fe46f Add Aho-Corasick automaton to new arena. 2020-02-26 20:45:18 +01:00
Victor M. Alvarez ae3d71d0a4 Move regexp code to new arena. 2020-02-26 12:31:53 +01:00
Victor M. Alvarez 2886e2aa1a Move code to new arena. 2020-02-25 15:43:10 +01:00
Victor M. Alvarez 64cc3532f0 Add semicolons. 2020-02-24 18:08:59 +01:00
Victor M. Alvarez 7dd81fa2b6 Null-terminated strings moved to new arena. 2020-02-24 18:04:33 +01:00
Victor M. Alvarez bff80be2f6 Add metas and strings to new arena. 2020-02-24 12:28:03 +01:00
Victor M. Alvarez 89f1e9a520 External variables moved to new arena. 2020-02-21 17:05:25 +01:00
Victor M. Alvarez 9c8ac05800 Rules moved to new arena. 2020-02-21 12:58:14 +01:00
Victor M. Alvarez 1cd9f535a0 Namespaces moved to new arena. 2020-02-21 09:18:31 +01:00
Victor M. Alvarez 0f4ff68714 Start implementing the new arenas. 2020-02-20 09:57:13 +01:00
Victor M. Alvarez 2e58ae5601 Fix potential memory leaks in case of error. 2019-11-28 00:22:18 +01:00
Victor M. Alvarez 1e403e9259 Fix memory leak happening with certain invalid rules. 2019-11-17 13:53:47 +01:00
Victor M. Alvarez 2c0690c47b Iterators (#1141)
This introduces a change in the way in which "for-in" loops work by introducing the concept of iterators. These kind of loops now accept an iterator that will return a sequence of values. Besides the integer ranges (N..M) and integer enumerations (X,Y,Z), this kind of loops now accepts other kinds of iterables, like arrays. This allows conditions like the following one:

for any section in pe.sections: ( section.name == "foo" )

Until now the same expression was written as:

for any i in (0..pe.number_of_sections - 1): ( pe.sections[i].name == "foo" )

The new syntax is more legible and opens door for more powerful features in the future. Backward compatibility is maintained.
2019-11-12 16:15:41 +01:00
Wesley Shields ce470b7bd4 Improved xor modifier. (#1132)
* Implement improved xor modifier.

* Fix breakage when xor(...) came before another modifier.

* Add documentation for xor improvements.

* Fixes from review by Victor.

- No more changes to lexer.
- Duplicate modifiers are now errors.
- Do not rely on undocumented behavior in bison.

* Apply fix to deal with undocumented bison feature we shouldn't rely on.

* Make xor work properly with other modifiers.

This commit now makes it so that "xor(1-255) wide" will no longer search for the
plaintext wide version of the string, but "xor wide" will because it implies
xor(0-255).

* Add support for xor(X) and tests.
2019-09-18 17:12:46 +02:00
Wesley Shields 627ef691d7 Fixes for xor modifier. (#1113)
* Fixes for xor modifier.

This commit makes the xor modifier no longer include "plaintext" versions of the
strings. In particular if you have:

$a = "This program cannot" xor

Then it will no longer match on the plaintext version of the string. It will
only match on the xor versions. If you want it to match on plaintext or xor then
you can use:

$a = "This program cannot" xor ascii

The same is true for wide.

* Make 'xor nocase' a compiler error.

* Add newline between ascii and wide strings.
2019-09-03 10:59:47 +02:00
MatejKastak 6fcef6b36b compiler:Open include files allowing other readers (#1114) 2019-08-21 15:06:16 +02:00
Victor M. Alvarez abe41c14aa Make Coverity happy by handling the case in which ftell returns -1. 2019-03-21 20:26:18 +01:00
Victor M. Alvarez 3bbf39b22a Fix #1028 at the source of the problem.
Returning ERROR_INVALID_ARGUMENT from yr_ac_automaton_destroy is ok, but then we should do the same for all yr_ac_xxx functions for maintaining API coherence. Let's stick to the policy of expecting a non-null automaton in all yr_ac_xxx functions and fix the problem at its root.
2019-03-01 10:40:49 +01:00
Victor M. Alvarez ab6bc59c5b Protect yr_compiler_define_x_variable and yr_rules_define_x_variable against null pointers. 2018-12-05 16:17:57 +01:00
Victor M. Alvarez d545b69ca3 Closes #928. 2018-08-08 15:18:42 +02:00
Victor M. Alvarez c6f175a459 Declare variables before first statement in block as required Visual C++ 9.0. 2018-07-30 13:16:52 +02:00
Victor M. Alvarez 486be8e756 Revert e84ea0cc1c as Visual Studio 10 C compiler doesn't support.
Visual Studio 10 is used for compiling yara-python, so we are still stuck with C89.
2018-07-30 12:55:06 +02:00
Victor M. Alvarez 8dd3f45abb If <stdbool.h> is not present, fallback to our own definition of "bool" type. 2018-07-30 11:32:34 +02:00
Victor M. Alvarez e84ea0cc1c Declare loop variables in "for" statements as allowed by C99. 2018-07-30 10:21:03 +02:00
Victor M. Alvarez c00ba72e93 Fix some warnings. 2018-07-25 14:11:07 +02:00
Victor M. Alvarez 68d059d7a2 Fix bug in yr_compiler_define_integer_variable, it was setting a wrong variable type. 2018-07-25 13:23:55 +02:00
Victor M. Alvarez 96bb602f3f Refactoring of compiler's error handling. (#919)
This simplifies the compiler's error handling logic by removing the last_result field.
2018-07-24 18:17:55 +02:00
Victor M. Alvarez 27e385271f Revert "Simplify compiler errors (#912)"
This reverts commit 307ba3d719.
2018-07-13 16:08:17 +02:00
Victor M. Alvarez ef7d5ecb19 Closes #915.
Also makes sure that external variables defined with yr_compile_define_xxx_variable retain their values when rules are saved and loaded again from disk.
2018-07-12 18:15:06 +02:00
Victor M. Alvarez 307ba3d719 Simplify compiler errors (#912)
* Unify compiler's last_result and last_error fields into a single last_error field.

* Rename RE_LEX_ENVIRONMENT's last_error_code to last_error, for coherency with YR_COMPILER.
2018-07-12 10:58:47 +02:00
Victor M. Alvarez 7c1c663523 Closes #908. (#909)
Calling yr_compiler_define_xxx_variable twice with the same identifier now produces ERROR_DUPLICATED_EXTERNAL_VARIABLE.
2018-07-10 18:50:40 +02:00
Victor M. Alvarez cf44d39fe1 Add argument to yr_compiler_set_atom_quality_table and yr_compiler_load_atom_quality_table for specifying an atom quality warning threshold. 2018-07-09 08:57:16 +02:00
Victor M. Alvarez 1744ea70b0 Fix atom quality heuristic calculation. Rename atom prevalence table to atom quality table. 2018-07-09 08:42:40 +02:00
Victor M. Alvarez f11a08f42e Adjust atom quality heuristic. 2018-07-07 02:45:28 +02:00