the digest carried in a signature can be shorter than the digest algorithm's output size, so the memcmp in parse_authenticode and pkcs9_countersig_new reads past the stored digest; compare the lengths first, the way countersignature.c already does at its other three digest compares.
When the backward scan in pe_parse_rich_signature finds a DanS marker only one to three dwords ahead of the Rich marker, rich_len ends up smaller than RICH_SIGNATURE. The unsigned subtraction rich_len - sizeof(RICH_SIGNATURE) then wraps and rich_count becomes a huge bogus value that drives the version_data allocation and copy loop. _rich_version already rejects rich_length < sizeof(RICH_SIGNATURE) for the same computation, so apply the same check before computing rich_count here.
When the file imported DLLs that contained a dot (.) in its name, like `glib-2.0.dll`, the file extension was not properly removed, causing the the imphash to be wrong.
A check in `pe_parse_debug_directory` was verifying that the size of the debug directory entry was a multiple of `sizeof(IMAGE_DEBUG_DIRECTORY)`. Although this is true in most cases, is not universally true for all PE files. This check was preventing the extraction of the PDB path from certain EFI boot files, like the ones below. The PDB path in such cases is not a file path but a GUID, though.
Some affected files:
`6c2abf4b80a87e63eee2996e5cea8f004d49ec0c1806080fa72e960529cba14c`
`5487a2ff41e360b1d3b0a58e393b0041c3f269f1053a3e367dfecd48e233457e`
* LNK module initial commit
* Update atom logic to properly score [A-Z]
The documentation says `Bytes [a-zA-Z] contribute 18 points each`, but it looks like in the code that only `[a-z]` is given 18 points, whereas `[A-Z]` is given 20 points. This commit will make sure these ranges have the proper scoring as expected.
* Add test for comparing atom quality ABCD == abcd
Add test entry which compares the atom quality of "ABCD" and "abcd" and asserts that they are equal.
* Update test-atoms.c
* Revert "Update atom logic to properly score [A-Z]"
This reverts commit bf7130ff0b.
* Revert "Add test for comparing atom quality ABCD == abcd"
This reverts commit 181dbf2326.
* Revert "Update test-atoms.c"
This reverts commit 81239034b3.
* Test
* Revert "Test"
This reverts commit 1c53212b63.
* Add objects for LinkFlags parsing
Also fixed `DWORD` compiling error, and replaced with `uint32_t` types.
* Add file attribute flags
* Start hotkey parsing code
* Add the rest of the hotkey values
* Add initial test case for LNK module
Using the sample LNK from the MS standard for LNK files for testing, and putting in an initial test:
https://winprotocoldoc.blob.core.windows.net/productionwindowsarchives/MS-SHLLINK/%5bMS-SHLLINK%5d.pdf
* Add initial LNK tests
* Add has_hotkey variable
* Formatting
* Fix timestamps set
Previous version always seemed to be an hour (3600 seconds) off what it should have been. This commit adds a test to make sure it's getting the right value.
I don't know why its always an hour off, but this should fix it!
* Refactor hotkey char select into its own function
* Add a couple of comments
* Add min size for parsing LNK
All the mandatory LNK header bytes add up to 76 bytes, and as such we won't parse an LNK file unless it is at least 76 bytes in length
* Add LinkTargetIDList parsing
* Begin working on code for LinkInfo parsing
* Refactor code into header + util files
Moving a lot of the definitions over to a separate header file to keep the main code clean.
Also following the PE module's structure of having a separate `lnk_utils` file to deal with some of the convenient functions it provides.
* Starting VolumeID parsing
* Adding free statement
* Add some VolumeID tests and fix free error
* Finish parsing VolumeID
* Parse LocalBasePath
* Refactor to prevent unsafe memory copying
Start taking into account the remaining size in the block being scanned, to make sure we don't break anything when dereferencing pointers.
* Parse CommonPathSuffix
Also create empty function for when we need to parse CommonNetworkRelativeLink
* Parse LocalBasePathUnicode and CommonPathSuffixUnicode
NOTE: these will need some testing before this will be production ready, as I'm making a lot of assumptions here.
* Return size of LinkInfo section
* Add StringData parsing
* Add initial code for parsing ExtraData section
Only parsing `TrackerDataBlock` at the moment, but setting up values in advance of setting up the other structures/parsers
* Add the remaining ExtraData structures
In some cases I don't make a structure where the parsing function will just handle the data (e.g. where there is variable data).
I will likely also ignore PropertyStore for now as that has separate documentation to parse it.
* More specific parsing of Droid GUIDs
* Move code out of VolumeID parsing indent
The rest of the LinkInfo section wasn't parsing unless a VolumeID was present, which shouldn't be how it works.
* Remove incorrect CommonNetworkRelativeLinkOffset setting
Also add structure for CommonNetworkRelativeLink parsing
* Implement CommonNetworkRelativeLink parsing
And add a couple of test cases for it
* Formatting
* Make LinkInfo parsing memory safe
* Remove unnecessary constant
* Make StringData parsing memory safe
Also convert `block_data_size_remaining` to `int` from `size_t`. This is because we don't want to underflow the `size_t` type, and assume we have more data left than we actually do.
* Fix up block data size remaining calculations in ExtraData
* Add more constants
* Add in overlay variable
* Make current ExtraData function memory safe
* Setup switch/case for ExtraData parsing
* ConsoleDataBlock parsing
* Add ConsoleFedData parsing
* Convert tracker data to struct and add more fields
* Add extra fields to current ExtraData structures
* Refactor structures
* Rename sample LNKs
* Refactor into parse_volume_id
* Parse DarwinData and fixed up block struct entries
* Add is_malformed entry
Checks whether a LNK has failed parsing due to it being malformed.
* Add overlay test case
* Parse EnvironmentVariableDataBlock
* Parse IconEnvironmentDataBlock
* Add KnownFolderDataBlock parsing
* Add placeholder parsing for PropertyDataStoreBlock
* Parse ShimDataBlock
* Parse SpecialFolderDataBlock
* Remove id_list_ptr var
* Refactor into its own function to parse IDList
* Parse VistaAndAboveIDListDataBlock
* Remove printf
* Refactor item_id_list into structure
* Add LinkInfoFlags as external variables
* Only process relevant link info if flags set
* Update StringData to also accept ASCII
I assumed it would only be ASCII, but the `IsUnicode` flag can help indicate what type of string it is.
* Only parse ExtraData if there is data left
* Retype to strings
* Fix environment var data parsing
I thought wchar_t was 16 bits, but turns out it can also be 32 bits... I'll need to change up other routines to take this into account as well
* Move inside if statement to prevent setting
* Fix icon env unicode issue
* Convert remaining wchar_t struct entries
* Add ExtraData test cases
* Add ExtraData tests (2)
* Make sure timestamps are UTC
* Start LNK module documentation
* Add ShowCommand constants
* Finish documenting LNK headers
* Change test to flag comparison
* Correct structure names
* Update test-lnk.c
* Document LinkInfo section
* Document StringData
* Add more constants
* Document ConsoleData
* Update lnk.rst
* Finish documenting LNK module
* Some extra comments
* Update lnk.rst
* Apply clang formatting
* Removed test for now
* Add more flags and renamed console_fe_data
* Formatting
* Update comment on PropertyStoreData
* Change variable names due to potential clashes
Changed the variables used in header files:
- Reserved1
- Reserved2
- Unused1
- Unused2
The CI/CD seems to fail with these values as a default, so I'm hoping changing them fixes things
* Change packing to address misaligned pointer error
* Add LNK module to VS Projects
* Changed `block_data_size_remaining` to `unsigned int`
Make variable `unsigned int` instead of just `int` to prevent the value being negative.
* Begin removing static of strings
Starting with the ones I can test, by dynamically allocating memory after doing the check, I'm removing the possibility of there being a buffer overflow on this data.
* Dynamically allocate `volume_id_data`
* Remove more statically typed char arrays
* Dynamically allocate unicode strings
Previously were statically set in length, now dynamically allocated
* Add clearer comment
* Add todo
* Add `yr_` prefix to necessary functions
Due to #1762, needed to add `yr_` as a prefix to functions like `set_string`
* Convert 'unsigned int' to `uint32_t`
* Make constants have consistent casing
* Add extra checks on block size remaining
I found a case where a function returned a section size that was greater than the remaining block size, which caused a crash. This patch checks each value returned to make sure the next stage doesn't fail to parse.
* Fix docs for icon_location
Add `lnk.` prefix to `icon_location` variable
* Add LNK module back in
* Correct LNK module source path
* Add lnk_utils to modules list
And update other libyara auto changes
* Add LNK module tests
* Fix up LNK test in Makefile
* Fix typo in test LNK
* Attempt to fix bazel build
* Testing to see why Bazel build fails
* Revert removing tests
It seems even one test gives the error `There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.`
* Include LNK module in BUILD.bazel
* Add missing deps line
* Add lnk.h to Makefile
* Add lnk.h to yara.bzl
* Add lnk_utils.h to yara.bzl
* Revert automatic changes
Change some C/header files that got automatically changed while compiling YARA to the same that they are on the main branch
* Include lnk_utils in Makefile
* Convert some console data variables to signed and update docs
Some variables in the console data section are actually signed variables. So I've converted them to `int16_t` instead of `uint16_t`, added some test cases to make sure these values are properly parsed, and added some examples in the docs.
* Fix typo for window_size_y
* Add flags for LNK module
* Add extra time tests
Add tests for access time and write time
* Temporary update for debugging
Attempting to update this temporarily in order to test big endian issues on this branch - revert before merging
* Revert "Temporary update for debugging"
This reverts commit 2120c7e2e5.
* Temporary change for debugging
* Change order
* Temporarily disable most tests
* Initial big endian test fix
* LNK header endian fix
* Update test-lnk.c
* Fix up time conversion function calls
* Re-enable item ID tests
* Focus on item_id_list_size test
* Fix endianness of parse_link_target_id_list
* Fix endianness of parse_id_list
* Fix endianness of link_info
* Finish applying endianness fixes
Enabled all tests now
* Roll back to test console data and string data
* Add missing endianness conversion
* Re-enable all tests
* Revert back to original state
* LNK module initial commit
* Update atom logic to properly score [A-Z]
The documentation says `Bytes [a-zA-Z] contribute 18 points each`, but it looks like in the code that only `[a-z]` is given 18 points, whereas `[A-Z]` is given 20 points. This commit will make sure these ranges have the proper scoring as expected.
* Add test for comparing atom quality ABCD == abcd
Add test entry which compares the atom quality of "ABCD" and "abcd" and asserts that they are equal.
* Update test-atoms.c
* Revert "Update atom logic to properly score [A-Z]"
This reverts commit bf7130ff0b.
* Revert "Add test for comparing atom quality ABCD == abcd"
This reverts commit 181dbf2326.
* Revert "Update test-atoms.c"
This reverts commit 81239034b3.
* Test
* Revert "Test"
This reverts commit 1c53212b63.
* Add objects for LinkFlags parsing
Also fixed `DWORD` compiling error, and replaced with `uint32_t` types.
* Add file attribute flags
* Start hotkey parsing code
* Add the rest of the hotkey values
* Add initial test case for LNK module
Using the sample LNK from the MS standard for LNK files for testing, and putting in an initial test:
https://winprotocoldoc.blob.core.windows.net/productionwindowsarchives/MS-SHLLINK/%5bMS-SHLLINK%5d.pdf
* Add initial LNK tests
* Add has_hotkey variable
* Formatting
* Fix timestamps set
Previous version always seemed to be an hour (3600 seconds) off what it should have been. This commit adds a test to make sure it's getting the right value.
I don't know why its always an hour off, but this should fix it!
* Refactor hotkey char select into its own function
* Add a couple of comments
* Add min size for parsing LNK
All the mandatory LNK header bytes add up to 76 bytes, and as such we won't parse an LNK file unless it is at least 76 bytes in length
* Add LinkTargetIDList parsing
* Begin working on code for LinkInfo parsing
* Refactor code into header + util files
Moving a lot of the definitions over to a separate header file to keep the main code clean.
Also following the PE module's structure of having a separate `lnk_utils` file to deal with some of the convenient functions it provides.
* Starting VolumeID parsing
* Adding free statement
* Add some VolumeID tests and fix free error
* Finish parsing VolumeID
* Parse LocalBasePath
* Refactor to prevent unsafe memory copying
Start taking into account the remaining size in the block being scanned, to make sure we don't break anything when dereferencing pointers.
* Parse CommonPathSuffix
Also create empty function for when we need to parse CommonNetworkRelativeLink
* Parse LocalBasePathUnicode and CommonPathSuffixUnicode
NOTE: these will need some testing before this will be production ready, as I'm making a lot of assumptions here.
* Return size of LinkInfo section
* Add StringData parsing
* Add initial code for parsing ExtraData section
Only parsing `TrackerDataBlock` at the moment, but setting up values in advance of setting up the other structures/parsers
* Add the remaining ExtraData structures
In some cases I don't make a structure where the parsing function will just handle the data (e.g. where there is variable data).
I will likely also ignore PropertyStore for now as that has separate documentation to parse it.
* More specific parsing of Droid GUIDs
* Move code out of VolumeID parsing indent
The rest of the LinkInfo section wasn't parsing unless a VolumeID was present, which shouldn't be how it works.
* Remove incorrect CommonNetworkRelativeLinkOffset setting
Also add structure for CommonNetworkRelativeLink parsing
* Implement CommonNetworkRelativeLink parsing
And add a couple of test cases for it
* Formatting
* Make LinkInfo parsing memory safe
* Remove unnecessary constant
* Make StringData parsing memory safe
Also convert `block_data_size_remaining` to `int` from `size_t`. This is because we don't want to underflow the `size_t` type, and assume we have more data left than we actually do.
* Fix up block data size remaining calculations in ExtraData
* Add more constants
* Add in overlay variable
* Make current ExtraData function memory safe
* Setup switch/case for ExtraData parsing
* ConsoleDataBlock parsing
* Add ConsoleFedData parsing
* Convert tracker data to struct and add more fields
* Add extra fields to current ExtraData structures
* Refactor structures
* Rename sample LNKs
* Refactor into parse_volume_id
* Parse DarwinData and fixed up block struct entries
* Add is_malformed entry
Checks whether a LNK has failed parsing due to it being malformed.
* Add overlay test case
* Parse EnvironmentVariableDataBlock
* Parse IconEnvironmentDataBlock
* Add KnownFolderDataBlock parsing
* Add placeholder parsing for PropertyDataStoreBlock
* Parse ShimDataBlock
* Parse SpecialFolderDataBlock
* Remove id_list_ptr var
* Refactor into its own function to parse IDList
* Parse VistaAndAboveIDListDataBlock
* Remove printf
* Refactor item_id_list into structure
* Add LinkInfoFlags as external variables
* Only process relevant link info if flags set
* Update StringData to also accept ASCII
I assumed it would only be ASCII, but the `IsUnicode` flag can help indicate what type of string it is.
* Only parse ExtraData if there is data left
* Retype to strings
* Fix environment var data parsing
I thought wchar_t was 16 bits, but turns out it can also be 32 bits... I'll need to change up other routines to take this into account as well
* Move inside if statement to prevent setting
* Fix icon env unicode issue
* Convert remaining wchar_t struct entries
* Add ExtraData test cases
* Add ExtraData tests (2)
* Make sure timestamps are UTC
* Start LNK module documentation
* Add ShowCommand constants
* Finish documenting LNK headers
* Change test to flag comparison
* Correct structure names
* Update test-lnk.c
* Document LinkInfo section
* Document StringData
* Add more constants
* Document ConsoleData
* Update lnk.rst
* Finish documenting LNK module
* Some extra comments
* Update lnk.rst
* Apply clang formatting
* Removed test for now
* Add more flags and renamed console_fe_data
* Formatting
* Update comment on PropertyStoreData
* Change variable names due to potential clashes
Changed the variables used in header files:
- Reserved1
- Reserved2
- Unused1
- Unused2
The CI/CD seems to fail with these values as a default, so I'm hoping changing them fixes things
* Change packing to address misaligned pointer error
* Add LNK module to VS Projects
* Changed `block_data_size_remaining` to `unsigned int`
Make variable `unsigned int` instead of just `int` to prevent the value being negative.
* Begin removing static of strings
Starting with the ones I can test, by dynamically allocating memory after doing the check, I'm removing the possibility of there being a buffer overflow on this data.
* Dynamically allocate `volume_id_data`
* Remove more statically typed char arrays
* Dynamically allocate unicode strings
Previously were statically set in length, now dynamically allocated
* Add clearer comment
* Add todo
* Add `yr_` prefix to necessary functions
Due to #1762, needed to add `yr_` as a prefix to functions like `set_string`
* Convert 'unsigned int' to `uint32_t`
* Make constants have consistent casing
* Add extra checks on block size remaining
I found a case where a function returned a section size that was greater than the remaining block size, which caused a crash. This patch checks each value returned to make sure the next stage doesn't fail to parse.
* Fix docs for icon_location
Add `lnk.` prefix to `icon_location` variable
* Add LNK module back in
* Correct LNK module source path
* Add lnk_utils to modules list
And update other libyara auto changes
* Add LNK module tests
* Fix up LNK test in Makefile
* Fix typo in test LNK
* Attempt to fix bazel build
* Testing to see why Bazel build fails
* Revert removing tests
It seems even one test gives the error `There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.`
* Include LNK module in BUILD.bazel
* Add missing deps line
* Add lnk.h to Makefile
* Add lnk.h to yara.bzl
* Add lnk_utils.h to yara.bzl
* Revert automatic changes
Change some C/header files that got automatically changed while compiling YARA to the same that they are on the main branch
* Include lnk_utils in Makefile
* Convert some console data variables to signed and update docs
Some variables in the console data section are actually signed variables. So I've converted them to `int16_t` instead of `uint16_t`, added some test cases to make sure these values are properly parsed, and added some examples in the docs.
* Fix typo for window_size_y
* Add flags for LNK module
* Add extra time tests
Add tests for access time and write time
* Fix imphash issue with empty import names.
If an import has an empty name skip processing it. This is consistent with the
behavior of pefile (https://github.com/erocarrera/pefile/blob/593d094e35198dad92aaf040bef17eb800c8a373/pefile.py#L5871-L5872).
Add a test case which is just the tiny test file with the first import name set
to all NULL bytes. I tested that pefile calculates the imphash of it, which
matches what YARA now calculates too:
>>> import pefile
>>> pe = pefile.PE('/Users/wxs/src/yara/tests/data/tiny_empty_import_name')
>>> pe.get_imphash()
'0eff3a0eb037af8c1ef0bada984d6af5'
>>>
Fixes#1943
* Add test file forgot in last commit.
* Handle invalid import names.
If an imported function does not contain ONLY a-zA-Z0-9 and a small subset of
special characters it will now be ignored. This aligns us better with pefile,
which checks for valid import names and skips them if they are invalid.
I've also updated the test file to check that these special characters are
handled properly.
* Fix test after alignment with pefile.
Turns out that the tiny-idata-5200 file is corrupted to the point that pefile
doesn't parse it. For example, it finds no imports to hash:
```
>>> pe = pefile.PE('tests/data/tiny-idata-5200')
>>> pe.get_imphash()
''
>>>
```
We were parsing imports from this file before these changes that were incorrect,
so fix the tests to reflect the fact that we parse no imports from this file
anymore.
As part of this I've split the checks for number of parsed imports and
successfully parsed imports into two different counters, which now means we are
accurately reflecting when we are able to parse the import table but not the
descriptors in it while still making sure we don't parse too many as we have
seen before.
* Move declarations to align better.
Move these declaractions so they are with the rest of the constants. This makes
the output of -D easier to read.
* Fix memory leak when handling corrupt imports.
If we have an invalid import name we need to free the name and continue to the
next thunk and function. While fixing this I noticed that if we fail to alloc an
IMPORT_FUNCTION* we would end up looping endlessly because we were never
incrementing the thunk pointer or function index. Fix it by ALWAYS incrementing
those at the end of the loop and conditionally populating the newly allocated
node.
The telfhash algorithm excludes x64 functions, see here:
https://github.com/trendmicro/telfhash/blob/d2a82f85a5b/telfhash/telfhash.py#L59
Looks like there was a copy paste error when porting this algorithm in
YARA, as the test for "is_bad_prefix" is copied, but this is not the
right test of "is_x86_64".
This PR properly fixes it.
* Fix `pe_rva_to_offset`
* Fix: checking if RVA is inside section
* Fix: real pointer to raw data is aligned down to sector size
* Add pe.import_rva() functions.
Add pe.import_rva("foo.dll", "func1") which returns the RVA of the imported
function. Also add pe.import_rva("foo.dll", 1) which does the same but the
import is done by ordinal.
* Implement delayed import RVA and add docs.
* Add math.length()
Add a math.length() which will return the length of the sequence of bytes,
including any NULL bytes.
Fixes#1778.
* Add string module.
Move the math.to_int() functions and math.length() over to the new string
module. I decided to move the to_int() because it seems logical to convert from
a string to an integer using a string module rather than the math module.
You still use math.to_string() to convert an integer to a string, and use
string.to_int() to convert a string to an integer.
Add tests and docs for string module. Move the appropriate tests from the math
module over to the new string tests.
While here, also add the console module to the bazel build as it was apparently
missing (this is untested).
* Remove unused include.
* Add tests for https://github.com/VirusTotal/yara/pull/1561/
* Fix copyright year in string module.
Co-authored-by: Peter Babka <[email protected]>
* Fix issue #1443.
As pointed out in the bug report the way of aligning the win_cert pointer to the next 8-bytes boundary is not correct. If the previous certificates ends at 0x1234000A, the next one should start at 0x12340010, not at 0x1234000c which is not aligned to 8 bytes.
When we hit the maximum match limit on a string we currently abort all scanning,
which can be problematic since it isn't a good reason to stop all scanning, and
is better off as a warning to the user.
This commit makes it so that when the string match limit is hit a warning is
printed and scanning will continue. The warning message indicates that the
results may be invalid. It is hard to know for certain if they are invalid
because it depends entirely upon the condition. If you use the # or @ operator
then your condition _MAY_ be invalid since the total number of matches and
locations is not known anymore. Just checking that the string exists won't be an
invalid state since we have confirmed the existence of _AT LEAST_ the maximum
number of occurrences, which clearly satisfies the existence of a single
instance of your string.
I decided to implement this as a callback so the user of libyara has some choice
on the default behavior. If they want to consider this a "stop all scanning"
error they are free to return CALLBACK_ERROR in their callback.
The current Mach-O parsing code attempts to call macho_rva_to_offset
while parsing load commands and before it has set number_of_segments.
Since macho_rva_to_offset uses number_of_segments this can result
in it using a number_of_segments that is uninitialized and is just a random
large number. In turn yara runs a really long time.
This adds a test binary based on this post
https://codegolf.stackexchange.com/questions/102471/smallest-possible-runnable-mach-o-executable/154685#154685
It has two load commands. The first is a LC_UNIXTHREAD command and
the second is a LC_SEGMENT command. It's a valid 386 Mach-O binary.
It reproduces this error.
UPX packed files will cause export parsing to stop parsing because the function
address is not a valid RVA, and I was refusing to parse that export in that
case.
This is a regression from how we used to do parsing (we were never checking the
export RVA so it was never considered in prior releases). This is causing rules
which used to work to no longer work on packed samples. Specifically, checking
for a DLL which is UPX packed using pe.exports("foo") is likely to not work in
4.0.0 when it used to work in 3.11.0.
Apologies for missing this, but to make sure it doesn't happen again in the
future I've added a test case for this (just packed an existing DLL that is in
the repo).
Add test for invalid use of base64 and fullword together
Skip second base64 encoding of single ascii byte
Add test for correct base64 encoding of single ascii byte
Add two new modifiers: base64 and widebase64.
These modifiers take the given text string and generate 3 different strings
being careful to trim off the bytes which are dependent upon leading or
trailing bytes in the larger search space.
I've implemented it by slightly cheating. I generate all the search strings in
a list and then creating one large string suitable for the RE compiler to deal
with. For example, the string "This program cannot" generates these three
base64 encoded strings:
VGhpcyBwcm9ncmFtIGNhbm5vd
RoaXMgcHJvZ3JhbSBjYW5ub3
UaGlzIHByb2dyYW0gY2Fubm90
Those three strings are then transformed into a RE that looks like this:
(VGhpcyBwcm9ncmFtIGNhbm5vd|RoaXMgcHJvZ3JhbSBjYW5ub3|UaGlzIHByb2dyYW0gY2Fubm90)
That string is then passed to the RE compiler for parsing and AST generation.
The AST is then emitted into the appropriate spot and YARA believes it was
given a regex from that point on.
I've also implemented support for specifying custom alphabets:
base64("!@#$%^&*(){}[].,|ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstu")
This means that I have to be careful to escape any special RE characters in the
string passed to the compiler. The base64 alphabet has to be 64 bytes long, and
does support escaped bytes properly too.
To avoid the need to deal with escaping I had a first implementation which
attempted to generate the AST by hand, which was mostly working but was very
cumbersome to maintain. In doing this I ended up improving yr_re_print_node()
so that it indents the tree properly, which made debugging that attempt easier.
* Fix Rich header parsing.
The rich header doesn't always start at 0x80. The proper way to find the rich
header is to start at the NT header and work backwards one dword at a time
(technically the NT header is supposed to be 8 byte aligned, if someone has a
sample which isn't I'd love to see it) looking for Rich. Once you find it you
can get the XOR key which follows it and then continue to look backwards for
the DanS (after applying the XOR).
Decent documentation on this is at
http://bytepointer.com/articles/the_microsoft_rich_header.htm.
Also, fun fact which we may want to consider implementing is that the XOR key is
actually derived from the DOS header, so it may be an interesting way to tell if
the rich header has been tampered with if they do not properly recalculate the
key.
Fixes#1149.
* Split Rich signature walking into two loops.
While here, add a test that makes sure the clear version of the rich signature
is properly calculated.
* Fixes for xor modifier.
This commit makes the xor modifier no longer include "plaintext" versions of the
strings. In particular if you have:
$a = "This program cannot" xor
Then it will no longer match on the plaintext version of the string. It will
only match on the xor versions. If you want it to match on plaintext or xor then
you can use:
$a = "This program cannot" xor ascii
The same is true for wide.
* Make 'xor nocase' a compiler error.
* Add newline between ascii and wide strings.
Add some tests for the dotnet module, which uses the smallest .NET binary I had
laying around that exercises a fair bit of the code. In particular it has a
resource, guids, assembly information and a couple of other things that make for
good tests.
Add a bunch of tests which exercise improved DLL parsing. The DLL I'm using for
testing was taken from a Windows 10 VM I had laying around and was chosen
because it was the smallest DLL I had which has at least one local function and
one forwarded function.
Also, I'm glad I wrote these tests because they exposed a copy/paste mistake in
exports_index_regex where I was returning 1 and not the index where the regex
matched.
jmestwa-coder