* Simplifies some logic.
* Ensures that there will still be PDF results (for samples submitted as
PDF) and that the "no pdf metadata" error message remains shown.
* Unit test for #1980 and fixes some other unit tests.
The .LNK parser certainly could use some more bounds checking, but for
now let's keep it as-is.
In the case of Office DDE files the Static module will run a number of
Yara rules and Extractor plugins. In order to record this in our report,
the Extracted processing module should be ran afterwards.
This commit also fixes a regression with regards to not recording PID
and first_seen information in the ExtractScripts class anymore.
This commit introduces and improves a number of things related to static
analysis of malicious Microsoft Office files. In particular:
* Extend Extractor logic to allow for static-only analysis.
* Enhance Yara integration allowing it to take external variables,
perform on in-memory buffers, and work correctly with Office archives.
* Improve parsing logic of cmd.exe in our Scripting engine.
* More Extracted Artefacts logic and handling as well as versioning.
* Various tweaks and improvements.
This commit is mostly complete, however, it could certainly use more
error/exception handling. We should also more generically catch
exceptions thrown in Extractor plugins.
Provided that large parts of LinuxSystemTap() aren't covered yet
(including the "command_line"-related tweaks) we should await some
additional unit tests.
In the event of an empty .pcap file (i.e., only the header is present)
our sort_pcap() method doesn't actually get to writing a new pcap
(provided dump_sorted.pcap is only created when there are packets
available). This commit fixes a related warning that'd be thrown in that
case. Also fixes a NoneType issue introduced [1].
[1]: db38a5093c
Thanks to liuchuang for bringing up this issue a while ago and chigley
for providing a patch. This commit merges that patch and simplifies it
a little bit as well as providing some basic unit tests.
Introduces the concept of Yara rules for extracted scripts. We'll likely
also have a category for extracted shellcode soon. First to work on
recursive extraction and allowing 3rd party plugins to, e.g., extract a
shellcode stream from a PowerShell script (statically).
Raises an exception for ImportError's not related to Volatility which
is useful in case a future update decides to add another dependency
that's not installed by default in our setups.
Removes the attempt at trying to unittest importing Volatility as this
is non-trivial in Python - it has been tested manually, though.
Renames XenServerMachinery to XenServer because "XenServer".lower()
matches the configuration section required by the call to config2(),
all the other machineries already had the correct non-lower names.
* Make Static handle archives transparently.
* Take the "exe" package for granted just as the others.
* Move get_keys() to File where it's more appropriate.
In order to show proper information about the file that has been
analyzed, these two need each other. From now on the reports will be
able to share information (size, hashes, etc) about the actual file
being analyzed rather than the archive around it.
Introduces the concept of temporary files that are removed after the
object expires / is no longer used.
* Identify and report Host <> Guest connection issues and drop hints to
the user in both the command-line logs as well as the Web Interface.
* Provide FAQ entry for these issues.
* Only keep unique errors & actions in the report.
* Some other documentation tweaks (regarding formatting etc).
It's most certainly not necessary to read in configuration files once or
multiple times for each processed analysis. We strip down on re-reading
configuration files by, yet again, moving away from Config() usage to
the config() and config2() variants (which cache results).
* Use config2() to obtain cached versions of the configuration values
for running Auxiliary, Processing, and Reporting modules.
* Get rid of Config() usage in some processing modules.
* Unittests for all of the above, including a unittest which ensures
that Config() is only called twice when processing multiple analyses;
once for reading in processing.conf and once for reporting.conf.
Without these changes, it would be possible to have a dozens (although
for larger analyses this could be much, much more) configuration reads
per processed analysis.
Ricardo van Zutphen