gecko-dev/third_party/rust/goblin
Markus Stange c6fc1caf62 Bug 1457481 - Run mach vendor rust. r=froydnj,erahm
Most importantly, this picks up "object" and "goblin" for ELF binary parsing.
We only use the ELF code from goblin, so the mach-O parsing code gets
eliminated by the linker. Overall, this increases the Android installer size
by 20KB.

Try pushes for reference:
before: https://treeherder.mozilla.org/#/jobs?repo=try&revision=834b56dc5ab3d63a43a32f740ee8212296ac726d&selectedJob=201600899
after: https://treeherder.mozilla.org/#/jobs?repo=try&revision=6983b27e8d3cb715d3b7e6cbd276683f6466e3cc&selectedJob=201600475

installer size: 34524820 -> 34542861 (34.52MB -> 34.54MB)

$ mach vendor rust
    Updating registry `https://github.com/rust-lang/crates.io-index`
      Adding goblin v0.0.17
      Adding memmap v0.6.2
      Adding miniz-sys v0.1.10
      Adding object v0.10.0
      Adding parity-wasm v0.31.3
      Adding plain v0.2.3
      Adding profiler_helper v0.1.0 (file:///Users/mstange/code/mozilla/tools/profiler/rust-helper)
      Adding scroll v0.9.1
      Adding scroll_derive v0.9.5
      Adding syn v0.15.5
      Adding thin-vec v0.1.0
      Adding uuid v0.6.5
 0:30.11 The following files exceed the filesize limit of 102400:

third_party/rust/miniz-sys/miniz.c
third_party/rust/syn-0.14.6/src/expr.rs
third_party/rust/syn-0.14.6/src/gen/fold.rs
third_party/rust/syn-0.14.6/src/gen/visit.rs
third_party/rust/syn-0.14.6/src/gen/visit_mut.rs

The syn dependency is not compiled for goblin, as far as I can tell - it's only
needed for the 'syn' feature of scroll_derive, and scroll does not ask for
scroll_derive/syn.

object -> goblin -> scroll -> scroll_derive -/-> syn

But it looks like other versions of syn were already in the tree.

Depends on D7021

Differential Revision: https://phabricator.services.mozilla.com/D7023

--HG--
rename : third_party/rust/syn/src/parsers.rs => third_party/rust/syn-0.14.6/src/parsers.rs
rename : third_party/rust/syn/src/verbatim.rs => third_party/rust/syn-0.14.6/src/verbatim.rs
rename : third_party/rust/uuid/.travis.yml => third_party/rust/uuid-0.5.1/.travis.yml
rename : third_party/rust/uuid/src/rustc_serialize.rs => third_party/rust/uuid-0.5.1/src/rustc_serialize.rs
rename : third_party/rust/uuid/src/serde.rs => third_party/rust/uuid-0.5.1/src/serde.rs
extra : moz-landing-system : lando
2018-10-02 01:50:56 +00:00
..
etc Bug 1457481 - Run mach vendor rust. r=froydnj,erahm 2018-10-02 01:50:56 +00:00
examples Bug 1457481 - Run mach vendor rust. r=froydnj,erahm 2018-10-02 01:50:56 +00:00
src Bug 1457481 - Run mach vendor rust. r=froydnj,erahm 2018-10-02 01:50:56 +00:00
tests Bug 1457481 - Run mach vendor rust. r=froydnj,erahm 2018-10-02 01:50:56 +00:00
.cargo-checksum.json Bug 1457481 - Run mach vendor rust. r=froydnj,erahm 2018-10-02 01:50:56 +00:00
Cargo.toml Bug 1457481 - Run mach vendor rust. r=froydnj,erahm 2018-10-02 01:50:56 +00:00
CHANGELOG.md Bug 1457481 - Run mach vendor rust. r=froydnj,erahm 2018-10-02 01:50:56 +00:00
LICENSE Bug 1457481 - Run mach vendor rust. r=froydnj,erahm 2018-10-02 01:50:56 +00:00
README.md Bug 1457481 - Run mach vendor rust. r=froydnj,erahm 2018-10-02 01:50:56 +00:00

libgoblin Build Status Current Crates.io Version

say the right words

Documentation

https://docs.rs/goblin/

changelog

Usage

Goblin requires rustc 1.19.

Add to your Cargo.toml

[dependencies]
goblin = "0.0.17"

Features

  • awesome crate name
  • zero-copy, cross-platform, endian-aware, ELF64/32 implementation - wow!
  • zero-copy, cross-platform, endian-aware, 32/64 bit Mach-o parser - zoiks!
  • PE 32/64-bit parser - bing!
  • a Unix and BSD style archive parser (latter courtesy of @willglynn) - huzzah!
  • many cfg options - it will make your head spin, and make you angry when reading the source!
  • fuzzed - " I am happy to report that goblin withstood 100 million fuzzing runs, 1 million runs each for seed 1~100." - @sanxiyn
  • tests

libgoblin aims to be your one-stop shop for binary parsing, loading, and analysis.

Use-cases

Goblin primarily supports the following important use cases:

  1. Core, std-free #[repr(C)] structs, tiny compile time, 32/64 (or both) at your leisure

  2. Type punning. Define a function once on a type, but have it work on 32 or 64-bit variants - without really changing anything, and no macros! See examples/automagic.rs for a basic example.

  3. std mode. This throws in read and write impls via Pread and Pwrite, reading from file, convenience allocations, extra methods, etc. This is for clients who can allocate and want to read binaries off disk.

  4. Endian_fd. A truly terrible name 😆 this is for binary analysis like in panopticon or falcon which needs to read binaries of foreign endianness, or as a basis for constructing cross platform foreign architecture binutils, e.g. cargo-sym and bingrep are simple examples of this, but the sky is the limit.

Here are some things you could do with this crate (or help to implement so they could be done):

  1. write a compiler and use it to generate binaries (all the raw C structs have Pwrite derived)
  2. write a binary analysis tool which loads, parses, and analyzes various binary formats, e.g., panopticon or falcon
  3. write a semi-functioning dynamic linker
  4. write a kernel and load binaries using no_std cfg. I.e., it is essentially just struct and const defs (like a C header) - no fd, no output, no std.
  5. write a bin2json tool (http://github.com/m4b/bin2json), because why shouldn't binary formats be in JSON?

Cfgs

libgoblin is designed to be massively configurable. The current flags are:

  • elf64 - 64-bit elf binaries, repr(C) struct defs
  • elf32 - 32-bit elf binaries, repr(C) struct defs
  • mach64 - 64-bit mach-o repr(C) struct defs
  • mach32 - 32-bit mach-o repr(C) struct defs
  • pe32 - 32-bit PE repr(C) struct defs
  • pe64 - 64-bit PE repr(C) struct defs
  • archive - a Unix Archive parser
  • endian_fd - parses according to the endianness in the binary
  • std - to allow no_std environments

Contributors

Thank you all ❤️ !

In alphabetic order:

Contributing

  1. Please prefix commits with the affected binary component; the more specific the better, e.g., if you only modify relocations in the elf module, then do "elf.reloc: added new constants for Z80"
  2. Commit messages must explain their change, no generic "changed", or "fix"; if you push commits like this on a PR, be aware @m4b or someone will most likely squash them.
  3. If you are making a large change to a module, please raise an issue first and lets discuss; I don't want to waste your time if its not a good technical direction, or etc.
  4. If your PR is not getting attention, please respond to all relevant comments raised on the PR, and if still no response, ping @m4b, @philipc, or @willglyn in github and also feel free to email @m4b.
  5. Please add tests if you are adding a new feature. Feel free to add tests even if you are not, tests are awesome and easy in rust.
  6. Once cargo format is officially released, please format your patch using the default settings.