Running the suite on GCC Compile Farm machines found test scripts that
don't work as intended on some hosts:
- on_qemu() took KVM guests on POWER for qemu-user
- the native POWER10 target ignored CPU and shared ppc64le's directory
- readelf 2.35 appends "[<localentry>: 1]" to POWER10 function names
- compressed debug sections hid .debug_str from `readelf -p`
- some tests assumed Linux, glibc, GCC-only flags or g++
- an x86-64 executable was run without $QEMU on a non-x86 host
- old binutils lack multiple .symver and SHF_LINK_ORDER
- GCC 8's `gcc -r` links start files into relocatable output
- debug-macro-section.sh grepped for an offset format objdump doesn't print
With -g3, GCC emits a COMDAT group for the .debug_macro contents of each
header file, and other .debug_macro sections refer to it. mold doesn't
deduplicate these groups because it would resolve references to discarded
copies to 0 (https://github.com/rui314/mold/issues/438), but -r output
dropped the groups while keeping SHF_GROUP on their members. Now -r keeps
every copy as a COMDAT group. GNU ld -r deduplicates them and redirects
the references to the kept copies; without that redirection, mold leaves
deduplication to the final link, where GNU ld, lld and mold handle the
groups as they would in the original objects.
--print-dependencies reads the relocations of every input section,
including non-allocated ones whose CREL tables are not decoded when the
file is read. It read them as an array, which reinterpreted such a table
as RELA records and failed an assertion. C++ mold's get_rels() decoded
the table on demand instead.
A read-only pass doesn't need the table materialized, so iterate with
the streaming relocation iterator as --stats does, and document when
the array accessor can be used.
rustc's LoongArch targets use the medium code model, whose calls reach
±128 GiB, but the cc crate passes no -mcmodel to the C compiler unless
-Ccode-model is given explicitly, so C code got the normal code model,
whose direct calls reach only ±128 MiB. A debug build of mold has more
than 128 MiB of text, so calls from C code such as zstd, mimalloc and
lto-message.c to the PLT failed to link with R_LARCH_B26 overflows on
a native LoongArch host. We now give -Ccode-model=medium explicitly.
A section group whose flag word is 0 is not a COMDAT group. It is never
deduplicated; it only ties its members together so that they are kept or
discarded as a unit. annobin emits such groups, e.g. into crt1.o on
AlmaLinux 8. mold ignored them, but -r output kept SHF_GROUP on their
members, leaving SHF_GROUP sections that no SHT_GROUP lists, which the
gABI forbids and objdump 2.30 rejects. -r output now keeps each such
group with flag word 0, as GNU ld and lld do. Other links still treat
its members as ordinary sections.
CREL tables of non-allocated sections are not decoded when an object
file is read; their relocations are streamed when they are applied.
On ppc64v1, rewrite_opd() also rewrites relocations of non-allocated
sections such as .debug_info, and rels_mut() reinterpreted such a table
as RELA records. Linking a CREL object built with -g failed an
assertion, and a table whose size was a multiple of 24 would be misread.
A CREL table can't be rewritten in its compressed form, so rels_mut()
now decodes it into the side table first, as get_rels() in C++ mold
did. relocation_iter() then returns the rewritten records from there.
This reverts aed5a38d1, which made -e accept a number as an entry
address if no symbol has that name, as GNU ld and lld do. New features
are deferred until after 3.0.0.
This reverts f03b74595, 48428d389 and 3a1b9c6c2, which merged
GNU_PROPERTY_AARCH64_FEATURE_1_AND, emitted BTI PLTs and landing pads
for range extension thunks, and added -z force-bti, pac-plt, gcs,
bti-report and gcs-report.
GCC 16 no longer records -mbranch-protection in .note.gnu.property but
in AArch64 build attributes, which we don't read, so outputs silently
lost the BTI, PAC and GCS bits and -z force-bti warned about every
object file. We'll bring the feature back with build attribute support
after 3.0.0. Until then, ARM64 outputs have no .note.gnu.property, as
in 2.42.1.
These sorts have keys that are unique among the elements, so a stable
sort produces the same order. Using an unstable sort, as C++ mold did,
avoids the stable sort's buffer and signals that ties don't matter.
Two mapping symbols can share an offset, e.g. when `ld -r` places a
section ending with a mapping symbol before one starting with another.
The later one in the symbol table applies, as in the C++ version, which
used stable_sort. An unstable sort could leave BE8 code unswapped.
Default features are turned off because uuid's std feature refers to
optional wasm32-only crates, and Cargo puts those into Cargo.lock even
though nothing builds them. With them off, uuid is the only new package.
When run in an extracted source archive, dist.sh cloned the release tag
as a bare repository into .git to get the commit hash and timestamp.
Cargo refuses to build in a directory whose .git is a bare repository,
and the embedded hash named the tag rather than the files actually
built. A worktree doesn't work either, because its .git file points
outside the build container and the hash is silently omitted. Stop with
an error unless .git is a directory.
Non-allocated strings were merged in a background Rayon task, but Rayon
has no task priorities: a thread waiting for a foreground parallel pass
could pick up a long piece of the merge, and the pass waited up to
200 ms for it. Resolve these sections in compute_section_sizes as before
2316c952a. Within noise on large debug builds (0.94-1.04).
Instruction pairs such as ADRP+ADD on ARM64 refer to the same string
with two consecutive relocations, and each got a dummy symbol of its own.
Reuse the previous relocation's dummy symbol if it refers to the same
section symbol with the same addend. Since both relocations of a pair now
refer to one symbol, ADRP+ADD relaxation applies to them too. On
chrome-release-arm64, reattach_section_pieces 39 -> 30 ms and peak RSS
5.86 -> 5.56 GB.
Range extension thunk creation and removal lay out input sections in a
serial loop that looked up each member through its file, costing a few
dependent cache misses per section. Read sizes and alignments into an
array in parallel first, and publish the offsets in parallel. On
chrome-release-arm64, compute_section_sizes 34 -> 27 ms and
remove_redundant_thunks 25 -> 15 ms; on chrome-debug-arm64, 87 -> 81 ms
and 72 -> 50 ms.
CIE deduplication, which both ICF and .eh_frame layout run, compared each
CIE with every distinct CIE found so far in a serial loop. Compute their
SipHash digests in parallel with a random key and number the classes
through a hash table instead, assuming that digests never collide as ICF
does. FDE offsets are now assigned in parallel too. uniquify_cies 9 -> 2
ms on chrome-release-arm64; .eh_frame layout 15 -> 8 ms on
chrome-debug-x86_64.
set_visibility() is only called on exclusively owned symbols, but it
updated the bits with a compare-and-swap loop. Take &mut self and write
them directly.
merged_resolve_members visited every file on one thread. Fold the
members of each file in parallel and append the per-thread lists in
file order. About 5 ms on chrome-release-arm64.
gather reserved each shard for every recorded occurrence of a name, about
three times the number of distinct names, so the sparse table touched far
more pages than it used. Estimate the distinct names with HyperLogLog
instead. chrome-debug-arm64 peak RSS 26.76 -> 25.91 GB.
Pattern::matches called memchr::memmem::find for the literal after each
star, which builds a searcher on every call. Build them when compiling
the pattern instead. apply_version_script on tensorflow-debug 260 -> 220
ms.
compute_import_export collected updates in parallel and applied them on
one thread because Symbol's attribute bits needed &mut. Make the bits an
AtomicU16 so that file-parallel passes can set them through shared
references, testing a bit before writing it as mark() does.
parse_symbol_version now finds versioned symbols in parallel; it still
applies them in input order because one may read a version that another
has just set. compute_import_export on tensorflow-debug 167 -> 9 ms.
Assemblers emit a call to a static function in another section as a
branch to a section symbol plus an offset, but a range extension thunk
jumps to the start of a symbol. On RELA targets, give such a branch a
symbol at its destination and a zero addend, as we do for references to
section fragments. This fixes Chromium's ARM64 debug build, which
9b9162146 made an error and 2.42.1 linked into calls to wrong addresses.
mold needs Rust 1.95 for std::hint::cold_path and the atomic update
methods, while its dependencies need only 1.85. Declaring it makes Cargo
reject older toolchains with a clear message before building, and lets
the resolver avoid dependency versions that need a newer rustc.
This reverts 6f8dbb64b. mimalloc uses 64-bit atomics, which GCC
implements as calls into libatomic on hosts such as 32-bit PowerPC.
The need comes from linking mimalloc statically into mold, so the
executable is the right place to add libatomic. Our mimalloc_rust fork
is pinned to the commit before its libatomic change, so the fork
differs from upstream only by the bundled mimalloc version.
This reverts bd97ebc90. mimalloc's page map still doesn't cover the
sign-extended high half of the sparc64 address space, but mimalloc maps
its arenas at hinted addresses in the low half, and since v3.5.0 a page
outside the map fails to register instead of corrupting memory. On an
UltraSPARC T5 running Linux, mold with mimalloc 3.5.3 passes the native
test suite.
mimalloc_rust now links libatomic itself on targets without 64-bit
atomics, so cli/build.rs, which did that for 32-bit PowerPC hosts, is
no longer needed.
readelf before binutils 2.39 can't decode version 9 indices and exits
with an error, so gdb-index-dwarf5-types ignores its exit status as
check_index already does. arch-x86_64-eh-frame-hdr-order links without
the C runtime, whose objects may have FDEs of their own.
mold handles .ARM.exidx and other users of SHF_LINK_ORDER directly
rather than implementing the flag in general. This reverts 49129fbd5,
which kept SHF_LINK_ORDER sections alive with their linked sections, and
restores the ARM-specific GC edge to .ARM.exidx. 9aacb62c0 now sets
sh_link only in relocatable output, which needs it to preserve the flag.
GCC emits no pubnames for type units, so the index listed a type only
for the compilation units referring to it by signature. GDB 18 doesn't
follow such a reference when it looks a type up through the index, so
ptype failed for types defined in type units. Read the name of the type
each type unit defines from its DIEs, resolving their section offsets
with input relocations, and add it to the index for the type unit.
A non-PIC object referring to an absolute or undefined weak symbol with
a PC-relative relocation, such as AArch64's ADRP, can't be linked into
a position-independent output, but recompiling it with -fPIC fixes that
by going through the GOT. The error message lost that hint when it was
reworded, which broke abs-error on aarch64_be.
od -w is a GNU extension, so arch-x86_64-eh-frame-hdr-order failed on
FreeBSD. head and grep -q exit before reading all of their input, so
under pipefail the producer may die of SIGPIPE and fail the pipeline,
which happened to mergeable-entsize-order on FreeBSD.
Clang gives the STT_TLS type to the labels of TLSDESC code sequences,
which are in .text. With --emit-relocs they are written to .symtab,
where their values were computed as offsets from the start of the TLS
segment, which underflowed. Write the address of an STT_TLS symbol
outside a TLS section as it is.
Assemblers emit a call to a static function as a relocation against a
section symbol plus an offset. A range extension thunk jumps to the
start of a symbol, so taking such a branch through a thunk landed at
the wrong address. Report it as an error instead.
Relocations in a relocatable output may refer to local symbols such as
RISC-V's .L labels, so removing them with -x, -X or -s left relocations
pointing to symbol 0. -r now ignores -x and -X, and -s strips only debug
info as -S does.
In the extreme code model, the pcalau12i with R_LARCH_TLS_GD_PC_HI20 or
R_LARCH_TLS_LD_PC_HI20 begins a 64-bit sequence whose lu32i.d and lu52i.d
supply the bits beyond ±2 GiB, so the GOT may be farther away than that.
The other *_PC_HI20 relocations were already applied without a range
check, as in GNU ld and lld, but these two rejected such a GOT with an
out-of-range error.
In the extreme code model, lu32i.d and lu52i.d follow pcalau12i by 8 and
12 bytes, and the upper 32 bits they set are relative to the pcalau12i's
page. We used page(P - 8) for both, which is the next page for lu52i.d if
the pcalau12i is the last word of a page. The result was wrong whenever
the borrow reached bit 52; e.g. a pcalau12i at 0x100000ffc materialized
0xfff0000080000000 instead of 0x80000000.
mold crashed at startup on sparc64 Linux, which maps user memory at
sign-extended high addresses. mimalloc v3 registers pages in a map that
covers only 48-bit addresses, so the first registration wrote out of
bounds. Until the bundled mimalloc handles that layout, sparc64 hosts use
the system allocator (https://github.com/rui314/mold/issues/1639).
mold -run looked for mold-wrapper.so in /usr/local/lib/mold unless it was
next to the executable or in ../lib/mold, and install-mold.sh always put
it in $PREFIX/lib/mold, so a package using lib64 or a multiarch libdir
could not make -run find it. The C++ version took the directory from
CMake's libdir. MOLD_LIBDIR now sets it at build time and install time.
util::prefetch assumed SSE on all x86 targets, but i586-unknown-linux-gnu
has none, so mold died with SIGILL on CPUs such as the Pentium, K6,
Vortex86 and Quark. The x86 prefetch is now compiled only when the target
has SSE. Such a target simply ignores the hint, as other targets without
a prefetch instruction do.
Versions of a symbol have the same name, so they tied in the unstable sort
of exported dynamic symbols. Rayon breaks ties with a pseudo-random
generator whose width follows usize, so .dynsym and its hash and version
tables differed between 32-bit and 64-bit hosts. The input order now breaks
the tie. .eh_frame_hdr had the same problem with FDEs for one function,
which are now ordered by their addresses.
On a 32-bit host, linking an output of 2 GiB or more panicked, because
the doubled mapping length wrapped around and sizes of 4 GiB or more were
truncated to usize. Each panic also left a multi-GiB temporary file. We
now check the size before creating the file and report a fatal error.
The limit is 2 GiB, as no Rust object can be larger than isize::MAX bytes.
A range extension thunk jumps to its destination with `br x16`, which
faults under BTI if the destination doesn't begin with a landing pad, as
is the case for a static function that is only called directly. For such
a function, we now put `bti c; b <function>` at the beginning of the first
thunk after the function's section, which is always within direct branch
range, and other thunks jump there instead, as lld's landing pad thunks do.
-z force-bti and -z pac-plt force the BTI and PAC bits of the output's
GNU_PROPERTY_AARCH64_FEATURE_1_AND, and -z pac-plt also makes PLT entries
authenticate .got.plt entries with autia1716 and sets DT_AARCH64_PAC_PLT.
-z gcs=implicit|never|always controls the GCS bit. -z bti-report and
-z gcs-report diagnose input files lacking the features, and as in lld,
-z force-bti and -z gcs=always imply the warning level.
We used to merge only x86 properties in .note.gnu.property, so an ARM64
output lost GNU_PROPERTY_AARCH64_FEATURE_1_AND even if all its inputs were
compiled with -mbranch-protection. We now AND-merge it as GNU ld and lld do.
If the output is BTI-enabled, the PLT header and the PLT entries that may be
reached by an indirect branch start with `bti c` as lld's do, which makes PLT
entries 24 bytes long, and DT_AARCH64_BTI_PLT is set.
R_RISCV_ALIGN has no symbol, and its addend is the size of the NOP
padding that a relaxing link shrinks to realign the code after it.
symidx_addend returned (0, 0) for any symbol not written to the output
symbol table, including the null symbol, so `mold -r` wrote R_RISCV_ALIGN
with addend 0 and a later link could no longer realign the code. We now
keep the addend of a relocation without a symbol, as GNU ld and lld do.
LoongArch's R_LARCH_ALIGN refers to a local symbol and was not affected.
GCC 14 uses R_LARCH_TLS_DESC64_PC_LO20 and R_LARCH_TLS_DESC64_PC_HI12
with -mcmodel=extreme -mtls-dialect=desc to extend the TLSDESC sequence's
GOT access to 64 bits, and we rejected them as unknown. We now apply them
like the other 64-bit PC-relative pairs and don't rewrite the sequence's
LO12 instruction with pcaddi. As GNU ld and lld do, we don't relax the
sequence to IE or LE, except in static executables, which can't use
TLSDESC and therefore get LE.
is_got_load_pair checks that a GOT load's PCREL_LO12_I refers to the
label at its AUIPC by comparing the label's value with the AUIPC's
r_offset. Symbol values are adjusted for removed bytes before relocations
are applied, but r_offsets are not, so a GOT load after a shrunk
instruction was no longer rewritten to AUIPC+ADDI. We now compare with
the label's st_value, as find_paired_reloc does.
R_RISCV_32_PCREL, R_RISCV_PLT32 and R_RISCV_GOT32_PCREL in allocated
sections were truncated to 32 bits without a range check, so a reference
to a symbol or a GOT slot more than 2 GiB away silently got a wrong
value. We now report an out-of-range error for them, as we already do
for R_RISCV_32_PCREL in .eh_frame and as lld does.
The .riscv.attributes parser knew the value types of only the three tags
mold uses, so the value of any other tag was read as a tag. For example,
an unknown tag 40 with value 5 was mistaken for Tag_RISCV_arch. As per
the psABI, an even tag has a ULEB128 value and an odd tag has a
NUL-terminated string, so we now skip other attributes by that rule.
They are still dropped from the output, as lld does for unknown tags.
The output's e_flags were the first input's plus EF_RISCV_RVC, so an
object compiled for the TSO memory model (-march=rv64gc_ztso) lost its
EF_RISCV_TSO bit if it came after an ordinary RVWMO object. RVWMO code
runs correctly on TSO hardware but not vice versa, so the output now has
the bit if any input has it. GNU ld does the same; lld still takes the
bit from the first file.
-Map wrote address zero as "0x0" where C++ mold wrote "0", and
--print-dependencies omitted the blank line that separated the comment
header from the dependency lines. Both outputs are now byte-identical
to those of C++ mold.
With MOLD_JOBS=1 and an empty XDG_RUNTIME_DIR, mold joined "" with
"mold-lock" and created the lock file in the current directory, so
processes in different directories did not serialize. Like other XDG
variables, an empty value now means unset, and mold falls back to
/tmp/mold-lock-$USER.
-rpath-link, -fuse-ld, -O, --sort-section and --max-cache-size are
accepted only for compatibility, and their values are never used. mold
nevertheless required the values to be UTF-8, so a non-UTF-8 directory
given to -rpath-link was a fatal error. Read the values as raw bytes.
The manual said -z separate-code was the default, but mold defaults to
-z noseparate-code and adds no padding between segments unless
--section-order is given, which implies -z separate-loadable-segments.
The manual documents --no-color-diagnostics, but mold rejected it as an
unknown option; accept it as lld does. Conversely, the manual and --help
listed -z nodefs and -z start_stop_visibility, which neither mold nor
GNU ld accepts; the options are -z undefs and -z start-stop-visibility.
--help also said --color-diagnostics means always, while it means auto.
--dynamic-list-data, --lto-pseudo-probe-for-profiling and a bare
--thinlto-index-only take no argument in GNU ld and lld, but mold read
the next command line argument as their value, silently dropping an
input file placed there. GNU ld's --package-metadata takes an optional
argument, so mold now accepts its value only after an equal sign.
Rust's io::Error appends " (os error N)" to the strerror text when it is
displayed, so mold printed "cannot open foo.o: No such file or directory
(os error 2)" where the C++ version printed just the text. Diagnostics
now format I/O errors with a strerror helper that returns the text alone
and falls back to the usual formatting for errors not from the OS.
scan_pcrel, which also handles GOT-relative relocations, told the user
to recompile with -fPIC whenever it couldn't resolve a relocation, even
though code using GOTOFF is PIC already. It now reports the reason: a
symbol at a fixed address, such as an absolute or undefined weak one, in
a position-independent output, or a preemptible symbol defined in the
shared object being created, which -Bsymbolic or hidden visibility fixes.
`-e 0x401000` was looked up only as a symbol name, so the link warned
that the entry symbol was undefined and set e_entry to 0. GNU ld and lld
fall back to parsing the argument as a number (decimal, 0x hex or octal)
when no symbol has that name, and so does mold now.
When a version script named the same symbol in two version nodes, each
exact-name match overwrote the previous one, so the last node won.
GNU ld and lld give the symbol the first node's version (lld warns about
the reassignment, and GNU ld rejects a name that is both local and
global). Apply only the first exact match. Wildcards still let the last
matching node win, as in both linkers.
A version script or --default-symver on a link that exports no dynamic
symbols, such as a static executable, made verdef::construct index the
empty .dynsym and panic. Symbol versions exist only for dynamic symbols,
so write no version sections in that case, as GNU ld does.
Undefined symbols are allowed by default with -shared or with
--unresolved-symbols=ignore-all, and a non-weak hidden one was then
silently resolved to 0. A hidden symbol can't be resolved at runtime,
so report it as an error regardless of those options, as GNU ld and lld
do. Weak undefined hidden symbols still resolve to 0.
--defsym=foo=bar with bar defined by a shared library copied bar's
library-relative value into an absolute foo, so calling foo jumped to a
bogus address. Such a symbol has no link-time address, so report it the
same way as an undefined one. GNU ld and lld reject it as well.
The first file on the command line owned a common symbol, so `int buf[4]`
in one file and `int buf[64]` in another gave a 16-byte buf, and writes
to it clobbered its neighbours. As GNU ld and lld do, let the largest
common own the symbol and align it to the largest alignment among them,
for .common, .tls_common and -r output alike.
--export-dynamic and --dynamic-list export symbols even from an
executable that links against no DSO. Such an executable has no
.dynamic, but the symbols were still added to .dynsym, and .dynstr
lacked its leading empty string, which only the creation of .dynamic
adds, so entry 0 of .dynsym got a symbol's name. Like GNU ld and lld,
emit no dynamic symbol table without .dynamic.
Mergeable sections that share a name, type and flags but differ in
entry size are merged into separate output sections. Their sort keys
tied, so their order depended on which one the parallel conversion of
input files happened to create first. Break the tie with the first
member's file priority and section index, as for output sections.
add_synthetic_symbols defined __start_ and __stop_ symbols for every
allocated chunk with a C identifier name, including the ELF and program
headers named EHDR and PHDR. fix_synthetic_symbols skips headers, so
the symbols were left in .symtab with value 0, and -z
start-stop-visibility=protected exported them. Exclude headers in
start_stop_name, which both functions use.
compute_address_significance marked exported symbols' sections as
address-taken by walking .dynsym, but .dynsym is not populated until
after ICF, so the loop did nothing and --icf=safe gave identical
exported functions of a DSO the same address. Walk each object file's
defined global symbols instead, whose is_exported bits are already set.
DT_INIT and DT_FINI refer to the --init and --fini functions, but only
the entry point was a GC root. If such a function was not exported
because it was hidden or made local by a version script, --gc-sections
discarded its section, DT_INIT and DT_FINI were emitted with value 0,
and the DSO crashed on load. Make them GC roots as GNU ld and lld do.
The $a, $t and $d symbols that mold creates for PLT entries and thunks
were STT_FUNC, copied from the symbols naming the entries, while AAELF
requires mapping symbols to be STT_NOTYPE. push_mapping_symbol now
builds the symbol from a section index and an address itself.
Only a final link converts big-endian ARM code to BE8, so relocatable
output keeps the input's BE32 code, but it was marked EF_ARM_BE8 anyway.
GNU ld rejected such an object as "already in final BE8 format", and
disassemblers decoded its code in the wrong byte order. -r also failed
with "--be32 is not supported" unless --be8 was given, which GCC and
Clang never pass with -r.
A 32-bit Thumb instruction is a pair of halfwords, each in the byte
order of the input's code, but NOP.W was written as a single 32-bit
word. On big-endian targets, that swapped its halfwords, so a Thumb call
to an undefined weak symbol and a TLSDESC call relaxed to local-exec
became garbage in BE8 output. NOP.W is now written a halfword at a time.
Clang assembles ADR and LDR (literal) referring to a symbol in another
section to R_ARM_ALU_PC_G0 and R_ARM_LDR_PC_G0 in ARM code and to
R_ARM_THM_ALU_PREL_11_0 and R_ARM_THM_PC12 in Thumb code, which failed
with "unknown relocation", as did 8- and 16-bit data. The offset's sign
picks ADD or SUB, or the U bit of LDR, and the Thumb forms are relative
to the PC aligned down to 4, producing the same bytes as GNU ld.
TBZ/TBNZ to a label in another section, .2byte/.4byte data referring to
a symbol, and GOT loads compiled with -mcmodel=tiny -fPIC failed with
"unknown relocation". TSTBR14 reaches ±32 KiB and is range-checked like
GNU ld and lld do, without thunks. ABS16 and ABS32 are too narrow for
dynamic relocations, so they must be link-time constants in PIC output.
GOT_LD_PREL19 is an LDR literal that loads the symbol's GOT slot.
Relocations copied by --emit-relocs refer to symbols in .symtab, which
--strip-all omits, so the combination crashed or produced relocation
sections pointing to a nonexistent symbol table. Report an error as
lld does. (GNU ld instead keeps the symbols the relocations need.) -r
sets --emit-relocs internally and keeps accepting --strip-all.
An SHF_LINK_ORDER section's sh_link refers to the section it describes,
but mold always wrote zero. Once passed through -r, such sections lost
their links, so relinking with --gc-sections dropped .ARM.exidx and
__patchable_function_entries, and C++ programs on ARM aborted when
throwing. Set sh_link from the members' linked-to sections. Since it
can name only one section, -r also separates SHF_LINK_ORDER sections
whose linked-to sections are in different output sections, as GNU ld
and lld do.
An SHF_LINK_ORDER section such as __patchable_function_entries holds
metadata for the section that its sh_link refers to, and usually
nothing else refers to it. --gc-sections therefore discarded all of
them except .ARM.exidx, for which there was an ARM-specific edge from
a section to its unwind index. Replace that edge with a generic one
from a section to the SHF_LINK_ORDER sections linked to it, as GNU ld
and lld do.
Allocated fragments of mergeable sections start out dead under
--gc-sections and are revived by the garbage collector, but mold -r
returns before running GC. As a result, `-r --gc-sections` discarded
every .rodata.str* and .rodata.cst* piece and left the relocations
referring to them with symbol index 0. Since mold doesn't collect
garbage for -r, ignore the option there. GNU ld and lld do collect
garbage with -r, taking roots only from -e and -u.
R_SPARC_WDISP16 is used by branch-on-register instructions such as brz,
whose 16-bit word displacement is split into d16hi at bits 21:20 and
d16lo at bits 13:0. We ORed the fields into the upper halfword of the
instruction instead, corrupting its opcode, and limited the displacement
to 64 KiB instead of 128 KiB. Place the fields as GNU ld does.
R_SPARC_OLO10 carries a second addend in the upper 24 bits of r_info,
which is added after the low 10 bits of S + A are taken. RelocSection
built its output records with ElfRel::new(), which zeroes that field, so
`ld [%g1 + %lo(var) + 8]` lost its +8 in -r and --emit-relocs output.
Build each record from a copy of the input one instead.
The 8- and 16-bit GOT and TLS relocations fill the displacement of a
(d8,An,Xn) or (d16,An) operand, which the CPU sign-extends, but we checked
them against unsigned ranges. A GOT offset of 32 KiB or more was thus
silently encoded as a negative displacement, and a local-exec TLS offset,
which is usually negative because the thread pointer is biased by 0x7000,
was rejected. Check them as signed as GNU ld does.
SH4 object files keep relocation addends in the relocated places even
though the relocations are RELA, and RelocSection already follows that
convention. EhFrameRelocSection, however, put adjusted addends only into
r_addend, which SH4 readers ignore, so after `ld -r` every FDE's pc_begin
pointed at the start of its output .text and C++ exceptions terminated.
Write the addends into .eh_frame as we do for REL targets.
This enables the three -r exception tests that were skipped on SH4.
On SH4, r2 carries the address of the buffer for a struct return value.
Our PLT header loaded the link map into r2 and set r0 to 0, which glibc's
resolver accepts, but the resolver then restores r2 to its entry value, so
a lazily bound function returning a struct overwrote the link map. The
header now passes the link map in r0 as GNU ld does, borrowing r2 only
until the jump's delay slot restores it from the stack.
A section folded by ICF has no output section, so with --emit-relocs a
relocation against its section symbol was emitted with symbol 0, and one
in .rela.eh_frame, such as an LSDA pointer, hit an expect(). Symbols
defined in a folded section are not in .symtab either, so references to
them lost their symbol or, in .rela.eh_frame, got a wrong one. Emit both
relative to the leader's output section, as the relocated contents and lld
do.
When relaxing TLSDESC to initial-exec, we write an AUIPC to the
TLSDESC_ADD_LO12's slot and a load to the TLSDESC_CALL's slot, but we
computed their GOT-relative offsets from the TLSDESC_HI20's address. The
two coincide only if the instructions in between are deleted, which is not
the case for objects lacking R_RISCV_RELAX on TLSDESC relocations, such as
clang 18's, so the load read a wrong GOT entry. Use the AUIPC's address,
as lld does.
With --emit-relocs, a TLSDESC_HI20 relaxed to IE or LE was rewritten to
R_NONE as soon as it was applied. The TLSDESC_LOAD_LO12, TLSDESC_ADD_LO12
and TLSDESC_CALL that follow find it with find_paired_reloc(), which
matches by type, so such links failed with "paired relocation is missing".
Rewrite TLSDESC_HI20s after all relocations have been applied. LoongArch is
unaffected because its TLSDESC relocations refer to the symbol directly.
shrink_sections adjusted each symbol by the r_deltas of its own input
section, but a section folded by ICF is never shrunk, so its symbols kept
their pre-relaxation offsets while being resolved against the shrunk
leader. With LoongArch GCC's leading R_LARCH_ALIGN nop, calls to a folded
function landed 4 bytes past the leader's entry. A folded section was
identical to its leader before relaxation, so use the leader's deltas for
both symbol values and st_size.
GCC calls _savegpr0_N, _restgpr0_N, _savefpr_N and similar functions
from -Os prologues and epilogues and expects the linker to provide them.
We synthesized them only for ELFv2, so ELFv1 programs built with -Os
failed to link with undefined symbols. ELFv1 calls them with plain
R_PPC64_REL24 relocations, not through function descriptors, and
expects the same code, so emit the same section in target byte order
and move its instruction table from ppc64v2.rs to the section's module.
R_PPC64_GOT16, _LO, _HI and _HA take the offset of a GOT slot from the
TOC base, but we subtracted the TOC base from the slot's offset within
.got rather than from its address, so the result was off by the address
of .got. GCC addresses data through .toc instead, so only hand-written
assembly using @got was affected.
With -fpic, PPC32 code loads a GOT slot with one instruction whose
R_PPC_GOT16 relocation is a signed 16-bit offset from the GOT, and
R_PPC_GOT_TLSGD16, R_PPC_GOT_TLSLD16 and R_PPC_GOT_TPREL16 are the
same. We truncated these offsets silently, so a program whose GOT grew
past 32 KiB linked but loaded the wrong slots. Report an error instead,
as the psABI requires overflow checks for these fields.
A static executable's startup code applies the R_IRELATIVE relocations
between __rela_iplt_start and __rela_iplt_end, and we placed the start
by counting only the GOT slots of IFUNC symbols. On PPC64, a data
reference to an IFUNC symbol gets its own R_IRELATIVE instead of the
canonical PLT address, so the range began too late and skipped a GOT
slot, and a call through that slot ran the resolver. Count them too.
A global symbol defined only in a member of a discarded COMDAT group has
no owning file, so it has nothing to resolve to. The relocation scan
reports a reference to it and skips the relocation, but --noinhibit-exec
downgraded the error to a warning and the relocation was applied anyway,
which panicked on a missing GOT or TLS entry on most targets. Report the
reference as a fatal error in that case.
The relocation scan skips a relocation to an undefined symbol when that
is an error, because the link fails anyway. --noinhibit-exec reports the
error as a warning and writes the output, so relocations that needed a
GOT, PLT or TLS entry were applied without one, and mold panicked on the
missing index. Such relocations are now scanned as they are with
--warn-unresolved-symbols, so the symbol gets the entries it needs and
resolves to 0, as it does in GNU ld and lld.
Every target crashed on a -fPIC object reading an undefined variable,
an undefined TLS variable, or both.
GNU as emits R_RISCV_ALIGN and R_LARCH_ALIGN for an alignment directive
in an executable NOBITS section. With -r, RelocSection sliced the output
section's bytes at each relocation before calling write_addend, but a
NOBITS section has no bytes in the file, so the slice was out of range.
Since only REL targets and SH4 keep addends in section contents, take
the slice only on those targets.
GAS and Clang emit R_RISCV_64 and R_LARCH_64 for `.quad sym` even in
ELF32 objects. On RV32 and LA32, they are not the word-size absolute
relocation that the output section applies, so they reached
apply_reloc_alloc, which had no case for them and panicked. Handle them
like other non-word-size absolute relocations, as lld does: scan them
with scan_absrel and write S + A as a 64-bit value.
With a plugin that lacks get_symbols_v3, mold re-executes itself with
the same command line. The new process applies -C again, but it was
started in the directory the first one had already changed to, so a
relative -C failed with "chdir failed". Remember the directory before
-C and return to it before re-executing.
Input file names already have --chroot applied. Relative thin archive
members and relative names in linker scripts are resolved against such
a name, but the result was opened with --chroot applied again, so the
link failed. Open these paths as they are. Also write the --repro
archive next to the output in the chroot rather than outside of it.
--repro reopened each input by its recorded name. Response files and
--retain-symbols-file are opened before -C changes the directory, so a
relative name no longer found them and the link failed, and a name with
--chroot applied got the prefix twice. Archive the contents already in
memory instead: no input has been rewritten in place at this point.
An existing output file is reused by writing it in place. When that file
is also a memory-mapped input, as in `mold a.o -o a.o` with an a.o larger
than 32 KiB, its private mapping sees the truncation and the writes while
the input is still being read, so mold crashed with SIGBUS or produced a
broken output. Such a file is now replaced with a fresh one. Inputs are
matched by device and inode, which covers archives and thin members too.
An absolute path in a linker script is prefixed with the sysroot if
the script is in the sysroot. is_in_sysroot returned false for an
empty sysroot, so such a path fell through to the -L search, and an
incompatible file was skipped with a warning. C++ mold treats an empty
sysroot as containing every file and opens the path as is, reporting
an error if it cannot be used. GNU ld and lld don't search -L either.
--dynamic-list and --export-dynamic-symbol-list files were opened by
the driver after parse_args had already changed the directory for -C,
so a relative path was looked up in the new directory. C++ mold reads
them in the option loop, as we already do for --retain-symbols-file.
Open them there too and keep the mapped files in DynamicListSource;
the driver still parses them, since the parser needs a Context.
A panic aborts the process in release builds, skipping the cleanup that
a fatal error or a SIGSEGV/SIGBUS does, so a failed link left its
.<output>.<pid> file behind. Install a panic hook that removes the file
and then calls the default hook, so the panic message, backtrace and
abort stay as they were. The hook runs on the panicking thread, so it
also covers rayon workers and the extern "C" LTO plugin callbacks.
Arguments are passed on as C strings, so a NUL byte in the -plugin path,
a -plugin-opt or the -o name made loading the LTO plugin panic. A
response file is the only way to get a NUL byte into an argument, as
the OS passes arguments as C strings too, so reject one when reading a
response file instead of at each C string conversion.
mimalloc uses 64-bit atomics, which GCC implements as calls into
libatomic on hosts such as 32-bit PowerPC. The mimalloc crate links
libatomic only on ARM, so mold failed to link there, which CI's cargo
check did not catch. Link libatomic when the target lacks 64-bit
atomics, as the C++ version did. It is a link argument rather than a
library because rustc would place the latter before mimalloc, where
--as-needed drops it.
libz-sys declares the length parameter of adler32_combine() as
libc::off_t, which is 32 bits on 32-bit glibc hosts, but Debian and
Ubuntu now compile C code for armhf and powerpc with 64-bit off_t, so
the bundled zlib reads a 64-bit length there. Every zlib-compressed
debug section larger than one 1 MiB shard got a wrong checksum, and
readelf, objcopy and gdb rejected it. Compute the combination in Rust
instead; it is a few lines of modular arithmetic.
apply_reloc_alloc read the 4-byte instruction at every relocation to
extract its register fields, so a 1- or 2-byte data relocation such as
R_SPARC_8 or R_SPARC_DISP16 in the last bytes of a section panicked.
Read the fields only for the relocations that rewrite an instruction.
i386 aligns 64-bit integers to 4 bytes, so Symbol and NameRecord are
smaller there than on other hosts, and the assertions that they are
exactly 48 and 16 bytes failed to compile. Assert upper bounds instead,
which still catch growth on 64-bit hosts, and check the i686 build in
CI.
std has no AtomicU64 or AtomicI64 on hosts such as 32-bit PowerPC,
32-bit MIPS and pre-v6 ARM, so mold did not compile there, whereas the
C++ version fell back to libatomic. Take those two types from
portable-atomic, which uses the std types where they exist and falls
back to locks elsewhere. Also check the 32-bit PowerPC build in CI; the
existing armv7 check did not catch this because armv7 has 64-bit
atomics.
See https://github.com/rui314/mold/discussions/1643#discussioncomment-18603572.
The .tbss layout loop gained a second end-of-list check next to the
existing one, for a case that cannot happen: .tbss is never the last
chunk. DigestMap::new was rewritten with repeat_with, which took five
lines to say what one line did. Restore both.
Follow-up to https://github.com/rui314/mold/pull/1685.
FileRecord::size was size_of::<Self>(), and record sizes were written
both ways. Use size_of everywhere. Unlike the trait method, it is a
const fn, so it also works in the compile-time assertions.
https://github.com/rui314/mold/pull/1683 unified the spelling in the
other direction.
An ignored SIGCHLD is inherited across exec, e.g. from `trap '' CHLD`
in a shell. The child that fork_child creates is then reaped
automatically, so the parent's waitpid fails with ECHILD. The parent
ignored that and read the untouched zero status as a successful exit,
so a link that failed exited with status 0. Report a failure instead.
Also read and write the completion byte through File, which removes
two raw libc calls.
Based on https://github.com/rui314/mold/pull/1682.
Both callers of from_hex already check that the byte is a hex digit,
so the new assertion was redundant. The utsname rewrite traded
MaybeUninit::assume_init for mem::zeroed without removing an unsafe
block, so restore the original code.
The fix it tests is target-independent, and only its last check was
x86-64-specific. Count the dynamic relocations against foo and bar
instead of looking for R_X86_64_64. PowerPC is skipped because GCC puts
the read-only pointers in writable sections there, so nothing is
promoted.
LLVMgold calls get_symbols with a null pointer and a zero count for an
IR file that has no symbols, e.g. one compiled from an empty source
file. slice::from_raw_parts does not accept a null pointer even for a
zero length, so the debug build aborted on its precondition check.
Use an empty slice for a zero count, in add_symbols as well.
Based on https://github.com/rui314/mold/pull/1680.
Building the thread pool fails when threads cannot be created, e.g.
under a low `ulimit -u`. That hit an expect, so mold aborted with a
panic message rather than reporting the error like other system
errors.
Taken from https://github.com/rui314/mold/pull/1678, which also
added checks for malformed inputs that are not included here.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Input files are opened in parallel, and each file is appended to the
file pool when it is opened, so the pool's order changed from run to
run. --dependency-file and --repro list files in that order. Return
the pool sorted by name instead.
The dependency-file tests expected a space after a.o, i.e. that a.o
was not the last dependency. That failed intermittently on Alpine,
where libc.so is not a linker script and no dependency reliably comes
after a.o.
report_out_of_range looked the record up again through the file's
relocation table while apply_reloc_alloc held that table mutably, which
created a shared view of memory behind a live unique reference, and it
walked the CREL iterator from the start to do so. Every caller has the
record at hand, so pass it in. The loop indices that only served the
lookup are gone with it.
Cherry-picked from https://github.com/rui314/mold/pull/1674.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
CieRecord stored a shared slice over its .eh_frame relocation records,
but reattach_fragment_relocations later rewrites those records in place
through the private input mapping, and the FDE and CIE accessors read
through the stale slice afterwards. The record now keeps only the
relocation section index and reads the current table through
ObjectFile::relocations, as decoded CREL tables already did. The raw
relocation view no longer claims a 'static lifetime.
Cherry-picked from https://github.com/rui314/mold/pull/1674.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
mold accepts a multi-letter option with a single dash, but the option
chain is first-match, and some single-letter options were tried before
long options starting with the same letter. "-entry=main" was thus read
as "-e ntry=main", "-filter libf.so" as "-f ilter" followed by an input
file, and "-fix-cortex-a53-843419" left a bogus DT_AUXILIARY entry. GNU
ld and lld read all of these as the long option.
Move each single-letter option past the last long option it shadows.
Conversely, GNU ld and lld read "-execute-only" as "-e xecute-only" and
"-omagic" as "-o magic", because those names take double dashes only.
Spell such names with double dashes in the option table, which
match_option now honors, replacing the rule that all names starting
with "o" need two dashes. Where GNU ld and lld disagree, follow GNU ld.
Thanks to @lu-zero for investigating this issue and proposing a fix in
https://github.com/rui314/mold/pull/1670.
Fixes https://github.com/rui314/mold/issues/1671
Zip the glob NFA's three parallel arrays and fill a bracket range with
one slice fill, merge the two glob module headers left over from two C++
files, check SipHash's output width at compile time and split its input
into words with as_chunks, mark Timer must_use so a dropped one is a
compile-time warning, remove a Windows TODO from the C++ demangler that
never applied to this implementation, and fix a typo in a comment.
The merged glob header no longer lists qt_private_api_tag* as an
example of an Aho-Corasick pattern, since prefix patterns are matched
directly rather than by the Aho-Corasick matcher.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Each output section promoted the imported symbols of its read-only
absolute relocations and classified its relocations in one pass, while
other sections ran in parallel. Whether a writable section's relocation
against the same symbol became a dynamic relocation therefore depended
on which section ran first, so the output changed from run to run. In a
non-PIE that references an imported variable and function from both
.rodata and .data, 37 of 100 links in one measurement emitted two extra
R_X86_64_64 dynamic relocations.
Collect all sections' relocations, promote, and only then classify, so
that the result is the same in every run: the word is resolved at link
time whenever a copy relocation or canonical PLT entry makes its address
known.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The --undefined names were cloned only because get_symbol takes the
whole context mutably, so intern them through the symbol table instead.
The section-word reversal swaps whole words, the .tbss layout loop no
longer indexes past the last chunk, the has_lto_obj predicate replaces
an inline copy of itself, and the --exclude-libs ALL test is hoisted out
of the file loop. The dynamic symbol groups are extended in place, the
IRELATIVE count loses a temporary, and thunk addresses zip symbols with
their offsets.
In ICF, the digest no longer converts to little-endian on the way into
a hash whose key is random per run, the digest map is filled with
repeat_with, and HashMap is imported once.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
apply_reloc_nonalloc on ARM32, PPC32, m68k and SH-4 read the relocation
section of a non-allocated section as an array of ordinary records. A
CREL table of a non-allocated section is not decoded ahead of time, so
for an object with SHT_CREL relocations those bytes are compressed
records: linking a PPC32 or m68k object that Clang 19 built with -g and
-Wa,--crel panicked with a failed length assertion in
records_from_bytes. Use the relocation iterator, which decodes CREL
tables, as the other targets already do.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
A relocatable link keeps every COMDAT group's members in output
sections of their own, so an object with many groups of one name, such
as the .debug_macro groups GCC emits with -g3, produces output sections
whose sort keys tie. The tie fell back to the order in which the
parallel creation phase happened to make them, which changed from run
to run: 30 runs of the link in relocatable-compressed-debug-info.sh
produced 13 different outputs. Break the tie with the first member's
file priority and section index, which are in input order.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
C++ mold adds the _savegpr0_* family after the __start_ and __stop_
symbols, and the internal object's symbol order is the order in the
output .symtab. Match it so that ppc64le outputs get the same .symtab,
and hence the same build ID, as C++ mold's.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
C++ mold defines these target-specific segments after PT_GNU_RELRO, but
the port defined them before PT_GNU_STACK. ARM and RISC-V outputs
therefore had their program headers in a different order than C++
mold's for the same input, and a different build ID. Define them where
C++ mold does.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The four places that write a target word through an IS_64 branch
(.got, .got.plt, .relr.dyn and an output section's absolute
relocations) now use the existing Word::<E>::new(value).write(slot).
The padding filler walks its buffer with chunks_exact_mut; thunk
symbols zip their two parallel vectors; relr_offsets concatenates its
shards once; the .ARM.exidx lookup uses find_map instead of position
plus a second match, and text_end borrows the section headers instead
of copying them; .gnu.hash looks each symbol's hash up once;
merged-section resolution takes the timers by destructuring instead of
cloning them; OutputSectionId::new folds its two checks into one; the
.sframe writer slices each FRE block by start and length; WORD_SIZE is
derived from the word type like IS_64; and ElfDyn derives what its
siblings derive.
The doc comment on ChunkHeader's local-symbol fields no longer starts
with a stray "For --gdb-index", and a few comment typos are fixed.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
shndx_at_in was identical to shndx_at, is_discarded_comdat re-implemented
is_discarded_comdat_sym, ObjectFile's Display duplicated display_file,
rels_mut looked its table up twice, find_null re-implemented a chunked
position search, the archive member name reader re-implemented cstr_at
and memchr, the file type probe indexed the header table it was
iterating, and the linker script reader spelled slice patterns as length
checks plus indexing.
The loop in is_dwarf32 now compares the end of the next unit header with
the section size instead of subtracting its start from the size, which
wrapped around when a unit's size field pointed past the end of a
malformed section.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The command-line and result types of the per-target link functions were
spelled out in the twenty target crates, the executable and the driver.
Name them Cmdline and LinkResult in the driver and use the names
everywhere else.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The record views in elf.rs relied on a debug assertion, and therefore on
the FileRecord contract alone in release builds, for the alignment that
their pointer casts require. An inline const assertion is evaluated per
instantiation, so a record type that violates the contract no longer
compiles.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Several comments compared the code to C++ mold's arena storage and
chunks, or named functions that do not exist in this implementation
(SharedFile<E>::mark_live_objects, Symbol<E>::get_addr and
open_file_impl); point them at the Rust functions or drop the
comparison. The ICF and thunks modules repeated their description in a
second module-level summary, and the TLS module docs had a stray title
line. drop_mappings did not say that it discards the relocation records
rewritten in place, and the madvise in write_build_id did not say why
dropping written pages of the output loses nothing. Also fix three
typos.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The page-aligned start of the interior range was turned back into a
pointer with an integer cast, which leaves it without the allocation's
provenance. Derive it from the caller's pointer with with_addr instead,
as the rest of the function already reads addresses with addr().
Co-Authored-By: Claude Opus 5.5 <[email protected]>
record_comdat_owner and set_visibility_bits hand-rolled fetch_min and
update. record_comdat_owner keeps its initial load so that a file that
does not lower the priority still leaves the cache line shared. The
packed visibility and flags bytes are no longer pub, since nothing
outside symbol.rs touches them and a direct store would corrupt the
state bits they share.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
x86-64 and i386 defined their own little-endian write helpers with the
same bodies as write_ul16, write_ul32 and write_ul64 in util::endian,
and i386 read REL addends with inline from_le_bytes calls. Call the
shared functions directly, as the other fixed-endian targets do. The
ARM64 and RISC-V instruction accessors now delegate to them, like
LoongArch's already do.
Also drop an unused parameter of relax_tlsdesc_to_ie, a redundant mask
on SPARC relocation types (the record keeps the type in a byte of its
own), two TRAP comments that described C++ target structs instead of
the instruction, and the pub on two PPC64 ELFv1 functions that only
their module uses. SH-4 reports an unknown relocation with error! like
every other target, so that all of them are listed in one run.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The key length was an UnsafeCell whose reads and write needed unsafe
blocks justified by the release/acquire pair of the key pointer. A
relaxed atomic expresses the same protocol, compiles to the same loads
and stores, and takes the probe comparisons out of unsafe code. Also
check the probe invariant at compile time and reject keys longer than
32 bits instead of truncating their lengths.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
driver is a public module, so `pub` made copy_chunks part of the
library's API although only the crate itself calls it; make it
pub(crate). from_hex now asserts its precondition in debug builds
instead of silently producing garbage for a non-hex byte.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Six flags in the ignored-for-compatibility chain were unreachable
because earlier arms already consume them: color-diagnostics,
eh-frame-hdr, fatal-warnings, enable-new-dtags, disable-new-dtags and
warn-once.
returns_etxtbsy zeroed a MaybeUninit<utsname> and then asserted it
initialized after calling uname. utsname consists of byte arrays, so a
zeroed one is already a valid value; start from that and drop the
assume_init step.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The runner starts each script in its own process group, so a script
that read the terminal would stop there until the timeout. Give it
end-of-file instead.
run already returns ExitCode, so report setup errors through it
instead of exiting the process. Also drop the second sort of the
discovered scripts (make_jobs sorts the combined list), the unused
Clone on TestJob and the intermediate vector in the PATH search, and
give the unsafe kill a SAFETY comment like the other unsafe blocks.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
R_X86_64_GOTPC32, R_386_GOTPC, R_390_GOTPC and their variants compute
GOT + A - P, the distance to the start of the GOT section rather than
to an entry in it. They were grouped with the GOT-indirect relocations,
so every link that referenced the GOT base got an entry for
_GLOBAL_OFFSET_TABLE_ that nothing ever read, plus a RELATIVE
relocation to fill it in a PIE.
R_ARM_REL32 was in the bucket of relocations that need no scanning,
so a reference to a symbol defined in a DSO was silently resolved as
if the symbol were at address zero. Route it through scan_pcrel like
PREL31 and MOVT_PREL, which gives imported data a copy relocation,
an imported function a canonical PLT entry, and a shared object an
error.
R_X86_64_GOTOFF64 and its counterparts on i386, ARM32, s390x and SH4
compute S + A - GOT, which is a link-time constant only if the symbol
is in the output file. We didn't scan them at all, so a reference to a
symbol defined in a DSO was silently resolved as if the symbol were at
address zero. Route them through scan_pcrel, which already makes the
right decision: a copy relocation for imported data, a canonical PLT
entry for an imported function, and an error in a shared object.
Fixes https://github.com/rui314/mold/issues/1668
The port kept C++ idioms in which a raw pointer or an unchecked index
stood in for a slice, guarded by an assertion. Add SyncUnsafeCell, a
slice-of-cells view for parallel passes, and use it for the file
mapping, the symbol-table allocators, the symbol editor and decoded CREL
tables. Make the fields that parallel passes write atomic, which turns
allocate_aux, the dynsym pass, the gdb_index prefix scan and the COMDAT
signature slots into safe code. Write merged sections and the gdb_index
tables through split slices, and look up files and sections with plain
bounds checks.
This removes 58 unsafe blocks and 16 assertions.
check_duplicate_symbols and check_symbol_types looked every reference
up in the symbol table, a cache miss each in a large link, before checks
on the file's own entry that skip it anyway. An undefined reference is
never a duplicate, and one without a type never mismatches.
Assisted-By: Claude <[email protected]>
R_AARCH64_TLSLE_LDST{8,16,32,64,128}_TPREL_LO12_NC were unhandled, so
objects using them failed to link with "unknown relocation". The immediate
is the local-exec TLS offset scaled by the access size.
addr_with decoded the tagged origin word into its five-variant enum up
front and matched it twice, which left the enum spilled to the stack on
every relocation. It only needs to know whether the origin is a fragment
or an input section, and each is a mask and a compare.
Assisted-By: Claude <[email protected]>
The check read every referenced symbol's ELF entry from its owning file,
a second cache miss per reference, only to get the type the symbol
already caches.
Assisted-By: Claude <[email protected]>
Sizing .rela.dyn walks every GOT symbol, a cache miss each, and the
section is sized three times on the way to the final layout. The walk
was the only serial one of that size left before the copy.
Assisted-By: Claude <[email protected]>
scan_abs_relocations already collects every R_X86_64_64 in an allocated
section, and apply_abs_rels applies it. Resolving the symbol again here
only to fall through an empty match arm is wasted work.
Assisted-By: Claude <[email protected]>
Six of the fifty relocation kinds need G, but every relocation paid for
its aux-table lookup and bounds check. Move the computation into a
closure, as the other targets already do.
Assisted-By: Claude <[email protected]>
A section's address and the GOT base do not change while its relocations
are applied, but the section offset is atomic and the context carries
interior mutability, so the compiler must reload both after every store
into the output buffer. Read them before the loop instead.
Assisted-By: Claude <[email protected]>
InputSection stores its contents as a pointer and sh_size. For a
compressed section, sh_size is the Chdr's uncompressed size while the
pointer led to the compressed bytes, so reading contents() before
uncompress() made a slice far longer than its buffer. Only a debug_assert
guarded this, and a crafted SHF_COMPRESSED .eh_frame reached it through
the .eh_frame parser.
Store a null pointer for compressed and NOBITS sections instead, so that
a nonnull pointer always has sh_size bytes behind it, and pass the
compressed bytes to uncompress() from the file's section header. The gABI
forbids compressing allocated sections, so reject them with an error.
Parallel writes into an output section went through OutputBuffer, a raw
pointer whose ranges were computed by the caller and only debug-asserted.
For ARM32 BE8 byte swapping those ranges came from mapping-symbol values
in the input file, so a corrupted symbol wrote outside of its section.
Replace OutputBuffer with for_each_member, which splits the buffer at
member offsets with rayon::iter::split and split_at_mut and hands each
member its own slice. Thunks get their slices by splitting the buffer
after the member loop, and the ARM32 swap indexes mapping-symbol ranges
within the member's slice. No unsafe code remains in either file.
Word-size absolute relocations were applied after the copy pass through
a raw pointer into the output section, with only a debug_assert on the
offset. A corrupted r_offset therefore wrote past the section in release
builds instead of failing.
Apply them in the per-member copy loop instead, indexing the member's
slice so that an out-of-range r_offset panics. abs_rels is built in
member order, so each member's relocations are one run of the vector;
AbsRel now records the member index and write_to finds the run with
partition_point. This removes write_word and its SAFETY assumption that
input relocations do not overlap.
The 80k-section tests take about 2 seconds alone but 13 to 14 seconds
on every target when the suite runs 128 jobs at once, so any other load
on the machine pushed some of them past 30 seconds. The limit exists to
catch hangs, and 60 seconds still does that promptly.
Layout carried only the file format: byte order, word size and the
record forms that differ between ELF32 and ELF64. No type implemented it
without also implementing Target, so the split earned its keep only in
RISC-V and LoongArch, where the record types depend on a const generic
and could not be chosen inside one Target impl.
Those two targets now take their record types from Class<IS_64>, a
helper private to the target module that maps the const parameter to
the ELF32 or ELF64 forms. ARM32 and SH4 lose their per-byte-order impls
and read section header words through u64::from like the other 32-bit
targets.
What the trait enumerates are targets, not architectures: Ppc64V1 and
Ppc64V2 share an architecture and differ in ABI, and Arm64 and Arm64Be
differ only in byte order. The architecture-level notion already has its
own name, Family. The rest of the tree already said target, in
Arm64Target, TargetTraits, emulation_to_target and the comments, so the
trait was the one place the vocabulary broke. The workspace directory
arch/ keeps its name because target/ at the repository root is Cargo's
build directory.
common.inc sets pipefail, and grep -q exits as soon as it finds a match.
If the producer still has output to write, it dies of SIGPIPE, the
pipeline fails, and a test that gates on the match silently skips or
fails. Plain grep reads its whole input, so the pipeline's status is
grep's own.
Clippy's match_same_arms lint reports 15 pairs. Most sit in relocation
tables that follow the relocation number order, where merging would
break the order, so only the two that are not in such tables change.
Apply clippy's redundant_pub_crate suggestion. An item in a private
module is crate-visible at most, so `pub(crate)` on it says nothing
that `pub` does not.
Apply clippy's ptr_as_ptr, borrow_as_ptr, ref_as_ptr and
ptr_cast_constness suggestions. `ptr.cast()` and `ptr::from_ref()`
cannot silently change the pointee type when a declaration elsewhere
changes, and `&raw mut x` passes a pointer to C without creating an
intermediate reference.
rewrite_opd looks up the symbol of a function descriptor by its offset
with a binary search. Aliases share a descriptor, so the search could
land on any of them, and a section-relative reference to a static
function then went through one of its global or weak aliases: it became
preemptible and resolved to whichever definition won. Use the lower
bound instead; the stable sort keeps the local symbol first.
`dist.sh arm` failed in the networked fetch step: cargo could not read
its git database for the mimalloc dependency ("Value too large for
defined data type"). ext4 hands 64-bit directory offsets to the QEMU
process, the armhf glibc reports EOVERFLOW when they don't fit a 32-bit
off_t, and the libgit2 built into the 32-bit cargo uses that interface.
Only the git dependency is affected; everything else goes through Rust's
std, which uses the 64-bit calls.
Keep cargo's cache on a tmpfs inside the networked container and have it
`cargo vendor --locked` into target/dist-vendor-$arch together with the
printed config. The offline build reads the vendored sources through
`--config` and no longer needs a cargo home at all.
Tests: ./dist.sh x86_64
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_0141irCwtPmPMZa9s7pCN3Cz
Two struct-size assertions hold only with 8-byte pointers: SymbolAux ends
in a Vec and PendingComdatSignature starts with a fat pointer, so they
are 48 and 16 bytes on a 32-bit host and rustc rejected the crate for
armv7. Gate them on target_pointer_width = "64"; the other five size
assertions have no pointer-sized fields and pass on both widths.
CI now runs cargo check for armv7-unknown-linux-gnueabihf over the whole
workspace so this cannot regress unnoticed.
Tests: cargo check --release --target armv7-unknown-linux-gnueabihf --workspace --all-targets
Tests: cargo test --release (pass=5681 skip=654 fail=0)
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_0141irCwtPmPMZa9s7pCN3Cz
The bullseye targets (aarch64, arm, ppc64le, s390x) left apt on the live
deb.debian.org mirrors. Those mirrors dropped the architectures Debian LTS
does not support when bullseye entered LTS, so `dist.sh ppc64le` and
`dist.sh s390x` fail in `apt-get install` with "Unable to locate package
build-essential". The pinned snapshot the images ship commented out in
sources.list still has them, and using it also gives these builds the
reproducibility the header comment already claims.
Make the snapshot switch the default apt_setup; riscv64 (deb822 sources)
and loongarch64 (live sid) keep their own.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_0141irCwtPmPMZa9s7pCN3Cz
Retain the original versioned reference when following undefined symbols
in shared libraries. This prevents an unversioned archive definition from
causing unnecessary extraction while preserving symbol interposition.
Skip the retained undefined references in the secondary symbol type check
and port both upstream regression tests unchanged.
Validation: full workspace tests passed with 34 unit tests and 5,684
integration tests across 14 target configurations; 651 skipped, 0 failed.
Four additional targets lacked a compiler or QEMU. The archive regression
fails before the fix, and both new tests pass on all available targets.
Upstream-Commit: 294e22a86a
Retain the original versioned reference when following undefined
symbols in shared li6braries. This prevents an unversioned archive
definition from causing unnecessary extraction while preserving symbol
interposition.
Fixes https://github.com/rui314/mold/issues/1657
Append 70,000 empty sections to GCC-generated LTO assembly for a tiny
program. Preserve the extended section numbering checks for slim and
fat LTO objects and verify that the fat object actually invokes LTO.
Validation: this test passes on all 14 available target configurations.
The observed LoongArch run dropped from 19 seconds to 4.4 seconds; the
14-target run completed in 5.7 seconds.
Replace 1,000 grep invocations with one awk process while checking all
500 DWARF32 and 500 DWARF64 names. This avoids timeouts spent repeatedly
spawning grep during parallel test runs.
Validation: the updated test passes on all 14 available target
configurations. The check accepts a complete dump and rejects missing
names and empty input.
Remove the NOBITS debug-link placeholder and remap section indices so
symbols and section headers still refer to the correct sections. Port
the upstream regression test unchanged.
Validation: separate-debug-file tests across 14 target configurations:
29 passed, 13 skipped, 0 failed. The regression fails before the fix and
passes afterward.
Upstream-Commit: f4edcbd37e
Remove the malformed NOBITS debug link and adjust section indices so
symbols and section headers still refer to the correct sections.
Add a regression test for following the debug link and preserving
symbols in later sections.
Fixes https://github.com/rui314/mold/issues/1656
The relocatable output lays out all leader CIEs before the FDE blocks
of every file, but EhFrameRelocSection::copy_buf emitted the
relocations per file: a file's CIE relocations followed by its FDE
relocations. When a file after the first has a CIE with a relocation,
which is any object with a personality routine, that relocation was
written after the previous files' FDE relocations, so .rela.eh_frame
was not sorted by offset.
Consumers pair CIEs and FDEs with a contiguous run of relocations in a
single scan over .eh_frame, and parse_ehframe is one of them. On the
unsorted section the personality relocation was attached to an earlier
file's FDE with a negative in-record offset, so the final link wrote the
value into the preceding FDE and left the CIE's personality pointer
zero. libgcc treats an encoded zero as "no personality", so functions
under that CIE lost their cleanups and catch handlers: an exception
thrown inside their try block escaped uncaught. Found by dev-util/perf,
whose build partially links a Rust workload object after C objects.
Emit the CIE relocations of every file first and only then the FDE
relocations, which matches the layout, and add a test that links a C
object before a C++ object with `-r`, checks that the relocations are
sorted, and catches an exception through the result.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_0141irCwtPmPMZa9s7pCN3Cz
The relocatable output lays out all leader CIEs before the FDE blocks
of every file, but the relocations for them were emitted per file: a
file's CIE relocations followed by its FDE relocations. When a file
after the first has a CIE with a relocation, which is any object with
a personality routine, that relocation was written after the previous
files' FDE relocations, so .rela.eh_frame was not sorted by offset.
Consumers pair CIEs and FDEs with a contiguous run of relocations in a
single scan over .eh_frame, mold included. On the unsorted section the
personality relocation was attached to an earlier file's FDE with a
negative in-record offset: we panicked on the slice index, and C++ mold
silently wrote the value into the preceding FDE and left the personality
pointer zero, which libgcc treats as "no personality", dropping that
function's cleanups and catch handlers. Found by dev-util/perf in the
Gentoo campaign; a Rust workload object partially linked after C objects
triggered it.
Emit the CIE relocations of every file first and only then the FDE
relocations, which matches the layout. The C++ implementation received
the same change.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_0141irCwtPmPMZa9s7pCN3Cz
Move shell cases and the runner directly under tests/, and remove the
standalone symbol output chunk test files and their references.
Make shell scripts executable and invoke them directly, honoring their
shebangs and execute permissions.
Context already uses E through its input files and output sections, so it
does not need a PhantomData field.
Validation: workspace check across all targets.
Reserve ReaderJob for command-line inputs. Store deferred scripts and LTO
files as tuples with required mapped-file references, and remove unused
archive metadata from the command-line job. Move reader contexts into the
script queue while preserving command-line sorting.
Validation: workspace check; 28 script, archive and LTO tests passed
(two 32-bit prerequisite skips); clang, Chrome and TensorFlow debug
outputs match the baseline.
Replace the manual registration flag with std::sync::Once. Keep counter
updates atomic and skip registration when statistics are disabled.
Validation: workspace check; clang, Chrome and TensorFlow debug links
with matching counter values under --stats.
Remove PhantomData<()> from LoongArchTarget. Its const parameter needs no
marker field, and the target remains a zero-sized type.
Validation: workspace check, including both LoongArch targets.
Replace the separate shuffle mode and seed fields with ShuffleSections.
Keep the last explicit seed while parsing, so a later bare shuffle option
reuses it even after --reverse-sections. Extend the existing seed test to
cover that option sequence.
Validation: workspace check; shuffle and reverse integration tests;
the new option sequence also passed before the implementation change.
Replace SectionOrderKind and its separate fields with a SectionOrder
enum. Keep names or numbers in their variants and retain the original
token only for address diagnostics.
Validation: workspace check and existing section-order integration tests.
Allocate character tables only for bracket expressions. Literal tokens
carry their bytes, and star and question tokens carry no data. Match the
variants directly when parsing patterns and building the NFA.
Validation: workspace check; four glob unit tests; 34 glob and version
script integration tests; clang, Chrome and TensorFlow debug benchmarks.
Replace BuildIdKind and the separate value/hash_size fields with one
BuildId enum. Each parsed option sets a complete variant, and note
emission reads its payload.
Validation: workspace check; existing build-id test; 10 checks covering
absent IDs, fixed byte values and option overrides.
Remove the redundant struct and constructors. Keep the iterator opaque
and retain always-inline dispatch for ordinary and deferred CREL records.
Validation: workspace check; 15 relocation and GDB-index tests; three
large debug links with byte-for-byte identical output.
Replace Arm32InputSectionExtra with u32, using u32::MAX for an absent
.ARM.exidx section. Add InputSectionExtra::empty so ARM32 starts at the
sentinel instead of u32's default zero.
Implement InputSectionExtra directly for Box<[RelocDelta]> and use it on
RISC-V and LoongArch. Remove the wrapper that only forwarded access to
its boxed slice.
Replace the section display wrapper and both file-name display wrappers
with formatting closures. Share the file-name formatter between input
parsing and merged-section diagnostics.
Use Rayon's recursive splitter and split_at_mut to give each task a
separate slice of the preallocated edge array. Preserve edge order and
the per-vertex count checks while removing EdgeBuffer and its unsafe
Sync implementation.
Use owned vectors for parallel thunk symbol collection, removing
ThunkSymbolBins and its unsafe Sync implementation. Atomic symbol marking
still deduplicates entries, and the final sort preserves output order.
Collect files and scripts in per-worker vectors, locking only to append
completed results. Consume the bins after the scoped jobs join and restore
command line order as before.
Keep one cache per Rayon worker and lock it once per input file. Remove the
custom UnsafeCell wrapper and its unsafe Sync implementation while
preserving cache reuse across jobs.
Report a parse error instead of silently defining address zero when a
decimal --defsym value exceeds u64::MAX, as for hexadecimal values.
Test the largest valid decimal and hexadecimal values, zero, and values
beyond the limit. The new regression fails with the original Rust linker;
all seven defsym integration tests pass with this fix.
Keep generated names as bytes and replace each non-ASCII-alphanumeric byte
with an underscore, matching C++ mold for valid and invalid UTF-8 input.
Add a regression that links references to ordinary start/stop symbols.
It passes on all 14 available target configurations and with C++ mold,
and fails with the original Rust linker.
Keep linker-script assignments, --defsym definitions and targets, retained
names, and forced undefined symbols as byte vectors through lookup and GC.
Accept raw command-line symbol names and trim retain-file entries like C++ mold.
Add a regression with distinct invalid UTF-8 and replacement-character names,
raw alias names, archive extraction, and GC roots. It passes with the Rust and
C++ linkers and fails with the original Rust linker. Existing defsym and
retain-symbols tests also pass.
Store the temporary winning file priority in an actual AtomicU32 instead of
casting an immutable u32 field to an atomic reference. Keep ordinary
symbol-index access behind getters and an exclusive setter.
Add a concurrent ownership regression verified under Miri with both Stacked
Borrows and Tree Borrows.
Store files in Vec<Box<T>> and live entries as u32 pool indices. Resolve
access through the pool so indexed access and live iteration do not rely
on cached pointers across Box borrows.
Use disjoint pool slices for mutable and parallel iteration, preserving
stable file indices while letting Box and Vec manage ownership.
Use the abort panic strategy for release builds to reduce generated
code. A panic now terminates the process without unwinding the stack.
An isolated five-run comparison across three workloads reduced .text
size by 12.7% on the local x86-64 machine and 13.6% on Fedora AArch64.
The measured link-time differences were small relative to run-to-run
variation.
Validated release builds for all 20 targets and successful links for
all three workloads on both machines.
Avoid clock reads, getrusage calls and shared record locks during ordinary
links. Explicit --perf keeps the existing timing output. Cover nested timers,
background handles and repeated stops in a unit test.
Match C++ output growth by passing the old length and appended size to
fallocate. Revisiting the already-written prefix flushes dirty extents on btrfs
and adds about a second to Chromium debug links on Fedora. The GDB-index suite
passes across available targets, and the change is neutral on ext4.
Inspect ELF symbol type and definition with readelf instead of requiring nm to
label foo as text. PPC64 ELFv1 function symbols name descriptors in .opd and
correctly appear as data to nm. This lets the existing path and reproduction
tests run across the full target suite.
Initialize reachability when creating object, LTO, and shared inputs so
mandatory objects outrank unextracted archive members during initial resolution.
Process every --undefined-glob match, including multiple matches in reachable
files, so GC retains each matching definition. Port both upstream regressions
verbatim.
Upstream-Commit: af7d056853
Mark dead fragments, copy relocations, and dead input sections as cold paths,
matching the upstream branch hints while retaining ICF leader resolution and
discarded-section diagnostics.
Upstream-Commit: 5bdd5e6284
Collect strong undefined references during the existing import/export traversal
and apply binding changes together with the import flags. Remove the later full
object-symbol scan, preserving weak binding when every undefined reference is
weak.
Upstream-Commit: a4bcbb45ae
Remember up to 32 merged-section keys and IDs in each conversion worker,
avoiding the shared lock on cache hits. Copy immutable keys rather than
retaining pointers into the growable section vector, and discard the cache when
conversion finishes.
Upstream-Commit: 4a83b5d856
Record shared-library names and default-version aliases directly into worker
bins during parsing. Reserve both symbol vectors first so their recorded
slots stay stable until the existing parallel gather fills them. FileList
retains the backing files through SONAME deduplication. This removes the
temporary name arrays and the later registration pass.
Upstream-Commit: 20eb664a7e
Load flag bytes before setting bits, skipping the read-modify-write when all
requested bits are already present. Apply the shared helper to symbol and
input-section setters; keep fetch_or for operations that need the previous
value.
Upstream-Commit: c72d19dcca
Initialize reachability when creating object, LTO, and shared inputs so
mandatory objects outrank unextracted archive members during the first
symbol resolution pass.
Process all --undefined-glob matches regardless of file reachability so
--gc-sections retains every matching definition. Cover archive ordering,
directly linked objects, and multiple matches in one object.
MergedSection::get_instance took a shared lock for every mergeable
input section, though nearly all of them resolve to the same few
output sections. Each worker now remembers up to 32 sections it has
already resolved, keyed on the same name, flags, type and entry size,
and checks them before taking the lock. The cache lives only for the
conversion pass, during which those fields are immutable.
Geometric-mean link time over the 21-program suite fell 0.87% at 32
threads on a Threadripper 7980X and 3.88% at 20 threads on an M1
Ultra, where create_merged_sections on clang-release went from 24 ms
to 10 ms.
Shared libraries inserted their symbol names into the symbol map one
at a time on the parsing thread, leaving a serial tail behind large
DSOs. Their unversioned names now go through the same per-thread bins
as object-file symbols, so gather_symbols creates the canonical
Symbols in parallel across hash shards. The symbol vector is reserved
up front, so the recorded slots stay valid until the gather fills them.
Geometric-mean link time over the 21-program suite fell 1.41% at 32
threads on a Threadripper 7980X, almost all of it from blender-release
(0.177 s to 0.142 s), which links against many shared libraries.
Relocation scanning requests GOT, PLT and TLS entries for the same
symbols repeatedly, and each request took exclusive ownership of the
symbol's cache line even when the bits were already set. Atomic's
operator|= now loads first and skips the read-modify-write when no
bit would change. fetch_or stays unconditional for callers that need
it.
Measured before the check moved from a symbol-specific helper into
the wrapper: geometric-mean link time over the 21-program suite fell
1.95% at 32 threads on a Threadripper 7980X and 2.71% on an M1 Ultra.
Use OsString for arguments and PathBuf for filesystem names so valid Unix paths
survive parsing and reach the filesystem unchanged. Preserve their bytes in
loader metadata, LTO callbacks, reproduction archives and dependency rules as
well.
Add a regression using compiler-generated objects with non-UTF-8 bytes and
spaces in their names. Cover direct arguments, nested response files, library
search, linker scripts, thin archives and reproduction replay.
Store compact chunk IDs in symbol origins instead of pointers with erased
lifetimes and target types. Resolve each ID through an append-only context
registry, preserving the 48-byte symbol layout and removing the raw pointer
dereference.
Add runtime tests for context moves, chunk reordering and 32-bit layouts, plus
compile-fail tests for dangling and incorrectly typed references. The unit
tests, API tests, doctests and 15 related native linker tests pass.
Keep SONAME bytes intact during library deduplication, dependency checks and
dynamic string and version-table output. Distinct non-UTF-8 names must retain
distinct library identities.
Add two versioned libraries whose names previously collapsed under lossy UTF-8
decoding. Verify both names in .dynstr and run the linked program. All 43 native
SONAME, dependency and version-script tests pass.
Make archive member paths absolute without resolving symlinks, matching the
names retained in response.txt. Reopen each input to copy its original contents
as before.
Add a regression that extracts and replays archives for a relative symlink and
an absolute path through a symlinked directory. Both native reproduction tests
pass.
Use the reader context to determine the target of a nested script before
accepting it. This lets an incompatible script beside its parent be skipped in
favor of a compatible script in a later library directory.
Add an x86-64 regression with competing i386 and x86-64 nested scripts. It fails
before the fix and passes in C++ mold; all 15 native linker-script tests pass.
Stop decompression when the requested output buffer is full, allowing the
twelve-byte DWARF classification probe to read a prefix of a larger frame.
Preserve the premature-end diagnostic for short output.
Add a test with verified Zstd-compressed DWARF32 and DWARF64 objects and check
their output order. The new test fails before the fix and passes in C++ mold.
All seven native Zstd and debug-sorting tests pass.
Recognize global:foo and local:* in the shared version-command parser so VERSION
blocks behave like standalone version scripts. Keep colons intact in input
filenames.
Add a regression covering local and global visibility and colon-containing input
names. Discover the new test/ directory alongside the existing shell suite. All
32 native version-script tests pass.
Record how all 27 upstream commits are represented in Rust, including
existing equivalents and C++-only changes. Describe the Rust-specific
optimizations and the completed build, integration, unit-test and static
checks.
Document the 21-workload release benchmark, build settings and noise
rejection criteria. Include raw samples supporting the measured 6.1%
improvement over the original Rust build and the remaining 2.9%
geometric-mean gap to C++.
Inline addr_with and the small auxiliary-index accessors used during
relocation and symbol-table writes. Move rare discarded-section and
.eh_frame diagnostics into a cold, non-inlined helper while retaining
ICF leader handling in the common address path.
Read the auxiliary record once when checking for PLT entries. These
changes reduce call and lookup overhead without changing computed
addresses or diagnostic behavior.
Force AhoCorasick::walk into its caller because it runs once per input
byte while matching symbol names. Keeping the state index and current
priority in the caller's registers removes per-byte call overhead. The
transition logic and highest-priority match rules stay unchanged.
Use mark_reachable for shared-library edges so repeatedly visiting an
already reachable DSO does not keep writing its reachability flag. This
avoids unnecessary cache-line traffic during parallel traversal.
Allocate child-work buffers only when a batch discovers new sections,
and transfer full buffers with mem::take. Exported symbols can already
make most sections roots, leaving many batches with no children; those
batches no longer need a heap allocation.
Feed ICF digest words directly to SipHash when its byte buffer is empty,
avoiding temporary byte arrays and buffering work during propagation.
Keep the byte-update fallback for partial buffers, preserve the native
digest byte order and wrapping length counter, and inline the hot update
and finalization paths.
Compare word updates with byte updates across partial prefixes, scalar
tails and length wraparound to verify that the digest is unchanged.
Replace the eight Option<u32> fields in SymbolAux with four-byte indices
using u32::MAX for absence. This removes separate discriminants and
reduces each auxiliary record from 96 to 64 bytes. Use u32::MAX - 1 for
the provisional dynamic-symbol index so it remains distinct from the
missing-entry sentinel.
Update table insertion and lookup helpers, assert the record size, and
test absent indices, index zero, the provisional value and reuse.
Upstream changes temporary-output-name construction to avoid a GCC
-Wrestrict warning from C++ string concatenation. Rust already builds
these names through formatting and has no matching expression to fix.
This empty commit records that no Rust change is needed.
Upstream-Commit: dad2695f3e
Allocate decoded_crel for all section indices on the first CREL section.
This avoids repeatedly growing and copying the table during section
parsing. Files without CREL allocate no table.
Upstream-Commit: ff0cb501b0
Parsing already omits losing COMDAT members and marks retained sections
alive, so skip redundant state updates for newly parsed files. Keep the
member walk when reusing parsed files, when LTO can change ownership, or
with --gdb-index.
Upstream-Commit: 91252d8aa5
Upstream increases per-thread arena blocks from 64 KiB to 1 MiB to
reduce global reservations, retaining the local-allocation cutoff at one
quarter of a block. Rust uses vectors and mimalloc rather than that
arena, so there is no corresponding block-size setting to change. Record
the C++ change with an empty commit.
Upstream-Commit: 4a11b7849d
Populate name-length caches during symbol registration. Scan global
names up to '@' or NUL, then scan the version suffix only when present,
retaining the raw length before version normalization or --wrap. This
removes a separate cache-population pass for both regular and LTO
objects.
Upstream-Commit: 53715c07ee
Mark the .dynsym, .plt and .plt.got add_symbol functions inline so the
compiler can expand their calls in the serial table-allocation loop.
This exposes the small insertion helpers to optimization while
preserving table insertion order and index assignment.
Upstream-Commit: 41d2f376c3
Reuse the version-suffix flag set during global-symbol registration
instead of scanning COMDAT signature names for '@' again. Handle a bare
trailing '@' separately because it does not set the flag.
Upstream-Commit: 3a58944ff9
Check dynamic-symbol version conflicts in parallel, using existing
symbol-map locking and synchronized diagnostics. Keep the error
checkpoint after all checks finish.
Upstream-Commit: ec7c803207
Create fragment records as section-symbol relocations are reattached,
collecting only the records each file needs in relocation order. Reserve
exactly their final slots in the central symbol vector, then construct
symbols and append their IDs to each file in parallel. This removes the
preliminary relocation-counting pass and unused dummy symbols. Retain
the cheap CREL-header capacity bound to avoid moving existing symbols.
Upstream-Commit: f9bdf4b4de
Rust already runs the synthesized-symbol, object-file and shared-file
symbol-table writers concurrently. Their symbol, string and
extended-section-index ranges are disjoint, so independent writers need
no shared append point. This empty commit records that the upstream
parallelization is already present.
Upstream-Commit: e9de4def18
Keep each dynamic symbol's name and string offset after sizing .dynstr,
reusing cached GNU hash sort keys when available. Both output writers
can then use independent parallel loops without repeating prefix scans
or name lookups. The cache costs 24 bytes per dynamic symbol.
Upstream-Commit: 18d173697c
Rust already caches exact u32 symbol name lengths while keeping Symbol
at 48 bytes. Its name accessors use that cache without rescanning long
names, so there is no corresponding layout or lookup change to apply.
This empty commit records the upstream equivalence.
Upstream-Commit: fa1e8edc39
Rust already uses flat hashbrown tables for symbol-map shards. Compare
cached hashes before strings and fold higher hash bits into bucket
placement so fixed shard-selection bits do not concentrate keys in the
same buckets. Preserve entropy in the high-bit lookup tags and keep
symbol storage stable while reducing lookup overhead.
Upstream-Commit: 841880212f
Resolve non-allocated merged sections and assign their offsets in a
background Rayon task while symbol processing, GC and relocation
scanning continue. Give the task owned metadata and immutable input
bytes, create .comment before starting it, and publish the result before
section sorting and sizing. Keep allocated-fragment layout after GC, and
reassign offsets when debug sections are reordered.
Upstream-Commit: f585fee6ca
Fill .gnu.version slots in parallel after allocating the array. Each
dynamic symbol owns a distinct slot, so these writes are independent.
Upstream-Commit: 95137b235e
Rust already collects newly marked GC roots in local Rayon worker
vectors and combines them before graph traversal. This avoids contention
on a shared concurrent vector, matching the upstream change. No source
change is needed; this empty commit records that equivalence.
Upstream-Commit: 95dbca6708
Give consecutive version-script patterns with the same version index one
priority. Stop matching when no higher-priority result is possible, and
check simple substring patterns before glob patterns.
Upstream-Commit: f8cb3daf00
Compute fragment hashes and cardinality estimates during the string or
record scan, avoiding a second traversal of the input contents and
fragment offsets.
Upstream-Commit: 0f582b0a23
Debug relocations often visit consecutive merged strings. Cache the last
symbol and mergeable-section metadata in each relocation stream, so
input sections do not share mutable lookup state. Try the next fragment
before binary search and prefetch upcoming entries.
Upstream-Commit: 2c402f707f
Process four unsigned bytes per iteration and factor the DJB recurrence
into pairs to shorten its dependency chain. Hash the remaining bytes
with the scalar loop.
Upstream-Commit: 046a644906
Add a stable partition helper that evaluates predicates and scatters
elements into disjoint output ranges in parallel. Use it for the
local/global and import/export partitions in sort_dynsyms, preserving
the relative order within both groups. Evaluate predicates sequentially
for small ranges.
Upstream-Commit: 8566358143
Prefetch symbol records ahead of their auxiliary records to hide the two
dependent pointer loads in the serial table-allocation loop.
Upstream-Commit: 9887339f72
Keep dynamic symbols grouped by their owning input file so auxiliary
records can be initialized and indexed in parallel without globally
sorting symbol IDs. File ownership makes writes independent. Preserve
first-use order within each file, suppress repeated IDs, and retain
existing auxiliary data before the serial table-allocation loop.
Upstream-Commit: 0706198355
Cache GNU hash bucket indices and name views before sorting dynamic
symbols. This avoids repeated pointer chasing, name-length lookups and
bucket calculations in the comparison function.
Upstream-Commit: cbcf79c0ef
Compute string offsets with parallel scans and write dynamic symbols and
strings in parallel. This removes serial work on large dynamic symbol
tables while preserving output order.
Upstream-Commit: 6060657622
Allocate decoded_crel for all section indices on the first CREL section.
This avoids repeatedly growing and copying the table during section
parsing. Files without CREL allocate no table.
Parsing already omits losing COMDAT members and marks retained sections
alive, so skip redundant state updates for newly parsed files. Keep the
member walk when reusing parsed files, when LTO can change ownership, or
with --gdb-index.
Populate name-length caches during symbol registration. Scan global
names up to '@' or NUL, then scan the version suffix only when present,
retaining the raw length before version normalization or --wrap. This
removes a separate cache-population pass for both regular and LTO
objects.
Reuse the version-suffix flag set during global-symbol registration
instead of scanning COMDAT signature names for '@' again. Handle a bare
trailing '@' separately because it does not set the flag.
Check dynamic-symbol version conflicts in parallel, using existing
symbol-map locking and synchronized diagnostics. Keep the error
checkpoint after all checks finish.
Create fragment symbols in the arena as section-symbol relocations are
reattached. Collect pointers locally and reserve the file's symbol
vector once before appending them in relocation order, eliminating the
preliminary counting pass.
Run synthesized-symbol, object-file and shared-file symbol-table writers
concurrently. They write disjoint symbol, string and
extended-section-index ranges; initialize section symbols before
starting these writers.
Keep each dynamic symbol's name and string offset after sizing .dynstr,
reusing cached GNU hash sort keys when available. Both output writers
can then use independent parallel loops without repeating prefix scans
or name lookups. The cache costs 24 bytes per dynamic symbol.
Store a signed 32-bit name length in Symbol's existing padding to avoid
rescanning long names. Saturate at INT32_MAX and scan the suffix only
for saturated lengths.
Replace per-key unordered_map nodes with linear-probing slots containing
cached hashes and pointers to arena entries. This reduces lookup
overhead while keeping symbol addresses stable.
Resolve non-allocated merged sections and assign their offsets in a
low-priority TBB task while symbol processing, GC and relocation
scanning continue. Create .comment before starting the task and join
before section sorting and sizing consume the results. Keep
allocated-fragment layout after GC, and reassign offsets if debug
sections are reordered.
Collect newly marked root sections in per-thread vectors, then flatten
them before graph traversal. This avoids contention on the shared
concurrent vector.
Give consecutive version-script patterns with the same version index one
priority. Stop matching when no higher-priority result is possible, and
check simple substring patterns before glob patterns.
Compute fragment hashes and cardinality estimates during the string or
record scan, avoiding a second traversal of the input contents and
fragment offsets.
Debug relocations often visit consecutive merged strings. Cache the last
symbol and mergeable section per thread, keyed by input section and
symbol index. Try the next fragment before binary search and prefetch
upcoming entries.
Process four unsigned bytes per iteration and factor the DJB recurrence
into pairs to shorten its dependency chain. Hash the remaining bytes
with the scalar loop.
Add a stable partition helper that evaluates predicates and scatters
elements in parallel. Use it for the local/global and import/export
partitions in sort_dynsyms, keeping the standard algorithm for small
ranges.
Initialize SymbolAux in the per-file collection pass, where symbol
ownership makes the writes independent. This reduces work in the serial
allocation loop.
Cache GNU hash bucket indices and name views before sorting dynamic
symbols. This avoids repeated pointer chasing, name-length lookups and
bucket calculations in the comparison function.
Compute string offsets with parallel scans and write dynamic symbols and
strings in parallel. This removes serial work on large dynamic symbol
tables while preserving output order.
Deferred LTO jobs already own archive names, but claiming each member
allocated a second copy. Move the name into the object and use the object's
archive name when setting as_needed.
Validation: all 27 native LTO tests pass. Workspace Clippy passes with
warnings as errors. All 545 upstream test files match C++ mold exactly.
Remove five Rust-only shell tests so the test cases and common support file
match the C++ suite exactly. The Cargo runner still provides test discovery and
execution.
Validation: compare every tracked test file and Git file mode against C++ mold
at 92eb17a07c60a88b3a1a8c9cef014ac84eaf0cf9; no missing, extra or modified test
cases.
Port C++ mold 92eb17a07c. Record IR files from
direct inputs, regular and thin archives, and linker scripts, then claim them
serially after sorting by their command line positions.
Remove the redundant claim mutex and once-only initialization guard. Require
mutable context access when claiming an IR file, and retain the simple
loaded-state check used by the serial plugin lifecycle.
Copy the upstream regression test unchanged. It reproduces an undefined
__cxx_global_var_init before the fix. All 27 native LTO tests pass afterward.
Port C++ mold 450832db6b. Apply the
internal-object binding rule after handling input-local symbols and relocatable
output, preserving global binding for exported synthetic symbols.
Copy the upstream symtab-binding test unchanged. It fails before the fix and
passes afterward. Native symbol-table, synthesized-symbol, start/stop, export
and relocatable-output checks: 24 passed, 1 skipped.
Port the test from C++ mold d3cf4a23e0 unchanged.
The Rust symbol and section arenas were already replaced with vectors, so there
is no large writable arena reservation to change.
Validation: the native rlimit-data test passes with the current allocator.
Port C++ mold 32626eba5d. Ignore empty SFrame
input before reading its header, as required for startup objects assembled by
GNU assembler 2.45.
Copy the upstream regression test unchanged. It failed with a corrupted-section
diagnostic before the fix and passes afterward. Native SFrame tests: 1 passed, 2
skipped because the assembler lacks SFrame v3 support.
The parallel file reader handed IR files to the LTO plugin in
whatever order it found them. LLVM's LTO assumes that the module
holding the prevailing copy of a COMDAT group arrives before the
modules holding the other copies, as it does with a serial linker;
otherwise the losing copies' internal symbols, such as
__cxx_global_var_init.N, become undefined references in the LTO
result. The reader now only records IR files and claims them after
all input files have been found, sorted by command line position.
Claims were already serialized, so no parallelism is lost, and the
serializing mutex and once_flag in lto-unix.cc are no longer needed.
Fixes https://github.com/rui314/mold/issues/1649
Co-Authored-By: Claude Fable 5.1 <[email protected]>
35f4a334 changed is_local() to derive a symbol's binding in .symtab from
its visibility. Linker-synthesized symbols never have their visibility
merged from the internal object's ELF symbols, so they went from
STB_LOCAL to STB_GLOBAL by accident. That adds _DYNAMIC, __bss_start,
_end and the like to the set of global symbols, which confuses ABI
checkers that read .symtab.
Restore the old rule for the internal object's symbols: they are local
unless we export them. The exported ones, such as __global_pointer$ on
RISC-V or __start_*/__stop_* under -z start-stop-visibility=protected,
stay global as before.
Reported by Yao Zi in https://github.com/rui314/mold/pull/1650
Fixes https://github.com/rui314/mold/issues/1651
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Keep Endian, byte order markers, and integer wrappers together in
util::endian. Add fixed-endian read/write helpers and use them from
Endian and architecture code.
Use one macro to provide new, get, and set consistently for all five
standard integer wrappers.
Make create_phdr's define and append helpers accept &ElfShdr directly.
Reuse section headers at their call sites and use short osec bindings
for optional sections.
Validation:
- cargo build --locked --workspace
- cargo test --locked --workspace: 5,487 integration tests passed,
637 skipped, no failures; all 14 library tests passed. Four cross
targets lacked a compiler or QEMU.
Rename the module directory and update its declaration, imports, and
call sites to use crate::chunks.
Validation:
- cargo check --locked --workspace
- cargo test --locked --workspace: 5,487 integration tests passed,
637 skipped, no failures; all library tests passed. Four cross targets
were unavailable because their compiler or QEMU was missing.
Give each section a sibling module containing its type and functions,
and update callers to use the direct paths. Remove the misc and version
grouping modules and redundant nested wrappers.
Validation:
- cargo check --locked --workspace
- cargo test --locked --workspace: 5,488 integration tests passed,
636 skipped, no failures; all library tests passed. Four cross targets
were unavailable because their compiler or QEMU was missing.
ConcurrentMap exposed lookup and traversal while insertion was still
possible. Move those entry points to FrozenMap and keep the construction
API limited to allocation, insertion, prefetch, and freezing.
Freeze the GDB name map before collecting sorted entry references.
Extend the existing map test to verify lookup by the original entry IDs
after freezing, including the associated keys and values.
ConcurrentMap::prefetch only issued a cache hint on x86-64. Move the hint
into a shared utility and use stable inline assembly for x86, ARM32/64,
RISC-V32/64, PowerPC32/64, s390x, and LoongArch64.
Assume SSE on both x86 variants. Encode RISC-V PREFETCH.R as its ORI hint
so it remains a no-op without Zicbop. Keep unsupported architectures as
no-ops and preserve the map's empty-table guard.
Move the Linux and Android MADV_HUGEPAGE calls into shared utilities,
with no-op implementations on other targets. Reuse them for output
mappings, concurrent-map storage, and symbol allocations, keeping the
symbol advice limited to complete pages within the allocation.
Extract platform-specific allocation and deallocation helpers for the
concurrent map, and guard page-alignment rounding against overflow.
The 8 GiB arena was mapped read-write, so on hosts with overcommit disabled
(vm.overcommit_memory=2) Linux ignored MAP_NORESERVE and charged the whole
reservation against the commit limit, and fork() charged it again for the
child. Even a hello-world link then failed with "fork: Cannot allocate
memory". Reserve the range with PROT_NONE instead and make it writable in
64 MiB chunks as the bump pointer advances, as the Windows code already does
with MEM_COMMIT. Only the chunks in use are charged, by mmap and by fork()
alike, and transparent huge pages still back them.
Under RLIMIT_DATA, which counts writable private mappings the same way, a
trivial link now needs 0.5 GiB instead of more than 8 GiB.
Fixes https://github.com/rui314/mold/issues/1644
Co-Authored-By: Claude Fable 5.1 <[email protected]>
InputSectionId stored an already packed integer while FragmentRef kept
its component indices. Store a private ObjId and zero-based u32 index in
InputSectionId, with direct field access in file() and index(). Confine
the packed representation to raw() and from_raw().
Keep the eight-byte size and the existing Origin bit layout. Use the
reserved u32::MAX index for NONE and apply the index-plus-one encoding
at the packing boundary so its payload remains zero. Extend the existing
round-trip test to cover zero indices, maximum supported indices, and
the placeholder value.
Origin was assembling and splitting FragmentRef's two indices itself,
while InputSectionId already supplied raw and from_raw methods. Add the
same encoding interface to FragmentRef and use it from Origin::new and
Origin::get.
Keep FragmentRef's typed fields and preserve the existing packed layout.
Origin still checks the 30-bit merged-section limit before shifting the
payload to make room for its tag.
ObjectFile section access and output-section construction already know
the owning file ID, but building a section ID reloads it from each input
section. Construct IDs from the known owner and the decoded dense index,
and pass the owner once into regular_ids_mut.
Move paired ID/reference lookup to ObjectFile so it resolves the ELF
section index once. Reuse the validated dense-index accessors for shared
and mutable section access, avoiding redundant owner loads and checks.
InputSectionIds are assigned when a section is inserted into a file's
dense vector, and their file indices refer to the stable file pool.
Resolving these IDs currently repeats bounds checks for both indices on
hot section-access paths.
Add internal unchecked accessors for the pool and dense section vector,
and use them in Context::input_section. Document their insertion-based
preconditions and retain debug assertions on both bounds. Keep the
unsafe operations within these accessors and the ID resolution method.
The early fragment estimate scans ordinary relocation tables before
archive extraction, including members that may never be selected. That
adds work to large links such as ClickHouse just to predict capacity.
Restrict the early estimate to cheap CREL header counts from direct
inputs and already reachable archive members. Leave ordinary relocation
counting to the existing preparation of selected files after COMDAT
selection. This retains the early reservation needed by CREL-heavy links
while avoiding an extra scan of ordinary relocation records.
Rewriting mergeable-section relocations can append millions of dummy
symbols after global symbols have been constructed. Reserving only local
symbols allows this later append to grow the vector and copy the already
populated table, which is particularly costly for CREL-heavy links.
Estimate fragment demand before gathering globals, using relocation
counts from CREL headers and inspecting ordinary relocations that refer
to mergeable section symbols. Include that estimate in the initial
reservation and compute it across files in parallel. Reapply huge-page
advice only when vector capacity actually changes.
The ClickHouse benchmark contains C++ symbol names longer than the u16
limit, causing valid input to panic in Symbol::new. Restore u32 storage
for the cached name length and retain the checked conversion into that
representation.
Change the boundary test to construct a 65536-byte name successfully and
verify that Symbol::name returns all of its bytes. This removes the
restriction introduced by narrowing the length field.
Replace obsolete arena terminology with the actual storage and lifetime
rules: symbols and section members live in vectors, input files belong
to a stable pool, and parallel construction reserves disjoint ranges
before exposing writable slots.
Remove stale ArenaPtr distance restrictions and describe background
debug-info work in terms of tasks and concurrency. These are comment
updates; allocation, ownership, and linker behavior are unchanged.
Reserve an upper bound for local symbols when gathering globals so that
appending locals does not immediately move a populated symbol vector.
Request transparent huge pages for whole pages inside that allocation,
leaving partial boundary pages that may contain allocator metadata
alone.
Let per-file section vectors grow with the inputs actually inserted;
reserving for every ELF header wastes space on relocation and metadata
headers that never become InputSections. Validate the packed file-index
limit when inserting files into the pool, and use debug assertions for
the redundant checks when encoding and decoding validated section IDs.
Section lookups were constructing an InputSectionId and decoding it
again even when the caller only needed a reference. Factor the ELF
section table decoding into input_index and use the resulting dense
index directly for shared, mutable, and paired ID/reference access.
Keep regular entries and entries with merge metadata on the same lookup
path. Extend the Origin round-trip test to cover the full supported
32-bit section-index range.
Have InputSectionExtra expose r_deltas_mut, returning a mutable
reference to the owned delta array. InputSection::set_r_deltas can then
perform the assignment itself while retaining its nonempty-array
assertion.
RISC-V and LoongArch return their embedded field. Targets without
relaxation metadata retain the unreachable guard for attempts to mutate
it, keeping the target-specific implementation limited to field access.
Rename MergeableSection to MergeInfo and update the section-table flags
and accessors to describe its role as metadata for an ordinary input
section. Update merging, symbol attachment, and debug-info callers to
use the new names.
Preserve the existing ownership model: eligible SHF_MERGE inputs remain
in their dense slots and are marked dead while their contents are
represented by fragments. Store merge metadata only for those inputs,
without adding a merge-info field to every InputSection.
Logical section IDs and directly owned target state allow input sections
to move when their storage grows. Store InputSection values in each
SectionList's dense vector and resolve IDs by indexing that vector. Keep
merged inputs in their slots so existing references remain valid.
Remove SectionArena, its thread-local allocation blocks, and the manual
section destruction code. Vector ownership now supplies allocation and
destruction, and the shared virtual-memory reservation module can be
deleted because neither symbols nor input sections use it anymore.
The self-relative pointer to InputSectionExtras prevents input sections
from moving and requires a separate arena allocation for target state.
Make InputSection generic over Arch and embed an associated
InputSectionExtra value directly in each section.
ARM32 stores its exception-index section link, and RISC-V and LoongArch
own their relaxation-delta arrays. Other targets use a zero-sized extra
type. Propagate the target type through section owners and callers, and
remove the self-relative pointer encoding and extras allocator.
Arena-relative section IDs tie callers to the location of an allocation.
Represent InputSectionId as an object-file ID and a dense index within
that file, packed into 64 bits. Reserve two bits for Origin's tag and
keep zero available as the placeholder value.
Add a dense per-file slot table that still resolves to the existing
arena during this transition. Update symbol origins, output-section
members, relaxation, and layout lookups to resolve the logical ID
through its owner. This allows section storage to change independently
of the IDs used throughout the linker.
Symbols are already referenced through SymbolId, so their addresses only
need to remain stable while a construction or mutation phase borrows
them. Replace the 8 GiB SymbolArena reservation with Vec<Symbol> and let
the vector own and release the initialized symbols.
Keep the existing parallel construction scheme: reserve capacity before
publishing writable slots, assign disjoint ranges to workers, and set
the initialized length after they finish. Remove the custom virtual
memory allocation, slice access, and destruction code for symbols.
Use u16 for Symbol::name_len, reducing the width of the cached length
field while keeping the existing thin pointer to the name bytes. This
representation imposes a maximum symbol name length of 65535 bytes.
Check the conversion in Symbol::new so that longer names produce an
explicit diagnostic. Add a regression test for a 65536-byte name to
exercise that boundary.
Let the compiler decide whether to inline private and generic helpers that do
not correspond to explicitly inline C++ code. Keep forced inlining on the hot
ELF and relocation paths identified by the C++ implementation and perf
profiles.
Firefox-debug performance remained neutral in both single-threaded and
32-thread runs. The full test suite and release clippy checks pass.
Merge duplicated C++ and Rust comment blocks into single explanations that
describe the Rust implementation while retaining the original rationale.
Remove stale C++ type references and restore the missing rustdoc summary for
sorted concurrent-map entries.
Remove helpers, accessors, and constants that are unreachable from the mold
command and its tests. This includes wrapper methods whose only callers were
other dead wrappers, unused ELF integer operations, and unused concurrent-map
APIs.
Keep the remaining internal len methods crate-visible so Clippy does not treat
them as incomplete public collection APIs.
Keep Origin eight bytes and use its low two bits to distinguish input
sections, output chunks, merged-section fragments, and symbols. Remove the
separate origin-kind bits from Symbol’s visibility byte.
This follows C++ mold’s tagged-pointer representation without the auxiliary
tables and per-variant codecs required by the discarded 32-bit experiment.
The experiment reduced Symbol from 48 to 40 bytes, but it required a
manual codec for every ChunkId variant, a separate namespace for allocated
merged sections, and context-dependent origin decoding. It added 434 net
lines.
It also made the previous Rust linker about 2% slower on Firefox debug and
10% slower on TensorFlow debug, even though it remained faster than C++ mold.
Prefer the simpler 64-bit origin until the memory saving justifies those
costs.
Use unary low-bit tags so fragment origins have a 31-bit payload. Pack
allocated merged-section IDs and fragment bucket IDs into that payload,
and use compact encodings for input sections, chunks, and symbols.
Also compact the remaining symbol metadata so that Symbol shrinks from 48
to 40 bytes.
Split the output-section and symbol-table borrows so
gather_thunk_addresses can record addresses while traversing thunks. This
matches the C++ loop and avoids a temporary allocation for each section.
Replace the callback-based internal parser with an Iterator built using
std::iter::from_fn. Keep the eager Vec results and member creation order
unchanged.
Return the original io::Error from the shared file-opening path instead of
discarding it and reading errno later. Keep optional file lookup semantics by
mapping NotFound to None, and remove errno_string.
Rewrite comments copied verbatim from C++ so they describe the Rust
implementation and terminology. Remove notes that have no Rust counterpart,
and update source-file references to their Rust names.
Keep C++ comparisons when they document an intentional implementation
difference or a remaining performance gap.
GNU assembler 2.45 emits a zero-size .sframe section for an input file
that needs no unwind info, such as glibc's Scrt1.o on Ubuntu 25.10.
parse_sframe() read a header past the end of such a section and reported
it as corrupted.
Fixes https://github.com/rui314/mold/issues/1646
Co-Authored-By: Claude Fable 5 <[email protected]>
Mold performs one link per process, so keep diagnostic settings, error state,
and output serialization in a process-wide singleton. This removes
diagnostic plumbing from contexts and helper APIs while preserving the mutex
that keeps messages from worker threads from interleaving.
Port mold dist.sh to build Rust release binaries in old distro
containers. Download a pinned official Rust toolchain as gzip, fetch
Cargo.lock dependencies before disabling the network, and normalize
package timestamps.
Allow install-mold.sh to consume an external Cargo target directory.
path_clean("../../a/b") returned "a/b": the ParentDir branch popped a
trailing ".." from the output instead of appending another one, so any
path with two or more leading ".." components collapsed into a wrong
path. C++ mold uses std::filesystem::path::lexically_normal, which
keeps leading ".." components intact.
Library search paths are cleaned with this function, so mold failed
with "library not found" whenever a Makefile referenced an in-tree
library as -L../../<dir>, which is a very common pattern. Found by the
Gentoo all-package campaign within its first hour: sys-fs/lvm2,
sys-fs/xfsprogs, sci-astronomy/montage, dev-libs/OpenNI2,
dev-erlang/fast_yaml, and app-backup/spideroak-bin all failed with
mold-rust and built with mold-c++.
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_0118ERbwNepcG1Z9gh5cm6Ut
Benchmark record, 2026-09-01 (unrelated to this change; this tree is
what was measured, built as 6becb4da before this message was amended):
C++ reference: mold at e322a3cc, i.e. the translation base fbaaa75f
plus 15 fixes that do not touch hot paths.
Protocol: ~/bench/tab4-bench.sh as-is — run 1 warms the page cache,
median of runs 2-6 reported; all runs of one linker complete before
the other linker starts; sync + dirty-writeback wait between runs;
--no-fork; default thread count (min(ncores, 32) = 32 on this
128-core machine); frequency boost off, performance governor,
THP madvise, idle machine.
Workload C++ (s) Rust (s) Rust vs C++
blender-debug 0.94 0.82 -12.8%
blender-release 0.20 0.17 -15.0%
chrome-debug-arm64 1.88 1.97 +4.8%
chrome-debug-x86_64 1.75 1.80 +2.9%
chrome-release-arm64 0.77 0.79 +2.6%
chrome-release-x86_64 0.66 0.69 +4.5%
clang-debug 1.49 1.28 -14.1%
clang-release 0.11 0.10 -9.1%
clickhouse-debug 1.14 1.09 -4.4%
clickhouse-release 0.43 0.43 +0.0%
firefox-debug-arm64 1.07 0.98 -8.4%
firefox-debug 0.87 0.87 +0.0%
firefox-release 0.21 0.20 -4.8%
godot-debug 0.51 0.47 -7.8%
godot-release 0.08 0.08 +0.0%
libmergedlo.so-debug 0.46 0.47 +2.2%
libmergedlo.so-release 0.20 0.18 -10.0%
pytorch-debug 0.93 0.84 -9.7%
pytorch-release 0.15 0.14 -6.7%
tensorflow-debug 3.16 2.89 -8.5%
tensorflow-release 0.69 0.63 -8.7%
Geomean -5.1% (Rust faster); 13 faster, 3 even, 5 slower. No workload
regressed against the import-time table in the initial commit; the
open item remains chrome's +2-5%.
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_0118ERbwNepcG1Z9gh5cm6Ut
Apply relocation deltas, adjust symbols and recompute executable output
section layouts in parallel, matching the three file and chunk parallel
loops in C++ mold.
Process object files in parallel as C++ mold does. Use the existing
disjoint-range output-buffer abstraction so each worker can reverse its
input sections in place.
Process object files in parallel as C++ mold does. Serialize symbol access
with the existing byte-sized symbol lock and rewrite each file's relocation
records in place.
Use stock zlib's low-level API to reproduce the C++ Z_BLOCK, deflatePending,
optional deflatePrime and Z_SYNC_FLUSH sequence. This avoids the old
libbacktrace boundary bug and preserves C++ mold's compressed bytes.
Install the Unix handlers with SA_SIGINFO and track the mapped output range
as it is created, extended or moved. Report a possible full disk only when
SIGBUS or SIGSEGV identifies an address in that range.
Port the C++ Counter facility and collect the same parser, section, symbol,
ICF, relocation, COMDAT, thunk and merged-section statistics. Keep counter
updates dormant unless --stats is requested.
get_api_version handed the plugin the buffer of a stack-local
std::string, which dies when the function returns. Make it static.
Co-Authored-By: Claude Fable 5 <[email protected]>
The thin archive reader skipped a __.SYMDEF symbol table without
advancing the read position, so it kept parsing the same member header
forever. Skip past the member contents like the other symbol table
branch does.
Co-Authored-By: Claude Fable 5 <[email protected]>
R_RISCV_TLSDESC, R_RISCV_GOT32_PCREL, R_SPARC_SIZE32 and R_SPARC_JMP_IREL
were printed as "unknown (n)" in diagnostics.
Co-Authored-By: Claude Fable 5 <[email protected]>
parse_defsym_value returned a default-constructed variant on a bad hex
string such as 0xz, which holds a null Symbol pointer. The null pointer
was then dereferenced when marking GC roots, crashing the linker instead
of printing a diagnostic.
Co-Authored-By: Claude Fable 5 <[email protected]>
reattach_section_pieces rewrote every symbol whose local definition is in
a mergeable section, including global symbols that lost resolution to a
definition in another file. The loser then redirected the symbol to its
own section piece. Since the pass runs on all files in parallel, it was
also a data race on the shared Symbol.
Co-Authored-By: Claude Fable 5 <[email protected]>
R_390_12, R_390_PC12DBL, R_390_PLT12DBL, R_390_GOT12 and R_390_GOTPLT12
OR'd their 12-bit value into the instruction through a little-endian
16-bit accessor. s390x is big-endian, so the value landed in the wrong
bytes, corrupting both the displacement and the base register field.
Co-Authored-By: Claude Fable 5 <[email protected]>
create_phdrs walks a filtered, address-sorted copy of ctx.chunks, but the
loop that merges adjacent note sections into one PT_NOTE examined
ctx.chunks with an index into that copy. The two vectors differ e.g. when
the ELF and program headers are not memory-mapped, in which case every
note section got its own PT_NOTE segment.
Co-Authored-By: Claude Fable 5 <[email protected]>
When a merged section is laid out in two regions, 32-bit-addressable
fragments followed by the rest, the loop that rebases per-shard offsets to
section-global ones started at shard 1. That is fine for the 32-bit region,
whose shard 0 needs no adjustment, but it left shard 0 of the 64-bit region
unadjusted, so its fragments overlapped the beginning of the section. Start
at shard 0; the 32-bit fragments of that shard just get zero added.
Co-Authored-By: Claude Fable 5 <[email protected]>
The alloc relocation path incremented the local pointer loc instead of the
byte it points to, so the relocated byte was left untouched. The result was
wrong whenever an 8-bit label difference spanned code whose size changed
during relaxation. The non-alloc path already did this correctly.
Co-Authored-By: Claude Fable 5 <[email protected]>
The non-alloc relocation paths of i386, ARM64, s390x, SPARC64 and PPC64V1
passed the relocation value and the relocation index to check_range in the
wrong order. The intended range check was thus never performed, and if the
relocation index happened to fall outside the value range, the error path
indexed the relocation table with the relocation value, reading out of
bounds.
Co-Authored-By: Claude Fable 5 <[email protected]>
Use the same KERN_PROC_PATHNAME sysctl as C++ mold when locating the
current executable on FreeBSD, where /proc may not be mounted. Run one
shell test at a time under sanitizers so large cases do not overlap,
while leaving two CPUs available to the linker under TSan.
Use Cargo per-target features in native and platform jobs so they do not
rebuild all 20 linker instantiations. Tests that require an i386 linker
already skip when that target is disabled. Keep test-all-targets using
defaults for complete QEMU coverage.
If an object file has SHN_LORESERVE (0xff00) or more sections, e_shnum
is zero and the actual number of sections is stored in the first section
header's sh_size field. is_gcc_lto_obj() used the raw e_shnum, so it
scanned no sections and misclassified such LTO objects as regular
object files.
Fixes https://github.com/rui314/mold/issues/1642
Co-Authored-By: Claude Fable 5 <[email protected]>
Give ThreadSanitizer extra time for the many-output-sections test.
Disable incremental artifacts and debug info in the disk-constrained FreeBSD VM.
Also suppress the non-x86 unused prefetch parameter warning.
Add the four platform jobs from C++ mold and make the Rust implementation
compile on each native platform.
Use the corresponding Windows virtual-memory, output-file, subprocess, LTO
and process-timer facilities. Keep the Unix paths for macOS and FreeBSD while
applying platform-specific system calls only where C++ mold does.
Tested with cargo build --release, cargo test and cargo clippy --release
--all-targets. The MinGW executable also links against dlfcn-win32.
The all-target CI job exhausted its runner disk because it retained the
output of every completed shell test. Remove each work directory after a
passing or skipped test while preserving logs and all failed-test files for
diagnosis.
The multi-architecture build generated incremental and debug artifacts on
an ephemeral runner, leaving too little room for the test outputs and
causing the one-hour timeout. Disable both for this job.
In Ubuntu 24.04 Podman, the clean build used 693 MB and the completed test
tree used 7.4 GB. All targets passed with pass=6782 skip=895 fail=0.
ARM BE8 instructions are little-endian, but data embedded in PLT and
thunk entries uses target byte order. Match C++ mold's ul32 instruction
templates and U32<E> literal stores.
CI-equivalent Ubuntu 24.04 Podman:
all targets: pass=6782 skip=895 fail=0
armeb: pass=368 skip=53 fail=0
All workspace crates are built before testing, but the target crates and CLI
binary contain no unit tests. Select the library and integration test targets
so Cargo does not exhaust the runner disk linking empty harnesses.
musl uses an unsigned type for statfs.f_type while libc exposes
TMPFS_MAGIC as a signed value. Convert the constant to the field's type so
the comparison builds with both musl and glibc.
Match C++ mold by attempting fallocate when fstatfs fails.
The job-level environment did not populate the container configuration, so
the Rust toolchain action still lost the system directories when it extended
GITHUB_PATH. Set PATH through container.env instead.
The Tumbleweed image does not declare PATH in its container
configuration. GitHub's implicit default disappears when rust-toolchain
adds Cargo's bin directory through GITHUB_PATH. Define the normal system
path explicitly for the distro matrix so Bash remains available.
Run install-test-deps.sh with /bin/sh so minimal container images can
invoke it. Expand the Debian cross-toolchain package list without Bash
brace expansion, install Bash on Tumbleweed, and remove the Alpine
bootstrap step from CI.
The Alpine base image does not include Bash, so install it before
running install-test-deps.sh. Keep the dependency script Bash-based so it
can use brace expansion.
Run the complete multi-architecture suite and native tests across the
same Linux distribution matrix as C++ mold. Install the extra toolchains
used by the upstream tests and fail CI if any expected target is
unavailable.
Build and test ASan and TSan configurations with nightly Rust. Use the
system allocator for sanitizer builds and instrument the two C adapters as
well. Limit TSan to two CPUs to avoid false reports from Rayon while still
exercising parallel execution.
Make install-test-deps.sh portable across the distribution containers.
Tested with cargo test: pass=5360 skip=636 fail=0. ASan passed 477
native tests with 9 skips, and TSan passed 454 native tests with 32 skips.
Match C++ mold's relaxation behavior by updating input relocation records
at the point where instructions are rewritten. Store decoded CREL records
in typed mutable side tables so emitted relocation sections observe the
same updates, and remove the relocation type reconstruction hooks.
Use I386, X86_64, M68k, and Sparc64 as representative layouts when
classifying GCC LTO objects, matching C++ mold. Use real target layouts for
the compile-time record assertions and remove Elf32Le, Elf64Le, Elf32Be, and
Elf64Be.
Also remove the obsolete Record trait. Direct file records use
FileRecord, and Record had no implementations or callers.
Checked with cargo check --workspace. The test suite was not rerun as
requested.
Rename ProgramHeaderRecord and CompressionHeaderRecord to the shorter
PhdrRecord and ChdrRecord names. Expand the four plain layout definitions
directly and remove the plain_layout macro.
Infer the ELF class from the layout word size and whether relocations
carry addends from RelRecord. This removes the duplicated IS_64 and IS_RELA
declarations from every target.
Tests: cargo test (pass=5360 skip=636 fail=0)
Lint: cargo clippy --release --all-targets
Keep input section headers and symbols in their endian-aware on-file
representation instead of decoding them into target-independent mirrors.
Carry that representation through output chunks and expose fields through
record traits, so reads and writes remain target-endian.
Remove SymbolEntry, SectionHeader, SymTable, ShdrTable, and RecordLayout
now that no converted record mirrors remain. Move the SFrame constants to
elf/consts.rs.
On the 21-workload, 32-thread benchmark, the geometric-mean time changed
by -1.1% versus 801fc. Excluding tensorflow-debug's unusually large
improvement, it was effectively flat at -0.1%.
Tests: cargo test (pass=5360 skip=636 fail=0)
Lint: cargo clippy --release --all-targets
Represent uniform ELF records with public target-endian fields and select
class-dependent symbol, program-header, and compression-header layouts
through Layout. Convert differing layouts as complete records instead of
adding field-by-field forwarding accessors.
Remove the raw st_info API and the old offset-based record readers.
Tests: cargo test
Tests: cargo clippy --release --all-targets
Define byte-backed integer and relocation types that mirror the C++ ElfRel
specializations, including the SPARC64 type-data field.
Select each relocation record through Layout and view input and output
tables as ordinary typed slices. This removes the host-native conversion
wrappers and the synthetic r_info representation.
Preserve target byte order independently of the host.
Tests: cargo test
Tests: cargo clippy --release --all-targets
Define relocation constants and diagnostic-name functions for each
architecture from a single macro invocation. Keep each macro input to the
relocation name and value, and use stringify! to produce the diagnostic
spelling.
Remove the separate relnames module so relocation names and values can no
longer drift apart.
Validation passes with cargo build --release and cargo clippy --release
--all-targets. Cargo test reports 5360 passed, 636 skipped, and no failures.
Replace the callback-only relocation traversal API with streaming Rust
iterators. Make CrelReader and RelocationIter implement Iterator while
preserving deferred CREL decoding without allocation.
Use ordinary for loops in the architecture backends and gdb-index reader.
Keep a specialized fold implementation so Iterator::for_each can select the
ordinary or CREL representation once.
The full test suite passes with 5360 cases passed, 636 skipped, and no
failures. Clippy is clean.
Five-run firefox-debug medians improved from 0.798 s to 0.765 s at 32
threads and from 11.135 s to 11.002 s with one thread. The C++ reference
at fbaaa75f measured 0.792 s and 11.999 s, respectively.
Configure Rayon's process-global pool after target detection and process
forking. Register the main thread as worker zero so --threads=N limits the
linker to N active threads.
Run the linker directly instead of wrapping its entire body in
ThreadPool::install. This follows mold's command-oriented execution model
and its process-wide TBB parallelism limit.
Rename Rust modules to match their C++ source counterparts where
practical. Use archive_file, cmdline, error, output_chunks,
shrink_sections, and perf while keeping mapped_file, output_file, lto,
and subprocess unchanged.
Remove filename-only comments added during the port. They were not copied
documentation and became redundant once the filenames were aligned.
Update all module paths and target entry points for the renamed modules.
Carry over mold's distribution-specific dependency installer as
install-test-deps.sh. Drop CMake packages that the Rust build no longer uses.
Keep test setup out of the user build instructions. Explain the stable Rust
and native C compiler requirements directly in README.
Copy the current README, manual, logos, and license from mold. Adapt the
build and installation instructions for the Rust implementation.
Add install-mold.sh to install the binary, preload library, manual, license,
and linker symlinks under a configurable PREFIX with DESTDIR support.
Move the two remaining C source files from csrc/ to c/ and update the
build script and LTO documentation to use the new paths. The files are
otherwise byte-identical.
Tests: cargo build --release
Tests: cargo test (pass=5360 skip=636 fail=0)
Tests: cargo clippy --release --all-targets
Tests: cargo fmt --all -- --check
Replace the local allocator wrapper and vendored mimalloc sources with the
rui314/mimalloc_rust repository pinned at:
cc58b72775b8b5bc89ee37752af61fcb4f7c2909
The dependency contains mimalloc 3.5.0 and its overlapping upstream files
are byte-identical to the deleted copy. Update the workspace, allocator
import, lockfile, and README.
Tests: cargo build --release
Tests: cargo test (pass=5360 skip=636 fail=0)
Tests: cargo clippy --release --all-targets
Tests: cargo fmt --all -- --check
Register a custom integration-test target for mold-cli and move the Rust
runner into the mold-tests workspace crate. Keep all 537 shell-test files
byte-for-byte unchanged under tests/cases, run every available target and
QEMU job by default, and stage Cargo-built test artifacts in the active
profile directory.
Remove run-tests.sh and make every workspace crate a default member so
plain cargo test is the complete test entry point.
Use explicit wrapping arithmetic for unsigned C++ operations exposed by
the debug-profile test suite.
Tested with:
cargo test
cargo build --release
cargo clippy --release --all-targets
cargo fmt --all -- --check
Full shell-suite result: 5,996 scheduled, pass=5360, skip=636, fail=0.
Treat every C++ implementation comment at mold commit
fbaaa75f9f as a first-class porting
asset. Restore all 2,179 non-namespace comment occurrences at their
Rust counterparts, including file-level explanations, ABI notes,
examples, TODOs, original typos, inline instruction annotations, and
repeated comments. Add provenance where one Rust implementation
consolidates multiple C++ implementations. Closing namespace labels
are the sole comments without a Rust construct.
Verify that all 537 upstream test files remain byte-for-byte identical
to the pinned C++ tree.
The audit also exposed a few implementation-fidelity gaps: reopen
input mappings before writing a repro archive, reset discarded-COMDAT
symbol indices after owner selection, mirror C++ GOT
dynamic-relocation accounting branches, and reserve version-section
output storage as C++ does. These are faithful translations, not new
algorithms.
Validated with cargo build --release, ./run-tests.sh --all (pass=5360
skip=636 fail=0), cargo clippy --release --all-targets, cargo fmt
--check, and git diff --check.
Mirror C++ ConcurrentMap sorted-entry handling: reserve each shard vector, use unstable sorting, retain direct stable entry references, and test occupancy through the atomic key pointer alone.
Store direct stable NameEntry references in indexed-name records so the table writer does not translate a bucket ID for every public-name occurrence.
Reference C++ mold: fbaaa75f9f
Validation:
- cargo build --release
- ./run-tests.sh --all: pass=5360 skip=636 fail=0
- cargo clippy --release --all-targets
32-thread benchmark, median of five, with sync before and sync plus sleep 1 after every linker invocation:
- chrome-debug-arm64: C++ 1.736 s, Rust 1.834 s (+5.65%)
- chrome-debug-arm64 --no-gdb-index: C++ 1.576 s, Rust 1.592 s (+1.02%)
- chrome-release-arm64: C++ 0.715 s, Rust 0.738 s (+3.22%)
The remaining Chrome debug difference is isolated to TBB low-priority arena behavior; Rayon has no corresponding task-priority feature.
mold demangles with the platform's __cxa_demangle, and FreeBSD's libcxxrt
spells `operator delete[]` as `operator delete []`, so the quoted name in
the test never matched there. Test literal matching of brackets with
`S::operator[](int)` instead, which every demangler spells the same way.
Co-Authored-By: Claude Fable 5 <[email protected]>
So far -z rewrite-endbr was implemented only for x86-64, where it
rewrites unnecessary ENDBR64 landing pads (Intel CET) with NOPs for
functions whose addresses are not taken. arm64 has an equivalent
control-flow-integrity feature, BTI, whose landing pad is `bti c`.
This commit generalizes rewrite_endbr into a template shared by both
targets: the landing pad and NOP byte patterns are selected per-arch
(ENDBR64/NOP on x86-64, `bti c`/`nop` on arm64). The whole-program
analysis that preserves landing pads for address-taken functions is
unchanged.
Unlike x86-64, arm64 needs range extension thunks, and a thunk reaches
its target with an indirect branch (`br x16`). A function that is only
called directly can therefore still be entered indirectly once its call
is routed through a thunk, so we must keep its landing pad. We preserve
the landing pad of every thunked symbol; the thunk symbol set has
already been narrowed to the ones that actually need a thunk by
remove_redundant_thunks().
Note that on arm64 with recent glibc, `main`'s landing pad is
legitimately removed: the CRT passes a local __wrap_main stub (whose
landing pad is kept) to __libc_start_main and reaches main through a
direct branch, which does not require a landing pad.
Also note that code built with -mbranch-protection=standard (BTI +
pointer authentication) starts functions with `paciasp`, which doubles
as a landing pad, so there is no separate `bti c` to rewrite and this
option has no effect on such code.
This was locally tested on an Apple M2 machine running Asahi Linux.
Fixes#1552.
The relocation has a 16-bit displacement from the TOC pointer, but the
ELFv2 backend applied it without a range check and silently truncated
displacements to .toc entries beyond 32 KiB, producing a load from the
wrong address. The ELFv1 backend already checked it; do the same here.
Co-Authored-By: Claude Fable 5 <[email protected]>
FDEs refer to their functions with 32-bit PC-relative relocations, and
.eh_frame_hdr stores 32-bit offsets, but mold applied them without any
range check. A function more than 2 GiB away from .eh_frame, which
happens with a huge text section or with -Ttext or --section-start
placing code far from the default layout, silently got a wrapped-around
FDE that the unwinder cannot use, while GNU ld and lld reject such
links. Those relocations are now range-checked like all others, and so
are the .eh_frame_hdr entries.
The aarch64 long thunk test placed code with unwind tables 4 GiB away
from .eh_frame and relied on the bogus FDEs going unnoticed. It now
compiles the far functions without unwind tables and keeps the far
section fixed while swapping the functions between the sections, so
calls in both directions are still covered.
Co-Authored-By: Claude Fable 5 <[email protected]>
GNU ld takes a quoted name in a version script or a dynamic list as a
literal string, while an unquoted one is a glob pattern. mold dropped
the quotes and treated everything as a pattern, so `"operator
delete[](void*)"` in an `extern "C++"` block never matched anything
because `[]` parsed as an empty character class. Glob metacharacters in
a quoted name are now escaped before the name is handed to the matcher.
Co-Authored-By: Claude Fable 5 <[email protected]>
The tokenizer keeps colons within a token for the C++ scope operator,
so `local:*;` lexed as a single token that the parser then took for a
global symbol pattern named `local:*`, and nothing was localized. The
label is now split off such a token.
Co-Authored-By: Claude Fable 5 <[email protected]>
mold took the instruction set of a branch target from the LSB of the
symbol value alone, but only STT_FUNC symbols encode the instruction
set that way. A Thumb BL to a plain label in hand-written assembly,
which is NOTYPE and has an even address, was rewritten to BLX and
crashed with SIGILL, and a Thumb B.W to it went through a mode-switching
thunk. Like GNU ld, we now switch the instruction set only for function
symbols and for PLT entries, which are always ARM code.
Co-Authored-By: Claude Fable 5 <[email protected]>
With -Os, GCC calls linker-synthesized routines not only to save and
restore general-purpose registers but also floating-point and vector
registers. mold only provided the former, so such links failed with
undefined symbol errors. The new routines use the same layout and
encodings as GNU ld; the section that holds them is renamed to
.save_restore_regs accordingly.
Co-Authored-By: Claude Fable 5 <[email protected]>
With -r, common symbols are not converted to .bss, but to_output_esym()
had no case for them and wrote them out as absolute symbols whose value
was their alignment, so linking the -r output again resolved them to a
bogus fixed address. They are now emitted as SHN_COMMON symbols with the
alignment in st_value, as in the input files.
Co-Authored-By: Claude Fable 5 <[email protected]>
The addend of R_RISCV_ALIGN is the number of NOP bytes the assembler
emitted, which is the requested alignment minus the minimum instruction
size, so the alignment is the smallest power of two greater than the
addend rather than the addend rounded up to a power of two. `.balign 4`
in RVC code (addend 2) and `.balign 8` in non-RVC code (addend 4) were
therefore treated as 2- and 4-byte alignments; in a static link that
left glibc's __sigsetjmp at a 2 mod 4 address.
Co-Authored-By: Claude Fable 5 <[email protected]>
An unresolved weak function resolves to address 0. In a huge executable,
a call site can be more than 2 GiB away from it, so a call relocation
such as R_X86_64_PLT32 cannot encode the displacement and mold reported
an out-of-range error. Such a call is never executed because the program
has to null-check the function first, so we now skip the range check for
call relocations against such symbols and write the truncated value.
This covers x86-64, RISC-V, s390x, SPARC64 and LoongArch; ARM already
rewrites such calls to NOPs, and PowerPC calls go through thunks. Data
references keep their range checks because their results feed the null
check.
This fixes linking Godot's sanitizer build, whose .text ends 70 MB past
the 2 GiB mark.
Fixes https://github.com/rui314/mold/issues/1640
Co-Authored-By: Claude Fable 5 <[email protected]>
Without .sdata, we defined RISC-V's __global_pointer$ as absolute
address 0. Since crt1.o initializes gp with a PC-relative
`lla gp, __global_pointer$`, such an executable could not be linked
more than 2 GiB away from address 0, for example with -Ttext-segment.
We now define the symbol at the beginning of the image, like GNU ld
which defines it relative to .data or .bss in that case. The exact
value doesn't matter because we don't relax gp-relative accesses.
Co-Authored-By: Claude Fable 5 <[email protected]>
reattach_section_pieces() traversed every relocation of every alloc
section once just to count the references to mergeable sections, so
that it could allocate their symbols at once, and then traversed them
again to create the symbols. The count needs only ELF data, so it is
now taken while the sections are parsed, where a CREL relocation table
has just been decoded and is still in cache.
Co-Authored-By: Claude Fable 5 <[email protected]>
The tokenizer tested each byte against six whitespace characters and
three quoting characters in turn. A table makes the scan a single
load and compare per byte, and halves the time to tokenize Chromium's
1.9 MiB response file.
Co-Authored-By: Claude Fable 5 <[email protected]>
The loop looked up every undefined symbol in the --wrap set even when
no --wrap option was given, hashing 5.1 million long C++ names on a
Chromium link, and fetched the thread's ShardedMap bin through TBB's
thread-local lookup once per symbol rather than once per file.
Co-Authored-By: Claude Fable 5 <[email protected]>
create_output_sections() copied each output section's members from its
per-file vectors into the final array on one thread and then freed the
vectors one by one, which took about 10 ms on a Chromium link. Copy
and free per file in parallel instead, computing the alignment in the
same pass.
Co-Authored-By: Claude Fable 5 <[email protected]>
The pass took each symbol's lock once per undefined reference, 5.1
million times on a Chromium link, although nearly every reference is
to a defined symbol, which the pass leaves alone. Popular symbols are
referenced from thousands of files, so their locks bounced between
cores. Find the references to symbols without a definition first,
which needs no lock because the pass does not change defined symbols,
and take the locks only for those.
Chromium release on a Threadripper 7980X at 32 threads: the pass goes
from 21 ms to 2 ms. Output files are bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
When a symbol with hidden visibility resolved to a DSO, symbol
resolution started over from scratch for all symbols with that DSO
definition excluded, and Chromium has three such symbols. Resolution is
independent per symbol, so only those symbols are resolved again, and
if one of them moves to an archive member that is not reachable yet,
liveness is propagated from that member alone. The final resolution
pass after section parsing recomputes everything else as before.
Chromium release on a Threadripper 7980X at 32 threads: resolve_symbols
goes from 143 ms to 115 ms. Output files are bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
Sorting the 44,683 files of a Chromium link into command line order
took 7 ms on one thread at the end of the input reading phase.
Co-Authored-By: Claude Fable 5 <[email protected]>
scan_abs_relocations() gathered each member's absolute relocations into
a vector of its own and then flattened the vectors on one thread. For
.data.rel.ro of a Chromium link that was 147,000 vectors holding 1.1
million entries, and the flattening and freeing took about 10 ms of
serial time. Count the entries per member first and write them to
their final positions in parallel instead.
Co-Authored-By: Claude Fable 5 <[email protected]>
The job that read a thin archive opened all of its members before
feeding them to the reader loop, so a large archive's members were
opened one after another on one thread. Chromium's libblink_core.a
has 2,424. The archive job now only lists the member paths, and each
member's job opens its file, in parallel with the others.
Co-Authored-By: Claude Fable 5 <[email protected]>
Each MergeableSection added itself to its MergedSection's member list
under a mutex from the parallel conversion loop. An output has only a
few merged sections, so on a Chromium link, 32 threads took the same
mutex 150,000 times and spent most of the pass in futex calls. Collect
the members from the files after the loop instead.
Chromium release on a Threadripper 7980X at 32 threads:
create_merged_sections goes from 55 ms to 34 ms and its system time
from 0.74 s to 0.07 s. Output files are bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
mmap(2) takes the process's address space lock, so with tens of
thousands of input files, the calls serialize at a few microseconds
each no matter how many threads make them. Linking Chromium maps 44,683
files, and that serialization alone bounds the input reading phase at
about 220 ms, more than a third of the link. Files up to 32 KiB, which
are 64% of Chromium's inputs by count but 16% by size, are now read
into malloc'ed buffers instead; read(2) takes no such lock, and larger
files are still mmap'ed so that their pages are shared with the page
cache rather than copied.
Chromium release on a Threadripper 7980X at 32 threads: the input
reading phase goes from 228 ms to 108 ms and the whole link from 0.69 s
to 0.56 s, with peak RSS unchanged. Output files are bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
The tokenizer pushed every byte through an ostringstream and copied
each token twice. Tokens without quotes or backslashes, which is nearly
all of them, are now substrings of the mapped file, and whitespace is
tested inline instead of through isspace().
Expanding Chromium's 1.9 MiB response file of 19,000 arguments took
33 ms of single-threaded startup on a Threadripper 7980X; it now takes
5 ms. Output files are bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
scan_abs_relocations() classified an output section's absolute
relocations and copy_buf() applied them in serial loops, although
write_dynrels() already processes them in parallel shards. A PIE
Chromium has 1.1 million of them in .data.rel.ro alone, so each loop
ran for about 30 ms on one thread while the others waited.
Chromium release on a Threadripper 7980X at 32 threads: scan_relocations
goes from 84 ms to 50 ms, copy from 89 ms to 75 ms, and the whole link
from 0.90 s to 0.86 s. Output files are bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
Most relocations refer to a section that another thread has already
visited, yet visit() always did an atomic fetch_or, so popular sections
bounced between cores on every reference. Check the bit with a plain
load first, as Atomic::test_and_set() does; it is never cleared during
marking, so a stale positive answer is still correct.
Chromium release on a Threadripper 7980X at 32 threads: the mark phase
goes from 65 ms to 25 ms, the whole link from 0.96 s to 0.92 s. Output
files are bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
GDB 10 aborts on GCC's DWARF 5 type units even when the executable
contains no .gdb_index section, which broke CI on Debian 11. Probe GDB
with an index-less executable first and skip the test if it cannot
print a type from it.
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01CKdZWtcaghcU7m5zoZtMfN
Emit version 9 indexes when type units are present so GDB 12 falls
back instead of interpreting DWARF 5 type-unit offsets as
.debug_types. Keep version 7 for indexes without type units.
Allow the regression test to run with older readelf versions and skip
it under TSAN, where relocatable links hit known oneTBB races.
With -gdwarf-5 -fdebug-types-section, GCC moves each type into its own
type unit and emits it as a one-section COMDAT .debug_info contribution,
so an object file contains several .debug_info sections. We assumed one
.debug_info section per file containing nothing but compilation units:
type units were missing from the index, and a pubnames offset could be
interpreted relative to the wrong contribution.
Read every live .debug_info contribution instead, classifying units by
their header's unit_type. Type units are collected into the .gdb_index
type-unit table along with their signatures, and each pubnames set is
matched to its contribution through the relocation on the set header's
debug_info_offset field. COMDAT group selection has already discarded
duplicate type units by the time we read them, so exactly one copy of
each type unit is indexed.
Fixes https://github.com/rui314/mold/issues/1634
Co-Authored-By: Claude Fable 5 <[email protected]>
If an IR file claims a COMDAT group, the LTO result provides the group's
contents. However, post-LTO COMDAT group selection started from a clean
slate, so an archive member extracted only after LTO could win a claimed
group, resurrecting a copy that collides with the LTO result.
Remember the claim on the signature symbol, and let only LTO-generated
files win such a group after LTO. LLVM keeps claimed groups in its
output; GCC emits their contents without a group, so no file wins.
Fixes https://github.com/rui314/mold/issues/1637
Co-Authored-By: Claude Fable 5 <[email protected]>
shrink_section() fills r_deltas once, and the later passes only read it,
so it does not need to be a growable vector afterwards. Build the deltas
in a local vector and copy them into the arena. This shrinks InputSection
by eight bytes on RISC-V and LoongArch and makes the class trivially
destructible on every target, so InputSection objects can live in the
arena without an owner that runs their destructors.
Co-Authored-By: Claude Fable 5 <[email protected]>
Several InputSection booleans were stored separately even though one
atomic byte is sufficient. Pack liveness, GC visitation, address
significance, decompression, and ICF-removal state into flags, and add
relaxed fetch_or/fetch_and operations for updating them.
Also remove fde_end by marking the final FDE in each section range in
existing FdeRecord padding. This reduces InputSection<X86_64> from 56 to
48 bytes without increasing FdeRecord or Symbol.
The output offset, dense ICF index, and arena-relative ICF leader are
needed at disjoint points in the link. Keeping separate members wastes
space and requires a separate eligibility flag.
Store them in a union, using icf_idx == -1 to represent ineligibility.
Process the gathered ICF sections directly and retain icf_leader only for
eliminated sections after ICF, allowing leaders to restore their offsets.
Counting distinct ICF classes costs about as much as a digest
propagation round, but it was done after every round. Since refinement
only splits classes, convergence does not need to be checked that often.
Run three propagation rounds between class counts and move digest-buffer
swapping into propagate(). This reduces counting overhead while preserving
the final partition.
When an ODR violation leaves a relocation referring to a discarded COMDAT
section, ObjectFile::scan_relocations() reports an error, but
scan_relocations() continues with the absolute-relocation pass. That pass
assumes symbols are resolved and may dereference a null Symbol::file,
causing mold to crash instead of exiting with the diagnostic.
Add a checkpoint between the two relocation passes so mold exits before
inconsistent symbol state reaches the absolute-relocation scan.
Fixes https://github.com/rui314/mold/issues/1635
create_comdat_group_sections assumed that a relocation section in a
comdat group is always SHT_RELA or SHT_REL. If an input file uses the
CREL format, the relocation sections are SHT_CREL instead, so we took
the branch for ordinary member sections and dereferenced a null
pointer, as mold does not create an InputSection for a relocation
section. mold -r had been crashing on such files since CREL input
support was added in mold 2.38.0.
Recognize SHT_CREL the same way as SHT_RELA/SHT_REL so that the
member's relocation section is propagated to the output group.
Co-Authored-By: Claude Fable 5 <[email protected]>
Commit 35f4a334 changed Symbol::is_local() to check visibility and
version-script localization, but get_output_sym_idx() also relies on it
to decide whether a symbol's slot offset is relative to the local or the
global partition of .symtab. An input-local symbol has default
visibility and no version, so is_local() returned false for it, and a
relocation emitted by --emit-relocs against a named local symbol
referred to an arbitrary global symbol.
This went unnoticed on most targets because GAS usually redirects
relocations against local symbols to section symbols, which take a
different code path. On LoongArch and RISC-V, the assembler keeps them
as-is because linker relaxation may change section-relative offsets, so
relocations against symbols like .LC0 were visibly wrong.
Fixes https://github.com/rui314/mold/issues/1631
Compilers may emit a meaningless FDE covering a zero-length address
range, e.g. for a function fragment that hot/cold splitting has
entirely optimized away. Such an FDE begins at the same address as the
FDE of the next real function, giving the binary search table in
.eh_frame_hdr duplicate keys. If a search lands on the empty FDE
instead of the real one, the unwinder fails to find an existing
exception handler and aborts the process.
We now write a tombstone value to .eh_frame_hdr for such an FDE. The
tombstone compares greater than any real address, so a binary search
never returns it. The FDE itself is harmless because an empty address
range doesn't match any program counter.
AArch64 range extension thunks are small and unlikely to benefit
measurably from aligning every entry to a 16-byte boundary. Padding
every entry with brk instructions wastes meaningful space when an
output contains many thunks.
Keep each thunk group 8-byte aligned, but pack its entries tightly so
the ARM64 short and long forms occupy 12 and 24 bytes instead of 16
and 32 bytes.
A weak TLS variable is usually guarded by a separate weak non-TLS
symbol indicating whether the TLS definition is available. A typical
use looks like this:
extern char foo_used __attribute__((weak));
extern __thread int foo __attribute__((weak));
if (&foo_used)
foo = 1;
If foo remains undefined, its access is unreachable and there is no
meaningful value to apply to its TLS relocations. Leave those
relocations unapplied.
This prevents AArch64 from rejecting an otherwise unreachable TLS
access when an R_AARCH64_TLSLE_ADD_TPREL_HI12 relocation is out of range.
https://github.com/rui314/mold/issues/1626
SH4 raises a slot-illegal exception when a PC-relative load is placed
in a branch delay slot. Our PLT entries did exactly that:
jmp @r0
mov.l 2f, r1
QEMU now checks for such illegal delay-slot loads as of
https://github.com/qemu/qemu/commit/b754cb2dcde26a7bc8a9d17bb6900a0ac0dd38e2,
matching real hardware and the SH-4 ISA. Older QEMU versions accepted
this sequence because they did not enforce the restriction, so upgrading
QEMU caused dynamically linked mold output to trap when entering a PLT
slot.
Move the relocation-index load before the jump and put a nop in the
delay slot. Grow each PLT entry from 16 to 20 bytes to make room while
keeping the two literal values 4-byte aligned.
`exit` in an awk script that follows readelf in a pipeline closes the
pipe as soon as the first match is found. If readelf has not finished
writing its output by then, it is killed by SIGPIPE, which pipefail
turns into a test failure. The race is timing-dependent and was
observed roughly once per thousand runs on a loaded machine.
Co-Authored-By: Claude Fable 5 <[email protected]>
This rewrites lib/gentoo-test.sh to make a full-distro run practical
and its results easier to trust:
- Build each package with mold first, and run the GNU ld control build
only when the mold build fails. Results are classified into success/,
failure/ (builds with GNU ld but not with mold, i.e. a mold bug) and
broken/ (doesn't build with either linker, so not our fault).
- Run test suites only for the package under test by scoping
FEATURES=test through package.env. Applying it globally drags in the
test dependencies of the entire dependency graph and makes a large
part of the tree unresolvable.
- Let emerge apply the configuration changes it would otherwise just
suggest (--autounmask-continue). An ephemeral container has no
hand-maintained package.use, so without this, thousands of packages
are unbuildable out of the box.
- Bring the container image to a consistent ~amd64 state with a world
update at image build time. A stable stage3 base with testing
packages on top otherwise accumulates unsolvable dependency
conflicts, such as perl disagreeing with its installed virtuals.
- Share work between containers through host bind mounts: source
tarballs are downloaded once into a shared DISTDIR, and binary
package pools let each dependency be built once instead of once per
container that needs it. The mold and GNU ld runs use separate pools
so that a control build can never install mold-linked binaries.
- Serialize the container image build with flock. Parallel invocations
used to race to build the same image, one per job slot.
- Work around a deadlock in portage's binary package extraction with
lib/gentoo-gpkg-workaround.py: tar_stream_reader.close() waits for
the external decompressor to exit without draining its stdout, which
hangs when the undrained residue exceeds the pipe buffer.
- Refuse to run if dist/mold doesn't embed a git commit hash, since
test results are keyed by it.
Co-Authored-By: Claude Fable 5 <[email protected]>
MergeableSection::resolve_contents inserts every section fragment into
the parent's ConcurrentMap. The table is typically much larger than
the cache, so each insertion stalls on a cache miss for its first
probe. Since split_contents has already computed all fragment hashes,
prefetch the bucket eight insertions ahead.
On firefox-debug at 32 threads, this cuts the CPU time of
compute_section_sizes, which performs the insertions, by about 15%
(2.7 to 2.3 CPU-seconds) and the pass's wall time from 0.11 s to
0.10 s; the end-to-end change is within noise. All 5809 tests pass.
Co-Authored-By: Claude Fable 5 <[email protected]>
get_sorted_entries splits the bucket array into NUM_SHARDS pieces, and
MergedSection::compute_section_size assigns fragment offsets with one
task per shard, so the shard count caps that pass's parallelism at 16
threads even when more are available. Raise it to 64.
Probing is confined to a shard, and a probe that visits MAX_RETRY
occupied slots aborts, so MIN_NBUCKETS grows with the shard count to
keep at least MAX_RETRY buckets per shard in the smallest maps.
Since fragment offsets are assigned shard by shard, the layout of
merged sections changes slightly (inter-shard alignment padding).
Outputs remain bit-reproducible for a given linker version, which is
the guarantee mold makes.
On firefox-debug at 32 threads, compute_section_sizes goes from
0.13 s to 0.11 s; the end-to-end change is within noise. All 5809
tests pass.
Co-Authored-By: Claude Fable 5 <[email protected]>
RelDynSection::update_shdr runs several times per link because the
size of .relr.dyn feeds back into section layout: set_osec_offsets
calls it until the size converges, and compute_section_headers and
the final pre-copy refresh add more calls. Each call walked all
chunks serially and ended with a serial sort of every RELR offset.
On a Firefox debug link (~360k GOT slots, ~1M relative relocations),
this accounted for most of a ~0.2 s single-threaded stretch between
layout and output copy that is plainly visible in a CPU utilization
trace.
Three changes:
- GotSection::get_num_dynrels no longer materializes the full GOT
entry array just to count relocations. Computing each entry's
value involves a symbol address lookup, which dominated the cost.
The counting logic mirrors get_got_entries' relocation choices.
This is most of the win: the GOT was the longest pole on every
call even with the collection parallelized.
- Collect per-chunk dynamic relocation counts and RELR offsets in
parallel; chunks are independent of one another.
- Process chunks in address order when concatenating their RELR
offsets. Each chunk emits its offsets in ascending order and
chunks occupy disjoint address ranges, so the concatenation is
sorted by construction and the global sort is gone.
The output is byte-identical on firefox-debug against the unpatched
linker at the same commit. All 5809 tests pass. An interleaved A/B
on firefox-debug shows the 32-thread link going from 1.18 s to
about 1.05 s (medians of five).
Co-Authored-By: Claude Fable 5 <[email protected]>
RELR needs only the addresses to which relative relocations apply. Even after
the parent commit, mold creates complete REL/RELA records, partitions them in
RelDynSection, and then discards every field except r_offset.
Have each output chunk return a compact vector of its RELR offsets. Aggregate
and sort those vectors in RelDynSection, then encode them directly. Each chunk
also records how many of its dynamic relocations moved to RELR, so it can write
the remaining records straight to its final output slice.
Android+RELR keeps a temporary array for the remaining records because the
size of their APS2 encoding affects layout.
Seven interleaved 32-thread runs with --no-fork gave these median wall times:
Parent This commit
Firefox release 0.472518 s 0.396796 s (-16.03%)
Chromium debug 2.081396 s 2.044275 s ( -1.78%)
Compared with 64b92151, the accumulated improvements are 17.79% for Firefox
release and 1.68% for Chromium debug.
An earlier seven-run A/B test found no consistent cost from returning each
vector instead of appending through an output parameter:
Firefox release: append 0.396 s, return 0.390 s
Chromium debug: append 2.005 s, return 2.028 s
The differences are less than 2% and point in opposite directions.
All 471 native x86-64 tests pass, with nine expected skips. A release build for
all supported targets also passes.
RelDynSection currently asks each producer for a vector and appends it to a
second vector. OutputSection also builds one vector per shard and flattens
them. Large links therefore allocate, copy, and destroy relocation records
multiple times.
Have each producer report its exact relocation count on Chunk. RelDynSection
allocates one destination array and gives each producer a pointer to its exact
slice. OutputSection records per-shard prefix counts so shards can fill
disjoint slices in parallel.
Recompute counts on every layout pass because synthetic GOT relocation counts
can change before layout converges.
Seven interleaved 32-thread runs against 64b92151, with --no-fork, gave these
median wall times:
Firefox release: 0.482679 s -> 0.472518 s (-2.11%)
Chromium debug: 2.079169 s -> 2.081396 s (+0.11%)
All 471 native x86-64 tests pass, with nine expected skips. Release builds for
x86-64 and PPC64V1 also pass.
After fix_synthetic_symbols, we re-ran set_osec_offsets
unconditionally. That pass is needed only under
--pack-dyn-relocs=android, whose packed relocation format encodes
addends in variable-length SLEB128, so fixing synthetic-symbol
values can change the encoded size of .rela.dyn. No other mode
emits anything whose size depends on addends; in particular,
RELR's size depends only on relocation locations, which are fixed
by this point. Make the pass conditional on the Android mode.
On the Firefox release benchmark with 32 threads and --no-fork,
this shortens the link from 0.44 s to 0.41 s. Outputs are
byte-for-byte identical before and after for both a RELR workload
(Firefox) and a RELA workload (Chromium release).
Co-Authored-By: Claude Fable 5 <[email protected]>
The scalar glob matcher is faster for ordinary version scripts, but its
work grows linearly with the number of patterns. Compile sets of 64 or more
complex patterns into one bit-parallel NFA. Each state occupies one bit, so
a machine word advances 64 states at once. Keep the existing scalar and
Aho-Corasick paths unchanged for smaller and simpler pattern sets.
On 200,000 synthetic 60-byte symbol names, the standalone matcher changes
as follows:
128 patterns: 1.00 s -> 0.197 s (5.1x faster)
256 patterns: 1.98 s -> 0.380 s (5.2x faster)
A normal TensorFlow debug link remains on the scalar path. A sanity check
shows apply_version_script changing from 0.177 s to 0.175 s.
The general glob matcher represents all patterns as one NFA and updates
several bit vectors for every byte of every symbol. Its cost therefore
depends on the total pattern length even when a version script contains only
a few complex patterns.
Compile each complex pattern into literal, star, question, and bracket
tokens. Match without recursion, remember the most recent star, and use
string_view::find for literal runs following a star. This replaces MultiGlob
and removes the Bitvector class.
Also store non-root Aho-Corasick edges as sibling lists. Most trie nodes have
one child, so this reduces a node from 1,040 to 24 bytes on a 64-bit host. A
dense root table keeps the common transition fast. Construct failure links
breadth-first, as required by the algorithm.
With symbol-table traversal held constant, the median of five 32-thread
TensorFlow debug links changes as follows:
apply_version_script wall time: 0.503 s -> 0.179 s (-64%)
apply_version_script CPU time: 14.137 s -> 5.467 s (-61%)
total wall time: 3.626 s -> 3.285 s (-9%)
Version script wildcard matching currently visits each object file and scans
its global symbol slots. A symbol is processed only when that file provided
the prevailing definition, so many entries are skipped. More importantly, an
entire file is one task, which leaves workers idle when large object files
reach the end of the pass.
Iterate over the symbol table instead. Its 64 shards visit every symbol once
and distribute the work more evenly. Rename ShardedMap::for_each to
parallel_for_each to make the scheduling behavior explicit.
With the same glob matcher and 32 threads, the median of five TensorFlow
debug links changes as follows:
apply_version_script wall time: 0.699 s -> 0.503 s (-28%)
apply_version_script CPU time: 13.456 s -> 14.137 s (+5%)
total wall time: 3.800 s -> 3.626 s (-5%)
A defined dynamic symbol at VER_NDX_GLOBAL with the VERSYM_HIDDEN bit set
is a legacy compat alias. GNU as emits one for `.symver foo, foo@VER`
without the `remove` modifier, so that binaries linked before the symbol
was versioned keep resolving. It must not satisfy a new unversioned
reference to `foo`; that has to bind to the default version `foo@@VER`.
SharedFile::parse cleared VERSYM_HIDDEN before classifying the symbol, so
such an alias took the "unversioned symbol" path and claimed the plain
name. With both it and the `foo@@VER` definition claiming that name,
resolve_symbols keeps whichever comes first in .dynsym, which is the
alias. No verneed entry is then emitted, and glibc refuses to hand out a
non-base version for an unversioned lookup, so the alias wins at runtime
too and the caller silently gets the old implementation.
lksctp-tools' libsctp.so.1 exports sctp_connectx this way. Linking with
mold selects the pre-SCTP_SOCKOPT_CONNECTX3 ABI, which reports success
but never fills in the caller's association ID.
Skip such symbols while parsing. They cannot be referred to by any name,
as there is no version string to form `foo@VER` from. ld.bfd reports an
undefined reference when one is the only definition, which is what
skipping produces.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Each provisional thunk owns independent symbol and offset vectors. Filter
unneeded entries and recompute their offsets concurrently instead of
shrinking the thunks one at a time.
On the Chromium Arm64 debug link, seven interleaved A/B runs reduce the
remove_redundant_thunks median from 0.080 to 0.073 seconds and median
wall time from 1.924 to 1.897 seconds. The output is byte-identical.
Symbol order does not affect thunk placement, so defer deterministic sorting
until all thunks for an output section have been constructed. Sort independent
thunk vectors concurrently instead of sorting each one in the serial batch
loop.
On the Chromium Arm64 debug link, nine interleaved A/B runs reduce median wall
time from 1.98 to 1.97 seconds. The compute_section_sizes median falls from
0.116 to 0.095 seconds. The output is byte-identical.
Initialize input-section offsets in parallel. Section offsets and end offsets
are monotonic, so use lower_bound to find A and C instead of walking over
millions of sections a second and third time.
On the Chromium Arm64 debug link, five interleaved A/B runs reduce median wall
time from 2.01 to 1.95 seconds. create_range_extension_thunks falls from about
0.17 to 0.11 seconds. The output is byte-identical.
Collect newly discovered thunk targets in per-worker vectors and concatenate
them after each batch. The existing atomic flag still removes duplicates, and
the final sort preserves deterministic output without serializing every vector
insertion through one mutex.
On the Chromium Arm64 debug link, five interleaved A/B runs reduce median wall
time from 2.19 to 2.03 seconds. The compute_section_sizes phase falls from
0.341 to 0.181 seconds, and its system CPU time falls from 5.146 to 0.620
seconds. The output is byte-identical.
Allocate MergedSection objects in the linker arena and store each fragment's
parent section as a 32-bit arena index. This shrinks SectionFragment from 24
to 16 bytes and its ConcurrentMap bucket from 64 to 32 bytes.
Chromium's x86-64 debug link contains about 2.17 million merge-map buckets.
Across five interleaved 32-thread runs, median wall time changes from 1.91 to
1.90 seconds and median peak RSS falls from 24,870,408 to 24,808,228 KiB, a
reduction of 62,180 KiB.
Build the symbol table and constant pool in a low-priority task after
debug-info offsets are fixed. The final output path then reads relocated
address ranges and copies the prepared tables.
The stages have different dependencies and useful parallelism. Reading
input name data needs only input sections and can use all available
workers. Building the table needs final .debug_info offsets but stops
scaling near 12 workers, so run it in a separate bounded arena. Reuse one
task group for both stages.
Keep the standard serial open-addressed hash construction. This preserves
deterministic output without the prototype atomic claim table and repeated
parallel probe rounds.
Against an unrestricted table arena on the same tree, the bounded arena
reduced the 10-run median from 1.935 s to 1.905 s and user CPU from
43.510 s to 42.865 s.
Chromium debug link, 32 threads, 10 interleaved run medians:
wall: 1.975 s -> 1.900 s (-3.8%)
user: 43.125 s -> 42.600 s (-1.2%)
system: 9.740 s -> 9.770 s (+0.3%)
peak RSS: 24,817,662 -> 24,889,408 KiB (+70 MiB)
Use the ELF section header for the stored input size and keep only the
data pointer in InputSection. Decompressed sections use sh_size because
their replacement buffer has the uncompressed size.
This reduces InputSection<X86_64> from 88 to 80 bytes. Relaxed RISC-V
and LoongArch sections continue to use the original section-header size
when copying their input bytes.
Chromium debug link, 32 threads, 10 interleaved run medians:
wall: 1.960 s -> 1.950 s (-0.5%)
user: 42.565 s -> 42.560 s
system: 9.775 s -> 9.615 s (-1.6%)
peak RSS: 24,834,922 -> 24,816,480 KiB (-18 MiB)
Parse each reachable object and record its COMDAT candidates in one
parallel traversal. Reuse the signature name while parsing instead of
recovering it from the selected Symbol for the GCC debug-macro check.
Chromium debug link, 32 threads, 10 interleaved run medians:
wall: 1.965 s -> 1.960 s (-0.3%)
user: 43.160 s -> 42.615 s (-1.3%)
system: 9.820 s -> 9.845 s (+0.3%)
Reserve a sparse address-space arena for symbols and input files. Objects in
that arena can refer to each other with four-byte self-relative pointers,
which halves the size of symbol-reference arrays and reduces Symbol from 48
to 40 bytes. On 32-bit hosts, reserve 256 MiB instead of 8 GiB; this places a
practical limit on linker metadata without affecting 32-bit targets linked on
a 64-bit host.
Keep the standard per-shard hash tables, but allocate their Symbol values in
stable arena blocks. Store global names beside those values, derive local
names from their owner files, and represent each COMDAT group in eight bytes.
Context still destroys file objects normally so their separately allocated
data is released before the arena is unmapped.
On the Chromium x86-64 debug benchmark with 32 threads, median wall time over
eight runs falls from 1.990 to 1.970 seconds. Median user time changes from
42.880 to 43.015 seconds, while system time falls from 9.965 to 9.890 seconds.
Median peak RSS falls from 25,625,100 to 24,841,684 KiB, a reduction of
783,416 KiB.
The generated output was byte-identical.
Chromium has 41.96 million ELF section-table slots but only 5.86 million
regular input sections. Store a 32-bit value per slot instead of an
InputSection pointer. Zero denotes an absent section, the high bit denotes a
mergeable section, and the remaining bits index stable per-file pools.
InputSectionTable owns both pools and presents the same indexed and range
interfaces used by linker passes. Moving a section to the mergeable pool is
now explicit instead of relying on a constructor that clears a pointer passed
by reference.
Chromium debug link, 32 threads, eight interleaved run medians:
wall: 2.005 s -> 2.000 s (-0.2%)
user: 43.595 s -> 43.375 s (-0.5%)
system: 10.160 s -> 9.865 s (-2.9%)
peak RSS: 26,145,360 -> 25,598,862 KiB (-534 MiB)
All section contents matched apart from .comment and the fast build-ID note.
GDB index type vectors contain fixed-width 32-bit records. Sort lists with at
least 256 records using four byte-wise radix passes, while retaining comparison
sorting for smaller lists. Reuse worker-local scratch storage so the linear
path does not allocate for every name.
Chromium debug link, 32 threads, eight interleaved run medians:
wall: 1.965 s -> 1.955 s (-0.5%)
user: 44.720 s -> 43.890 s (-1.9%)
system: 9.940 s -> 10.185 s (+2.5%)
The complete 4.86 GiB output was byte-identical.
Chromium debug has 18.39 million local symbols, and 9.71 million belong to
discarded COMDAT sections. After COMDAT elimination, allocate Symbol objects
only for retained local symbols and map every discarded local symbol to one
global zero-valued placeholder. The ELF gABI requires local symbols in a
discarded COMDAT member to be discarded and forbids references to them from
outside the group, so the placeholder does not preserve per-symbol metadata.
Chromium debug link, 32 threads, six interleaved run medians:
wall: 2.095 s -> 2.060 s (-1.7%)
user: 48.095 s -> 46.735 s (-2.8%)
system: 10.110 s -> 10.280 s (+1.7%)
peak RSS: 26,946,944 -> 26,241,790 KiB (-0.67 GiB)
Chromium sections matched apart from version and fast build-ID metadata.
Chromium debug contains 93.8 million relocations in nonallocated sections.
Its 230 MiB CREL input expands to about 2.1 GiB of ElfRel records. Decode
and apply those relocations in one pass instead of retaining the expanded
arrays. Keep exact arrays for allocated sections, whose relocations are
consumed later.
for_each_reloc calls a target-specific callback for every relocation. GCC
15 does not inline these large callbacks on its own, leaving a function call
for each ordinary REL or RELA relocation. Mark the callbacks ALWAYS_INLINE;
this has no effect on CREL's representation but avoids that regression.
Chromium debug link, 32 threads, six-run medians:
wall: 2.215 s -> 2.115 s (-4.5%)
user: 50.145 s -> 48.415 s (-3.5%)
system: 11.065 s -> 10.215 s (-7.7%)
peak RSS: 27.96 GiB -> 25.64 GiB (-2.32 GiB)
On an Asahi machine, adding ALWAYS_INLINE to the streaming implementation
changes the native clang-debug-arm64 link at 16 threads as follows:
wall: 2.36 s -> 1.85 s (-21.6%)
user: 35.3 s -> 27.1 s (-23.2%)
Chromium output matched apart from the fast build-ID payload. The two Asahi
outputs were byte-identical.
Compilation-unit headers and GNU pubnames do not depend on relocated
debug contents. Read them from per-file input sections and build the name
map in a low-priority TBB arena while foreground passes continue. Address
ranges and final serialization still wait until debug relocations are applied.
Chromium debug link, 32 threads, six-run medians:
wall: 2.255 s -> 2.175 s (-3.5%)
user: 49.670 s -> 49.485 s (-0.4%)
system: 11.460 s -> 11.260 s (-1.7%)
peak RSS: 27.95 GiB -> 27.98 GiB (+36 MiB)
The generated .gdb_index was byte-identical.
Most regular COMDAT signatures already name a global symbol. Reuse that
Symbol as the group identity instead of interning 14.8 million signatures in
a second map. Local, section, and versioned signatures use the same symbol
table through a name-based fallback.
Symbol resolution has been cleared when COMDAT owners are selected, so
sym_idx is temporarily available. Record the lowest file priority there, mark
each group whose file has that priority as the owner, and restore sym_idx with
one symbol-map traversal before normal resolution resumes. This keeps
ownership on the signature itself and needs no dense group IDs or auxiliary
side arrays.
On the Chromium x86-64 debug benchmark with 32 threads, median wall time over
eight runs falls from 2.310 to 2.250 seconds. Median user time falls from
49.520 to 47.390 seconds, while system time changes from 11.720 to 11.935
seconds. Median peak RSS falls from 29,485,754 to 29,103,198 KiB, a reduction
of 382,556 KiB.
The generated output was byte-identical.
ELF symbols already contain their string-table offsets. Cache only a one-byte
NameLen instead of retaining a 16-byte string_view for each input symbol.
Longer lengths use logarithmic lower bounds, so lookups scan only the uncached
suffix. Chromium has 37.8 million input symbols, making the per-symbol
reduction substantial.
On the Chromium x86-64 debug benchmark with 32 threads, median wall time over
eight runs changes from 2.320 to 2.315 seconds. Median user and system time
fall from 49.800 and 11.625 seconds to 49.645 and 11.580 seconds. Median peak
RSS falls from 30,132,552 to 29,503,344 KiB, a reduction of 629,208 KiB.
The generated output was byte-identical.
Store short name lengths exactly in one byte and use the remaining values as
logarithmic lower bounds. Looking up a long name then scans only its suffix.
Copy a long version-prefix name into the string pool when its original storage
is not NUL-terminated. This reduces Symbol<X86_64> from 56 to 48 bytes.
On the Chromium x86-64 debug benchmark with 32 threads, median wall time over
eight runs falls from 2.355 to 2.340 seconds. Median user time falls from
50.475 to 49.875 seconds, while system time is unchanged at about 11.85
seconds. Median peak RSS falls from 30,330,582 to 30,146,730 KiB, a reduction
of 183,852 KiB.
The generated output was byte-identical.
An input section's object and section header already identify its name in
the section-header string table. Store only the 16-bit length in InputSection
and recover the pointer from that existing metadata. This reduces
InputSection<X86_64> from 104 to 88 bytes.
On the Chromium x86-64 debug benchmark with 32 threads, median wall time over
eight runs changes from 2.365 to 2.360 seconds. Median user and system time
fall from 50.940 and 12.005 seconds to 50.495 and 11.700 seconds. Median peak
RSS falls from 30,470,228 to 30,335,754 KiB, a reduction of 134,474 KiB.
The generated output was byte-identical.
A CREL header gives the relocation count before decoding starts. Add
ExactArray, which owns an uninitialized array of a fixed size and provides
indexed access, and fill it directly. Existing relocation consumers receive a
span over the array.
This avoids vector capacity slack and initialization that the decoder
immediately overwrites.
On the Chromium x86-64 debug benchmark with 32 threads, median wall time over
eight runs falls from 2.400 to 2.370 seconds. Median user and system time fall
from 51.655 and 12.115 seconds to 50.855 and 11.965 seconds. Median peak RSS
falls from 31,006,116 to 30,470,208 KiB, a reduction of 535,908 KiB.
The generated output was byte-identical.
Chromium contributes 35.8 million GDB name records. Keep each temporary
record at 16 bytes, replace its parsed form with its indexed form after
interning, and deduplicate names within each compilation unit with a small
linear hash table instead of sorting them.
Compute each string hash and length in one scan, assign constant-pool ranges
with a parallel prefix sum, and record type-vector slots while interning. Keep
the slot-claiming protocol shared by both ConcurrentMap insertion interfaces.
On the Chromium x86-64 debug benchmark with 32 threads, median wall time
over eight runs falls from 2.550 to 2.390 seconds. Median user and system time
fall from 54.735 and 12.390 seconds to 51.275 and 12.095 seconds. Median peak
RSS falls from 32,086,034 to 30,999,878 KiB, a reduction of 1,086,156 KiB.
The generated output was byte-identical.
The cardinality estimators used shared atomic registers even though their
callers already partition work across threads. Give each worker a local sketch
and merge the registers after the parallel scan. Apply the same scheme to GDB
index names and mergeable-section fragments.
Chromium debug link, 32 threads, five interleaved run medians:
wall: 2.60 s -> 2.57 s (-1.2%)
user: 56.22 s -> 55.95 s (-0.5%)
The generated .gdb_index was byte-identical.
Section construction repeatedly recovered names from .shstrtab even when the
caller had just looked up the same name. Pass known names into constructors and
defer other lookups until a name is actually needed.
Chromium debug link, 32 threads, five interleaved run medians:
wall: 2.61 s -> 2.60 s (-0.4%)
user: 57.38 s -> 56.22 s (-2.0%)
The generated output was byte-identical.
Duplicate COMDAT groups account for millions of sections in large debug
links. Read the symbol table and group membership first, choose the winning
groups after archive extraction, and only then construct input sections and
local symbols for retained members.
Let ShardedMap report whether gather created each symbol so new symbols can
be initialized in the same pass. An empty section table means that a file has
not reached section construction, avoiding a separate parsing-state flag.
Chromium debug link, 32 threads, five interleaved run medians:
wall: 2.77 s -> 2.61 s (-5.8%)
peak RSS: 33.2 GiB -> 30.9 GiB (-2.3 GiB)
The generated output was byte-identical.
Canonical PLTs and copy relocations are alternative ways to give an
imported symbol an address in the output. Which one is needed follows
from the symbol type, so recording separate scan flags duplicates
information.
Use one CANONICAL action and NEEDS_CANONICAL flag, then choose the table
to populate when dynamic symbol entries are assigned.
We choose between the two PPC64 ABIs by endianness because ELFv1 is
normally big-endian and ELFv2 little-endian. However, that
correspondence is a convention rather than a rule. In particular, musl
uses ELFv2 on big-endian too, so linking for example Alpine ppc64
objects silently produced an ELFv1-style executable with a synthesized
.opd whose startup code crashes immediately.
We support only the usual combinations, so make get_elf_type() treat the
others as unrecognizable, just like an ELF file for an unsupported
machine. Also give unrecognizable ELF files a proper error message; we
used to print "incompatible file type: ppc64 is expected but got " with
a trailing empty string.
Fixes https://github.com/rui314/mold/issues/1498
Co-Authored-By: Claude Fable 5 <[email protected]>
We computed the binding of a symbol in the static symbol table from its
dynamic import/export status, so in an ordinary executable, every global
symbol was emitted as STB_LOCAL. That misleads all symbol table
consumers; nm prints "t main" instead of "T main", and gdb had to work
around it for a feature that depends on symbol bindings.
Only symbols hidden by visibility or localized by a version script
should be demoted. GNU ld keeps hidden symbols STB_GLOBAL with hidden
visibility while lld converts them to STB_LOCAL; we follow lld, which
also matches our previous output for such symbols.
One wrinkle: claim_unresolved_symbols() assigned version index 0
(VER_NDX_LOCAL) to imported symbols so that their .gnu.version entries
would be 0, which would make the new is_local() misclassify them as
version-script-localized. Leave ver_idx unassigned for imported symbols
and have the .gnu.version writer emit 0 for unversioned undefined
symbols directly, so that VER_NDX_LOCAL on a symbol means only
"localized by a version script".
Fixes https://github.com/rui314/mold/issues/906
Co-Authored-By: Claude Fable 5 <[email protected]>
A default-versioned symbol `foo@@VER` can be referred to not only as
`foo` but also as `foo@VER`, so defining both `foo@@VER` and `foo@VER`
is a duplicate definition of the versioned name. We didn't diagnose it
and silently emitted a dynamic symbol table containing two definitions
of `foo@VER`, leaving it up to the dynamic loader which one a versioned
reference binds to. GNU ld and lld reject this; now we do too.
We check for duplicates by scanning the dynamic symbol table once it's
populated, because the problem manifests if and only if both symbols
are exported, and the exported set is much smaller than the set of all
symbols. GNU ld additionally reports the error for non-exported
symbols; lld doesn't, and we behave like lld.
Fixes https://github.com/rui314/mold/issues/828
Co-Authored-By: Claude Fable 5 <[email protected]>
We ignored defined dynamic symbols whose .gnu.version entry is 0
(VER_NDX_LOCAL) on the assumption that such symbols are bound locally.
That assumption is wrong; a symbol that's actually local to a DSO
doesn't appear in .dynsym at all, so such an entry is ill-formed. GNU
ld briefly emitted version index 0 for unversioned defined symbols
(https://sourceware.org/bugzilla/show_bug.cgi?id=33577), and linking
against a DSO created by such ld made us report confusing "undefined
symbol" errors for symbols that plainly exist in the DSO.
lld rejects such files as corrupted. Follow suit and report an error
pointing at the offending file and symbol instead of silently ignoring
it.
Fixes https://github.com/rui314/mold/issues/1534
Co-Authored-By: Claude Fable 5 <[email protected]>
We silently accepted a shared object file given on the command line even
in the static mode and simply ignored it, producing an executable whose
references to the shared object's symbols remain unresolved. Such a
binary crashes at runtime. GNU ld and lld reject this at link time, and
so should we.
Fixes https://github.com/rui314/mold/issues/1616
Co-Authored-By: Claude Fable 5 <[email protected]>
We used to read files under .git directly to avoid a build-time dependency
on git, but the on-disk repository format is not stable. The hash lookup
broke for repositories using the new reftable format, and it never worked
in git worktrees, where .git is a file rather than a directory.
Just ask git instead. Building from a git checkout now requires git, which
is not much to ask; building from a source tarball still works and simply
omits the hash, as before.
Fixes https://github.com/rui314/mold/issues/1621
Co-Authored-By: Claude Fable 5 <[email protected]>
After LTO, we reset the reachability of archive members and redo symbol
resolution to re-derive which members are actually needed, because the
LTO-generated code may no longer use some of the members that were
extracted to satisfy references from IR files. However, the results of
the pre-LTO COMDAT group selection survived into the second resolution:
ComdatGroup::owner kept the first winner's priority, and the losers'
member sections remained dead.
Therefore, if a group's winner was an archive member that is not
re-extracted after LTO, no remaining file could provide the group's
sections, and we reported a false "refers to a discarded COMDAT section"
error for a perfectly valid program.
Reset the COMDAT group state in do_lto() so that the post-LTO symbol
resolution selects groups from a clean slate among the files that
actually remain in the link.
Fixes https://github.com/rui314/mold/issues/1565
Co-Authored-By: Claude Fable 5 <[email protected]>
Like the previous commit did for comdat groups, symbol interning no
longer goes through a single concurrent hash table. Object file
parsing only records each global symbol name along with which slot of
the file's symbol vector it belongs to; gather_symbols() then
deduplicates the names into Symbol objects and fills in the files'
symbol pointers, processing each hash shard on a single thread with
no synchronization. Besides avoiding concurrent hash table
insertions, this hashes each name only once and no longer constructs
a temporary Symbol per lookup, which the concurrent map's insert
interface required. Linker-synthesized symbols, shared libraries and
linker scripts keep interning through a locked slow path in
get_symbol().
Linking a Chromium debug build on a Threadripper 7980X saves ~12
CPU-seconds (83 -> 71) and about 0.25 seconds of wall time at 32
threads. Output files are bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
We used to intern comdat group signatures into a single concurrent
hash table as object files are parsed. The concurrent map is costly
under 32-way insertion, and every file redundantly hashed the
signature strings, which are long mangled names for C++ programs.
Signatures are now handled by a new ShardedMap class: parsing only
appends them to thread-local buffers binned by hash, and once all
files have been read, each bin is deduplicated by a single thread
with no synchronization, filling in the files' ComdatGroupRefs. A
signature is hashed once, when it's recorded. Deduplication runs in
resolve_symbols() right before comdat elimination rather than at the
end of read_input_files() so that files parsed later, such as ELF
files produced by the LTO plugin, are covered when we redo symbol
resolution. LTO comdats otherwise take a mutex-protected slow path.
Linking a Chromium debug build on a Threadripper 7980X is about 4%
faster (3.22s -> 3.06s at 32 threads), saving ~4 CPU-seconds. Output
files are bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
We used to read input files one at a time in the command line order,
with a single loop opening and mmap'ing each file and walking archive
files. That loop, not parsing, is the bottleneck of the read phase;
for a Chromium debug build with ~45,000 input files, it takes ~1.3
seconds, about 40% of the total link time.
We now open and read files with a parallel for-each loop with the
feeder pattern; an archive file feeds one job per member. The command
line parser records the reader state (--as-needed etc.) and a
hierarchical position for each input file argument, and files are
sorted back into the command line order afterwards to assign
priorities. Linker scripts, which can mutate the linker context, are
parsed after the parallel loop, one at a time in the command line
order. LTO claims are serialized because the plugin interface has
shared state.
Linking Chromium debug on a Threadripper 7980X is ~2% faster at 32
threads and ~14% faster at 64 threads, where the serial loop
previously made additional threads a net loss. Output files are
bit-identical to before.
Co-Authored-By: Claude Fable 5 <[email protected]>
mremap, which we used to grow the output file mapping for
.gdb_index, is Linux-specific, so commit 9b7c5267 broke the build on
the BSDs and macOS. Instead of growing a mapping that was created
with the exact file size, we now map the output file with twice as
much address space as its size upfront. The mapping beyond the end
of the file may not be touched, but growing the file with ftruncate
makes the tail accessible, so extend() no longer needs any mmap
call, and the output buffer never moves. Reserving address space
does not consume any physical memory; pages materialize only when
they are first touched.
This also fixes the madvise(MADV_HUGEPAGE) call in extend(), which
had always failed with EINVAL because its address was not
page-aligned. That went unnoticed because an mremap'ed mapping
inherits the huge-page flag from the original mapping.
Co-Authored-By: Claude Fable 5 <[email protected]>
A large link creates tens of millions of InputSection objects, one
heap allocation each, and the profiler attributed 5% of all cycles
to their construction. Allocate them from a per-file std::deque
instead and refer to them through plain pointers. Section death is
represented by a null entry as before; the object itself stays in
the pool until the file is destroyed.
This removes the InputSection constructor from the profile and
cuts the allocation count by 4x, but the end-to-end link time is
unchanged within noise on a Chromium debug build; the constructor
cost was evidently not on the critical path. The change still pays
its way by making `sections` a plain pointer vector, which drops
the unique_ptr indirection and the .get() noise throughout.
Co-Authored-By: Claude Fable 5 <[email protected]>
Compilers emit numerous temporary local symbols such as .L.str.42
for unnamed program elements. They are compiler-internal: nothing
references them through the symbol table, and DWARF refers to their
data by section offset. Nevertheless we kept them in the output by
default, following GNU ld. A Chromium debug build contains 2.26
million of them, and they added ~180 MiB of .symtab and .strtab to
the 4.9 GiB output.
Discard them by default, as lld does. The new --discard-none option
restores the previous behavior. -r and --emit-relocs continue to
keep all local symbols, since emitted relocations may refer to
them, and RISC-V and LoongArch no longer need their special case,
as the default now covers them.
Link time is unchanged within noise; the benefit is a ~4% smaller
output for large debug builds and a symbol table that no longer
resolves addresses to compiler-internal labels.
Co-Authored-By: Claude Fable 5 <[email protected]>
Previously, data appended to the output file, such as .gdb_index,
was staged in a heap buffer or a separate memory mapping. We can
instead just grow the existing mapping with mremap(MREMAP_MAYMOVE).
Consumers of the output buffer derive pointers from ctx.buf, so the
mapping is free to move, and since filesize now includes appended
data, the build ID and the .gnu_debuglink CRC naturally cover the
entire file.
Co-Authored-By: Claude Fable 5 <[email protected]>
When a live section refers to a __start_<name> or __stop_<name>
symbol, every section named <name> must be kept alive. The garbage
collector marked those sections one by one in the thread that
found the reference. A single reference can fan out to an enormous
number of sections: ClickHouse is built with XRay instrumentation,
which emits one xray_instr_map section per function, and the XRay
runtime references __start_xray_instr_map, so one thread marked
and visited a couple hundred thousand sections while the rest of
the machine waited for the mark phase to finish.
This was a regression introduced by 384527fd: before it,
C-identifier-named sections were unconditional GC roots and
entered the root set through a parallel loop. That commit
correctly made their retention conditional on a reference to
their marker symbols but expanded the fanout serially. Keep the
precise semantics and spread the fanout over threads with a
nested parallel loop, storing the sections of each name as a
vector instead of entries of a multimap.
This shortens ClickHouse's debug link from 1.98s to 1.45s at the
default 32 threads; its gc pass drops from 0.56s to 0.09s.
Co-Authored-By: Claude Fable 5 <[email protected]>
ConcurrentMap allocated its hash table with MAP_POPULATE, which
zeroes every page of the mapping inside the mmap system call, on
one thread. The string-merge map for a Clang debug build spans
several gigabytes, so every other thread stalled for ~170ms at the
barrier between splitting and inserting strings while one thread
pre-faulted the table.
Map the table lazily and advise MADV_HUGEPAGE instead. Populating
the table on demand from concurrent inserts spreads the fault cost
over all threads, huge pages make each fault populate 2MiB rather
than a base page, and they also cut the TLB miss rate of the
random probes into the table. Explicit parallel pre-faulting was
measured to add nothing on top of this. This shortens a Clang
debug link from 2.13s to 1.85s and a Chromium debug link from
3.38s to 3.27s at the default 32 threads.
Co-Authored-By: Claude Fable 5 <[email protected]>
.gdb_index was constructed in a malloc'ed buffer that close()
appended to the output with a single buffered write, copying the
entire section (534 MiB for a Chromium debug build) through the
page cache on one thread, into disk blocks that ext4 had to
allocate inside the write path.
Add OutputFile::extend, which returns a buffer for data appended
past the mapped image. A memory-mapped output file grows with
ftruncate, preallocates the new disk blocks, and memory-maps the
appended range, so write_gdb_index constructs the section directly
in the output file, in parallel. The range gets its own mapping
because growing the existing one with mremap could move it,
invalidating pointers into it held by the caller. Output files
that cannot be mapped (special files such as stdout, the separate
debug file, and Windows) keep buffering appended data on the heap,
now private to each implementation; the shared buf2 field is gone.
The fallocate calls are now guarded by a configure-time
HAVE_FALLOCATE check rather than #ifdef __linux__.
This shortens a Chromium debug link from 3.31s to 3.18s and makes
close_file effectively free.
Co-Authored-By: Claude Fable 5 <[email protected]>
RelDynSection::update_shdr collected dynamic relocations by walking
all chunks serially, and a single output section such as
.data.rel.ro can account for most of an output's dynamic
relocations (~1.5 million for a Chromium debug build, mostly
R_RELATIVE for vtables). The walk computes each relocation's offset
and addend with Symbol::get_addr, which made it a ~0.1s
single-threaded stretch between parallel passes.
Process an output section's absolute relocations in fixed-size
parallel shards and flatten the per-shard results, so the resulting
section contents are identical to the serial version. This shortens
a Chromium debug link from 3.58s to 3.49s at the default 32
threads.
Co-Authored-By: Claude Fable 5 <[email protected]>
parse_nonpositional_args tested each command line argument against
every option name, and each test allocated a few heap strings in
add_dashes. A command line for a large program consists almost
entirely of input file paths (a Chromium debug link passes ~19,000
paths but fewer than 100 options), so we were spending several
hundred milliseconds of serial time on failed option matches before
opening the first input file.
An argument that does not start with '-' cannot match any option,
so such arguments now go straight to the output list. This shortens
a Chromium debug link from 3.98s to 3.59s on a 64-core machine
because option processing delayed the start of the parallel parsing
phase.
Co-Authored-By: Claude Fable 5 <[email protected]>
SH-4 object files store relocation addends in the relocated places
rather than in r_addend, and mold assumed that dynamic relocations
work the same way, emitting them with zero r_addend and the addend
written to the relocated place. The glibc dynamic linker, however,
follows the usual RELA convention: it reads the addend of a symbolic
dynamic relocation only from r_addend, and it reads the relocated
place only for R_SH_RELATIVE relocations whose r_addend is zero. Any
symbolic dynamic relocation with a non-zero addend thus lost its
addend at load time.
A typical victim is the vtable pointer of a typeinfo object in a
position-independent executable. It must point into libstdc++'s
vtable (e.g. _ZTVN10__cxxabiv117__class_type_infoE + 8), so its
dynamic relocation carries a non-zero addend. With the addend
dropped, virtual calls on such a typeinfo dispatch to wrong vtable
slots, which crashes the C++ exception machinery as soon as a
personality routine tries to match a catch clause. The icf-lsda test
caught this on sh4aeb because its CI toolchain creates PIEs by
default; the little-endian sh4 toolchain defaults to
position-dependent executables, in which the vtable is copy-relocated
and no dynamic addend is involved, so the same test passed there.
Remove the ElfRel specializations that zeroed addends for symbolic
relocations; the generic RELA record types have the identical layout
and keep addends. Relocation records emitted for -r or --emit-relocs
keep following the object-file convention, since their consumers
expect addends in the relocated places.
Co-Authored-By: Claude Fable 5 <[email protected]>
Creating a .gdb_index was slower than it should have been for three
reasons. read_pubnames_cu looked up compunits with a linear scan,
which made it quadratic in the number of compunits; the compunit
vector is sorted by offset, so a binary search suffices. gdb_hash
was computed for every name occurrence even though the same name
typically appears in many compunits; we now compute it only when a
name is inserted into the map for the first time. Finally, nametypes
were sorted with a stable sort, but their ordering is total, so a
plain sort produces the same result without the extra buffer.
On a Chromium debug build, this makes the write_gdb_index pass 2x
faster with --threads=1 (10.2s to 5.2s) and 28% faster at the
default 32 threads (0.69s to 0.50s). The output is bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
Symbols are matched against version script patterns with an
Aho-Corasick automaton (for single-substring patterns) or an
NFA-based glob matcher, both of which scan the entire symbol name on
every query. But almost all patterns in real version scripts are a
match-all ("local: *;"), an exact name, or a name with one trailing
or leading "*", all of which can be matched with a plain string
comparison.
Handle such patterns directly and reserve the automatons for the
ones that actually need scanning, such as "*foo*" or "a*b".
Co-Authored-By: Claude Fable 5 <[email protected]>
The `README` gives users commands to copy, but `install-build-deps.sh`
is not ran as root, even though the CMake install step is indicated to
run with `sudo`. FreeBSD seems not to come with `sudo` by default, so
modifying the install script might be unwise. Simply place it in front
of the script invocation itself in the `README`.
The affinity partitioner replayed each round's chunk-to-thread
placement so that a thread would re-read the digests it wrote in the
previous round from its own cache. That no longer pays off: after the
recent ICF rework, the propagation loop streams over the digest and
edge arrays and is bound by memory bandwidth rather than cache reuse.
Linking Chromium on a Threadripper 7980X (64 cores, 8 CCDs), the
propagate timer is unchanged at 16 and 128 threads and loses only
about 2 ms (4%) at 32 threads, which is invisible in the total link
time. In exchange, the default auto partitioner retains full work
stealing, which balances better against the skewed per-section edge
counts, and the partitioner no longer needs to be threaded through
two function signatures.
Co-Authored-By: Claude Fable 5 <[email protected]>
compute_digest hashes an eligible relocation target without identity,
relying on a graph edge to keep different targets distinguishable
through the propagation rounds. But gather_edges only created edges
for a section's own relocations, not for the relocations in its FDEs,
so the reference from an FDE to its LSDA never became an edge. As a
result, two functions with identical machine code whose exception
tables catch different types were considered identical, and folding
them changes which exceptions the survivor catches. The bug is as old
as mold's ICF and also affects --icf=safe, since such functions are
usually not address-taken.
Create edges for FDE relocations, mirroring exactly what
compute_digest hashes. The included test, two byte-identical
functions catching different exception types, aborts with an uncaught
exception without this fix.
On 32-bit ARM the equivalent bug still exists through a different
mechanism: exception tables are .ARM.extab sections tied to functions
via .ARM.exidx rather than FDE references, which ICF does not yet
track. The test skips ARM for now.
A Chromium debug build is unaffected (same rounds, same number of
folded sections) since Chromium builds with -fno-exceptions.
Co-Authored-By: Claude Fable 5 <[email protected]>
gather_sections and gather_edges each kept a per-element count array
next to a starting-index array and combined the two at the end.
Counting directly into the index array shifted by one and running the
prefix sum in place removes the count arrays, and the extra entry at
the end means edge_indices[i + 1] is valid for every vertex, so the
propagation loop no longer needs to special-case the last vertex to
find where its edge list ends. The empty-input early return in
gather_edges falls out as well.
No behavior change.
Co-Authored-By: Claude Fable 5 <[email protected]>
A propagation round used to compute a vertex's next digest by hashing
its initial digest together with its neighbors' current digests,
which required keeping three digest arrays: the initial one and two
that alternate between current and next.
Hash the vertex's current digest instead of its initial one. This is
the textbook formulation of iterated partition refinement, and it
distinguishes exactly the same vertex pairs in every round: a current
digest determines the initial digest injectively (up to hash
collisions, which the whole algorithm already assumes away), so the
per-round partitions, the number of rounds and the final equivalence
classes are all unchanged.
The initial digest array disappears, saving 16 bytes per vertex, and
the current/next arrays are now two plainly named vectors that swap
after each round instead of an indexed triple with a flip-flop slot
index.
Co-Authored-By: Claude Fable 5 <[email protected]>
The propagation rounds used to record which vertices' digests had
stopped changing in a `converged` bitmap and skip rehashing them in
later rounds. That was worthwhile when rounds were expensive, but by
now a round is a cheap, uniform data-parallel pass, and the
bookkeeping costs as much as it saves: reading and comparing two
digests per vertex per round just to skip a short hash computation.
Rehash every vertex on every round instead. The convergence check was
the last user of Digest's equality operator, so that is gone too.
For a Chromium release build, the whole ICF pass got slightly faster
(0.166s to 0.152s wall, 4.7s to 4.3s of CPU time).
Co-Authored-By: Claude Fable 5 <[email protected]>
Leaf sections, those with no outgoing references, used to be merged
upfront so that they don't have to participate in the propagation
rounds as graph vertices. That was worth the code when the rounds and
the convergence check were expensive, but both have since become
cheap, and the propagation machinery handles leaves naturally: a
leaf's digest never changes, so it is marked as converged after the
second round and costs almost nothing from then on.
This deletes merge_leaf_nodes, is_leaf and the icf_leaf flag, and
moves eligibility classification to gather_sections. The leader
branch in hash_symbol is gone too because no section has a leader
anymore by the time digests are computed.
Also tidy up the neighborhood: the counting pass of gather_edges
skipped symbols that point to a section fragment before asking for
their input section, but a symbol's origin is a tagged pointer that
is either a fragment or an input section, so get_input_section()
already returns nullptr for fragment symbols. Dropping the redundant
check makes the counting pass and the filling pass identical in
shape.
For a Chromium release build, the ICF pass takes the same time as
before; the propagation rounds absorb roughly what merge_leaf_nodes
used to cost. The graph gets about 30% more vertices, which costs
some transient memory.
Co-Authored-By: Claude Fable 5 <[email protected]>
A digest used to be std::array<uint8_t, 16>, so DigestMap had to
memcpy the bytes into 64-bit words before deriving the slot index and
tag from them. Defining Digest as a struct of two u64 members gives
the words names and makes the copies disappear.
icf.cc is compiled once per target, so the struct is defined in every
one of those translation units. The definitions are all identical, so
this is fine under the one-definition rule, just as the old type
alias was.
No behavior change.
Co-Authored-By: Claude Fable 5 <[email protected]>
Leaf sections used to be merged with a dedicated hash map keyed by
raw section contents, with its own hasher and equality functor. That
equality was subtly weaker than the one the propagation rounds use:
it compared FDE bodies but ignored which CIE each FDE points to, so
two identical functions whose CIEs recorded different personality
routines could be merged incorrectly. It also ignored section flags.
Instead, compute the same digest for leaves that compute_digest
produces for graph sections and merge leaves through DigestMap, the
same structure the propagation rounds use. One definition of section
equality now covers both paths, which fixes the personality bug, and
LeafHasher, LeafEq and the last digest-keyed use of tbb's hash map in
this file disappear.
The trade-off is that leaf merging now relies on 128-bit digest
uniqueness instead of comparing contents exactly, but that is the
same assumption the rest of ICF is already built on.
For a Chromium debug build linked with --icf=all, merge_leaf_nodes
went from 0.07s to 0.10s because a leaf's contents are now hashed by
SipHash twice, once to insert and once to look up. 152 of 3.6M folded
sections are no longer merged; those are the sections the weaker
equality was conflating.
Co-Authored-By: Claude Fable 5 <[email protected]>
To group sections at the end of ICF, we used to build a concurrent
hash map from digest to the lowest-priority section holding it, which
duplicated most of what the digest-counting hash set already does.
That map was also slow enough to construct and destruct that we
deferred freeing it until exit.
This patch generalizes the counting set into DigestMap by adding a
section pointer to each slot. The counting rounds now elect the
lowest-priority section for each digest as a side effect, so by the
time the propagation loop converges, the final round has already
elected the leader of every equivalence class, and the grouping step
reduces to a single lookup pass. The digest-keyed tbb map and its
std::hash specialization disappear, and so does ctx.on_exit, whose
only purpose was to defer freeing that map.
For a Chromium debug build linked with --icf=all, the grouping pass
went from 0.23s to 0.01s and the propagation loop got 0.11s slower
for maintaining the leaders, making the whole ICF pass about 10%
faster (0.89s to 0.79s). The ICF fold decisions are unchanged.
Co-Authored-By: Claude Fable 5 <[email protected]>
To detect convergence of the ICF propagation rounds, we count the
number of distinct digests and stop when it ceases to increase. We
previously gated the count on a cheaper necessary-looking condition:
as long as the number of changed digests per round was still moving,
we assumed the graph could not have converged and skipped counting.
That assumption is wrong. A changing digest does not imply a changing
partition, because sections in the same equivalence class change
their digests in lockstep. For example, in a chain of sections that
are all distinct from the start, the partition is final after the
first round, but the changed-digest count keeps moving until the
propagation depth reaches the chain length, so the gate would delay
termination by the depth of the chain.
Since counting became cheap, we can simply count on every round and
stop as soon as the count repeats, which is the earliest round at
which convergence is provable. propagate() no longer needs to count
changed digests and report them to the caller.
For a Chromium debug build linked with --icf=all, this reduces the
number of propagation rounds from 40 to 28. The 12 saved rounds are
ones in which most sections had already converged, so the wall clock
time is unchanged. The output file is bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
To detect convergence of the ICF propagation rounds, we count the
number of distinct digests and stop when it ceases to increase. We
previously obtained the count by copying all digests and running a
parallel sort, which was expensive enough that we ran it only once
per 10 propagation rounds, overshooting the convergence point by up
to 9 rounds plus a full confirmation batch.
This patch replaces the sort with a concurrent open-addressing hash
set. Since the number of distinct digests cannot exceed the number of
digests, the table is allocated once at full size and never grows.
Slots are stamped with the round number, so the table needs no
clearing between rounds; and a slot match effectively compares an
entire 128-bit digest, so the count is exact under the same
hash-collision assumption the algorithm is already built on. Counting
is now cheap enough to run on every round that passes the existing
changed-digest gate, so the loop exits at the earliest round at which
convergence is provable, and the batching heuristic is gone.
For a Chromium debug build linked with --icf=all, the propagation
loop got 2.2x faster (0.73s to 0.34s), the number of propagation
rounds dropped from 59 to 40, and the whole ICF pass went from 1.29s
to 0.90s. The output file is unchanged.
Co-Authored-By: Claude Fable 5 <[email protected]>
This changes the bundled mimalloc.
mimalloc calls madvise(MADV_HUGEPAGE) on suitable memory regions only
when the caller also allows large OS pages, i.e. hugetlbfs pages.
That condition makes sense for hugetlbfs, whose pages are physically
committed at map time and thus shouldn't back lazily-committed
segments, but transparent huge pages have no such cost because their
pages materialize on fault. Since both were gated on the same flag,
huge mimalloc objects, which are allocated through a code path that
never sets the flag, were never eligible for THP under the madvise
THP mode that Linux distributions ship by default. mold's largest
data structures, such as the symbol and relocation arrays, are
exactly such objects; several gigabytes of them were backed by 4 KiB
pages.
This patch makes the THP advise depend only on the allow_thp option
(enabled by default except on Android) and leaves the hugetlbfs gate
as is. mimalloc v3 effectively behaves this way already; only the v2
line has this problem.
Linking a Chromium debug build gets about 2% faster in the default
fork mode (3.37s to 3.29s) and 15% faster with --no-fork (4.05s to
3.43s) on our machine, as the heap becomes almost fully backed by
huge pages, which also makes the process exit much cheaper.
Co-Authored-By: Claude Fable 5 <[email protected]>
The kernel reclaims an exiting process's address space in a single
thread, which takes an appreciable amount of time for a process that
maps tens of gigabytes of input files. We now drop the page table
entries for input file mappings with madvise(MADV_DONTNEED) on all
threads before exiting. File contents stay in the page cache, so warm
rebuilds are unaffected.
This makes linking a Chromium debug build with --no-fork about 11%
faster (4.55s to 4.05s) on our machine. Without --no-fork, the
teardown happens in the detached child process, which now releases
its memory sooner.
Co-Authored-By: Claude Fable 5 <[email protected]>
The PR_SET_THP_DISABLE prctl flag is inherited across fork and exec,
so a linker spawned from a process that has disabled transparent huge
pages for itself also runs without them, no matter what the system
THP setting says. mold allocates on the order of 10 GiB of heap for a
large link, and backing it with 4 KiB pages instead of 2 MiB pages
makes linking nearly twice as slow (6.2s vs 3.4s for a Chromium debug
build on our machine), so clear the inherited flag at startup.
Co-Authored-By: Claude Fable 5 <[email protected]>
lld does this by default. See: https://crbug.com/529102889
If a DSO is only referenced by sections that are garbage-collected, the
DSO's DT_NEEDED entry will be omitted.
These changes broke cross builds for ppc64le and s390x. I couldn't fix
it easily. Every fix attempt also takes a long time because of slow
qemu-user emulation.
I'll revert CMakeLists.txt and dist.sh to the original state. I don't
think we need to upgrade distros used by dist.sh as long as the binaries
produced using these images work.
Now our tar version is recent enough to use the `--sort` option.
We still need to run gzip explicitly in the pipeline because we want
to pass -n (--no-name) to not save the current time as the file
timestamp in a gzip header.
* update x86-64 to Debian 10. This has glibc 2.28 which matches the oldest actually used distro these days (rhel 8) and gcc-8.5 is new enough to bootstrap clang and libc++, which saves the whole gcc build step
* set the riscv64 target to use debian Trixie, which was the first stable release for that arch
* bullseye (arm,ppac,s390), trixie (riscv), and the loongarch sid snapshot all have clang-19 available
* make sure all images install the libc++-19-dev package to get libc++ static version
* MOLD_USE_LIBCXX now configures the build to look for and use llvm's libc++. This is already the default on macOS and BSD
* Added MOLD_STATIC_* flags for several libs to give more config beyond the overall MOLD_MOSTLY_STATIC
When mold relaxes an instruction, the relocation attached to it no
longer matches the rewritten code. With --emit-relocs that mismatch is
written to the output and confuses consumers such as BOLT. This change
addresses it by rewriting each emitted relocation to the type that
matches the relaxed instruction.
RISC-V and LoongArch already did this in a separate rewrite_reloc_types()
pass that could drift from apply_reloc_alloc; fold it into the latter and
delete the pass. ARM64, x86-64, i386 and s390x did not rewrite types
and now do, e.g. x86-64 GOTPCRELX => PC32, i386 GOT32X => GOTOFF, s390x
GOTENT => PC32DBL, ARM64 GOT loads => ADR_PREL_PG_HI21 and TLSDESC =>
IE/LE. Markers that GNU ld also keeps, such as x86-64 TLS and sparc64
GOTDATA_OP, are left unchanged.
Fixes https://github.com/rui314/mold/issues/1610
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
A symbol exported with an empty default version (foo@@) was treated as a
plain unversioned symbol, so a version script's "local: *" wildcard
localized it and dropped it from the dynamic symbol table; GNU ld keeps
it exported. Route foo@@ through parse_symbol_version() and assign it
VER_NDX_GLOBAL so it overrides the wildcard.
Signed-off-by: Sun Yuechi <[email protected]>
SFrame is a compact alternative to .eh_frame used for asynchronous stack
unwinding by profilers and debuggers. As of binutils 2.46, gas emits
.sframe (Version 3) by default with -Wa,--gsframe, so object files now
routinely carry it. Like .eh_frame, .sframe can't be treated as opaque
bytes: an output section is a single header followed by a PC-sorted index
of Function Descriptor Entries (FDEs) and a blob of Frame Row Entries
(FREs), so the linker has to parse the inputs and rebuild it.
This adds SFrame Version 3 support. We parse each input .sframe into its
FDEs, drop the entries for garbage-collected or ICF-folded functions,
concatenate the survivors, sort the index by function address and rewrite
the header. The FRE data carries no relocations and is position
independent, so it is copied verbatim; only each FDE's func_start is
relocated, which we resolve to a PC-relative offset (the output always
sets SFRAME_F_FDE_FUNC_START_PCREL). A PT_GNU_SFRAME segment is emitted so
the runtime can find the section.
For a relocatable output we can't resolve addresses yet, so the merged
index is left unsorted and each func_start is emitted as a relocation in
.rela.sframe for the final link to resolve, matching GNU ld -r.
We support only Version 3. A .sframe with the wrong magic is reported as
an error, whereas a section written in an older version is ignored.
Fixes https://github.com/rui314/mold/issues/1511
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Large C++ debug libraries on ARM64 (e.g. deal.II's libdeal_II.g.so) can
have a single batch reference enough distinct PLT-routed DSO imports that
the first-pass thunk group exceeds the old fixed 1 MiB max_thunk_size,
tripping the assertion in create_range_extension_thunks().
Replace the branch_distance/5 formula with explicit per-target batch
sizes (32/8/16 MiB for ARM64/ARM32/PPC) and derive max_thunk_size from
the batch as batch_size/2 (16/4/8 MiB). The reservation in d_thunk_end()
already accounts for max_thunk_size, so a larger cap is safe by
construction, and batch_size/2 is about the largest reservation ARM32's
16 MiB branch range can accommodate without throttling the batch.
Add a regression test that calls 50,000 imported functions from a single
function, which on ARM64 produces a first-pass thunk group well over the
old 1 MiB limit.
Fixes https://github.com/rui314/mold/issues/1538
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
actions/upload-artifact rejects names with a forward slash or colon, so
the test-results upload failed for the opensuse/tumbleweed matrix entry
(and would have failed for gentoo/stage3, debian:11, and the ubuntu:*
entries on a test failure). Sanitize matrix.distro by mapping / and :
to - before using it as the artifact name.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
After relaxation, the relocations attached to the relaxed instructions no
longer match the new, shorter instructions. With --emit-relocs we copied the
input relocations verbatim, emitting stale types at stale offsets.
Rewrite each relocation's offset and type in RelocSection::copy_buf to match
the relaxed code, recomputing the outcome the same way apply_reloc_alloc
recomputes its instruction rewrites. -r output is not relaxed, so its
relocations are passed through unchanged.
Fixes https://github.com/rui314/mold/issues/1584
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
The Android packed relocation section (SHT_ANDROID_RELA/REL) holds a
variable-length SLEB128-encoded delta stream, not a table of fixed-size
entries, so sh_entsize should be 0 and the byte stream needs no word
alignment. We were inheriting sizeof(ElfRel) and sizeof(Word) from the
plain .rela.dyn path. These fields aren't consulted at runtime (bionic
locates the blob via DT_ANDROID_RELA/RELASZ), so this only corrects the
metadata and lets the section pack without spurious padding.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
When folding identical sections, ICF was discarding the alignment of the
folded sections and only keeping the alignment of the chosen leader. If
a section with stricter alignment (e.g. 4-byte) was folded into a
section with looser alignment (e.g. 1-byte), the folded function could
be placed at an odd address.
On x86/x86_64, member function pointers use the lowest bit of the
address for virtual/non-virtual distinction. If a non-virtual member
function is folded and placed at an odd address, it is incorrectly
interpreted as virtual, leading to crashes. This fix ensures that we
update the leader's alignment to be the maximum alignment of all folded
sections. We do this in a global pass over all sections (both leaf and
non-leaf merged sections) before the sweep phase.
Fixes#1596
If a relocation points to a preemptible (imported) symbol, it should not
be considered identical to a relocation pointing to a local symbol, even
if their targets currently have identical bodies. Preemptible symbols
can be overridden at runtime, so we must preserve their unique identity.
Fixes#1598
With a PT_LOAD segment, we need to make sure p_offset % p_align ==
p_vaddr % p_align. In set_file_offsets, we try to assign contiguous file
offsets when sections are contiguous in memory. But when the alignment
of section changes and it's bigger than page size, we cannot do that.
The offset might need to be adjusted to follow the alignment rule.
A test binary with
__attribute__((section(".sum"))) __attribute__((aligned(8192))) void sum() {}
Section Headers:
[16] .plt.got PROGBITS 00000000000015c0 0005c0 000008 00 AX 0 0 16
[17] .sum PROGBITS 0000000000002000 001000 00000b 00 AX 0 0 8192
.sum section is assigned contiguous offset to its previous section. As a
result, in create_phdr, .sum is mapped to the same PT_LOAD segment as
previous sections.
Program Headers:
LOAD 0x000588 0x0000000000001588 0x0000000000001588 0x000b80 0x000b80 R E 0x2000
p_offset = 0x588
p_vaddr = 0x1588
p_align = 0x2000
So p_offset % p_align != p_vaddr % p_align, it violates the alignment
rule. As a result, if this scenario happens to a shared library, it
cannot be dlopen() on a system with glibc < 2.35 which is before this
commit:
https://github.com/bminor/glibc/commit/163f625cf9becbb82dfec63a29e566324129c0cd.
Fix it by not assigning contiguous offset when section's alignment is
different and it's bigger than page size.
Several parse-time sites converted `symbol_strtab.data() + st_name`
implicitly to string_view, running strlen on each call. Precompute
a string_view vector parallel to elf_syms and read from it.
InputSection::name() implicitly converts a `const char *` to
string_view, which calls strlen on the shstrtab entry at every call.
Precompute the name as a string_view once in the constructor.
Linking clang Debug is ~1.3% faster; mlir-opt Debug is ~2.6% faster.
ConcurrentMap::insert reads ent.key before writing, so on an empty
slot the read first installs a PTE pointing at the system zero page,
then the CAS triggers wp_page_copy and a cross-CPU TLB shootdown.
Under heavy parallel insertion this shows up as ~3% of link time in
smp_call_function_many_cond plus a much larger inclusive cost in
asm_exc_page_fault.
Pass MAP_POPULATE to pre-fault the whole map in the mmap() thread.
Reads no longer install zero-page PTEs and writes no longer trigger
CoW. Linking clang-23 Debug is ~5% faster, llvm-tblgen ~7%,
clang-tblgen ~9%; max RSS rises by 0.1-0.6%. Falls back cleanly on
platforms that don't define MAP_POPULATE.
Several parse-time sites converted `symbol_strtab.data() + st_name`
implicitly to string_view, running strlen on each call. Precompute
a string_view vector parallel to elf_syms and read from it.
InputSection::name() implicitly converts a `const char *` to
string_view, which calls strlen on the shstrtab entry at every call.
Precompute the name as a string_view once in the constructor.
Linking clang Debug is ~1.3% faster; mlir-opt Debug is ~2.6% faster.
Commit 6d6a128b moved the -C chdir(2) into parse_nonpositional_args(),
which is invoked by both the speculative X86_64 mold_main() and the
re-run via redo_main(). When -C is in effect, the second invocation
re-entered expand_response_files() from the post-chdir directory, so
@rsp with a relative path could no longer be opened.
Save the cwd before parsing and restore it before redo_main() so the
re-run resolves relative paths from the original directory. Also
switch the chdir call itself to std::filesystem::current_path().
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
src/mold.h (HANDLE in MappedFile) and lib/perf.cc (FILETIME /
GetProcessTimes / GetCurrentProcess) both use Windows APIs inside
_WIN32 blocks but neither included <windows.h>. The MSYS2 UCRT64 build
worked because <tbb/enumerable_thread_specific.h> (pulled in via
lib/lib.h) used to #include <windows.h> for its TLS code.
oneTBB 2023.0.0 replaced that bulk include with forward declarations
of just the TLS functions ("declare prototypes of just the needed TLS
functions, instead of including whole windows.h"). When MSYS2 picked
up 2023.0.0, build-msys started failing with "'HANDLE' does not name
a type" at src/mold.h:224 (and the same shape of error in lib/perf.cc
after the mold.h fix landed).
Include <windows.h> explicitly in both translation units so the build
doesn't rely on transitive paths through third-party headers.
Last passing run: https://github.com/rui314/mold/actions/runs/25585325507/job/75112599227
First failing run: https://github.com/rui314/mold/actions/runs/25767054228/job/75681837767
A compile unit's DW_AT_ranges attribute in DW_FORM_rnglistx form holds an
index into the offset table at DW_AT_rnglists_base; the corresponding
entry locates the range list for that DIE. mold was ignoring the index
and iterating every offset, picking up range lists belonging to child
DIEs (subprograms, lexical_blocks, inlined_subroutines) that use
DW_FORM_rnglistx as well.
The resulting .gdb_index address table contained child-DIE ranges nested
inside the CU's range. GDB 17 detects the overlap, prints "warning:
.gdb_index address table has a range (...) that overlaps with an earlier
range, ignoring .gdb_index", and discards the index. Older GDBs accept
the bloated table without complaint -- the duplicate entries all point
to the same CU, so lookups still resolve correctly -- which is why this
went undetected for years.
The bug only triggers when a CU contains a non-contiguous lexical_block,
which clang emits for scopes with EH landing pads from throwing RAII
destructors. Replace the loop with offsets[ranges.value], matching the
DW_FORM_sec_offset branch and GDB's own read_rnglist_index.
The new test invokes clang++ directly since DW_FORM_rnglistx is a
clang-specific behavior; gcc through 13 uses DW_FORM_sec_offset, which
mold has always handled correctly. The test skips cleanly if clang++
isn't installed or doesn't emit rnglistx for the source.
Reported by @MikeWang000000.
Fixes https://github.com/rui314/mold/issues/1578
When --dynamic-linker is passed alongside -shared, mold currently emits
a .interp section and a PT_INTERP program header. Both lld and BFD ld
silently drop --dynamic-linker for -shared outputs because shared
libraries are loaded by the dynamic linker rather than bootstrapped by
the kernel, so PT_INTERP serves no purpose there.
Gate .interp creation on (!shared || pie) so we keep it for regular
executables, -pie executables, and -shared -pie runnable shared
libraries (the libc.so.6-style case), and drop it only for the pure
shared-library case. The PT_INTERP program header at
output-chunks.cc:219 is conditional on ctx.interp being non-null, so
the segment vanishes automatically with no second edit.
Fixes https://github.com/rui314/mold/issues/1572
The Warn() machinery has consulted ctx.arg.suppress_warnings since
6d2dbd42 ("Preparation to support Apple ld64's -w option", March
2023), but no cmdline flag ever set it. Wire -w (the GNU/BFD
spelling) and --no-warnings (the lld long form) to set the field.
Because Warn::Warn early-returns when the flag is true, before the
ctx.arg.fatal_warnings branch sets has_error, -w also cancels
--fatal-warnings — matching lld's documented behavior.
The "warnings only" semantics match GCC, clang, and lld: real errors
still print and still fail the link. BFD ld's -w is broader and
suppresses error messages too, which would hide the cause of link
failures.
Fixes https://github.com/rui314/mold/issues/1493
Linker scripts produced for lld (e.g., the version script Chrome's
build emits) use the POSIX/shell convention `[!...]` to negate a
bracket character class. mold previously accepted only `[^...]`, so
those scripts failed to match. Accept both, treating `!` and `^`
identically at the start of a bracket expression.
Fixes https://github.com/rui314/mold/issues/1575
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Commit 6b8293bf changed the early `ver` computation in
SharedFile::parse so that undef symbols use the raw versym instead
of being forced to VER_NDX_GLOBAL. Some BFD ld versions (notably
binutils 2.35 on Debian 11) emit VER_NDX_LOCAL (0) as the versym
for unversioned undefined dynamic symbols, so the subsequent
`if (ver == VER_NDX_LOCAL) continue;` check silently dropped those
undef references. With the DSO's reference invisible to mold, the
matching object-file definition was never marked as exported, and
at runtime the dynamic loader failed with "undefined symbol".
Restore the old behavior: defined symbols with VER_NDX_LOCAL are
still skipped (they aren't really exposed for dynamic linking),
but undef references are always processed.
Also fix the verdef/verneed merge in the same path. vd_ndx and
vna_other share a single versym index space (a producing linker
assigns them from one counter), so a single vector covers both.
read_verdef and read_verneed are folded into one read_version_strings
that populates that shared-index vector directly, replacing the
buggy `append` concatenation.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
For an undefined symbol in a shared library, mold treated the versym
as VER_NDX_GLOBAL and registered the symbol under its bare name. That
meant an undef foo@VER_1.0 in libbar.so was looked up as just foo, so
a non-default foo@VER_1.0 in another DSO did not satisfy it and
--no-allow-shlib-undefined wrongly reported "undefined symbol: foo".
Parse .gnu.version_r and, when an undef has a versym indexing into it,
register the symbol as name@VERSION so the existing versioned-symbol
resolution path picks up matching definitions.
Fixes https://github.com/rui314/mold/issues/1571
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
`readelf | grep -q` plus `set -o pipefail` is fragile: when grep -q
matches it stops reading immediately, readelf gets SIGPIPE on its next
write, and pipefail propagates the resulting 141. Alpine (musl) hits
this where glibc happens to escape it. Drop -q so grep drains stdin.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
The new StartStopMap parameter contains std::string_view, so the
unqualified call to visit() inside mark() and the recursive call inside
visit() itself were resolved to std::visit by ADL on gcc 10 (Debian 11),
breaking the build.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
A section whose name is a valid C identifier was treated as a GC root,
keeping it (and everything reachable from it) alive even when nothing
referenced its synthesized __start_/__stop_ marker symbols. That could
pull in unrelated code with undefined references and cause spurious
"undefined symbol" errors that lld and ld.bfd don't produce.
Build a multimap of c-identifier-named sections up front and, during
marking, when a relocation targets __start_<name> or __stop_<name>,
mark every section named <name> alive.
Fixes https://github.com/rui314/mold/issues/1571
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Pre-Android-12 (API 30 and earlier) bionic loaders predate the official
ELF gABI assignment of DT_RELR/DT_RELRSZ/DT_RELRENT and use legacy
OS-specific tag values DT_ANDROID_RELR (0x6fffe000), DT_ANDROID_RELRSZ
(0x6fffe001), DT_ANDROID_RELRENT (0x6fffe003), and section type
SHT_ANDROID_RELR (0x6fffff00). With --use-android-relr-tags, mold emits
those instead of the standard ones so binaries linked with
--pack-dyn-relocs=relr can be loaded on those older devices. The flag
is undocumented (not listed in --help) since it's only relevant to
shipping to legacy Android targets.
https://github.com/rui314/mold/issues/1544
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Implements the Android Packed Relocations Format (APS2) used by Android
binaries to compress dynamic relocations. Two new modes are accepted:
`--pack-dyn-relocs=android` encodes all dynamic relocations as APS2 in
.rela.dyn (SHT_ANDROID_RELA) and emits DT_ANDROID_RELA(SZ) instead of
DT_RELA(SZ); `--pack-dyn-relocs=android+relr` is the same but with
R_RELATIVE relocations diverted to .relr.dyn via the existing RELR path.
The encoder groups consecutive relocations sharing offset_delta and
r_info to amortize SLEB128 overhead. The encoded bytes are decoded by
Android's bionic loader and by llvm-readelf, which the new test uses to
verify correctness.
main.cc gains a final set_osec_offsets() call after fix_synthetic_symbols.
Symbol values assigned there change addends for relocations that
reference synthetic symbols, which can shift the SLEB128-encoded section
size; re-running the layout convergence keeps sh_size consistent with
the final encoding.
Fixes https://github.com/rui314/mold/issues/1544
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Previously each chunk wrote its dynamic relocations into a magic offset
in .rel.dyn (tracked via per-chunk reldyn_offset), with a separate
construct_relr pass building per-chunk relr vectors. Now each chunk
returns its dynrels via a virtual collect_dynrels(), and RelDynSection
aggregates them into a single relocs vector that it owns and writes.
RELR-eligibility is determined by checking r_offset alignment of the
collected ElfRel entries (replacing the ABS_REL_RELR kind).
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Generalize the zstd:N syntax added in the previous commit so that the
zlib compression level can also be specified. zlib levels 0 through 9
are accepted, where 0 means no compression. Plain `zlib` remains
equivalent to `zlib:1` (the previous hardcoded default).
Also drop the objcopy/zstdcat round-trip from the zstd-level test in
favor of a simpler readelf check, matching the new zlib-level test, and
add error-message coverage for out-of-range levels.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Change the compression level syntax from zstd-<level> to zstd:<level>
as requested by @rui314 for consistency with lld.
Signed-off-by: Pierluigi Lenoci <[email protected]>
Allow users to specify a zstd compression level (1-22) for debug section
compression via --compress-debug-sections=zstd-N, where N is the desired
level. Higher levels achieve better compression ratios at the cost of
compression speed. The plain 'zstd' option remains equivalent to 'zstd-3'
(the previous hardcoded default).
Closes: https://github.com/rui314/mold/issues/1290
Signed-off-by: Pierluigi Lenoci <[email protected]>
pcaddi computes PC + (imm << 2), so the relaxation from a GOT load
(pcalau12i + ld.d) to pcaddi is valid only when the symbol's address
is 4-byte aligned. Without this check, a reference to a misaligned
symbol was relaxed to a pcaddi that resolves to the address with its
low two bits cleared, producing a wrong pointer.
Fixes https://github.com/rui314/mold/issues/1567
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
The parallel writer that fills (type, cu_idx) tuples for each name
appended entries in thread-scheduling order, so names referenced from
multiple compilation units produced different .gdb_index bytes across
runs. Sort each name's tuple array after the parallel append so the
output is reproducible while preserving parallelism.
Fixes https://github.com/rui314/mold/issues/1566
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Previously, --omagic only flipped segment permissions to RWX but
still page-aligned PT_LOAD segments and emitted PT_GNU_RELRO, so
text and data ended up in separate page-padded segments. This
mismatched the documented behavior ("makes the first data segment
not aligned to a page boundary") and GNU ld's --omagic, which also
disables shared linking.
Make --omagic imply --Bstatic, disable RELRO, and stop forcing
PT_LOAD's p_align to the page size. Also gate chunk->is_relro on
z_relro so a disabled RELRO no longer leaks into address layout.
Fixes https://github.com/rui314/mold/issues/1562
When linking with LTO, archive members extracted pre-LTO to satisfy
references from IR objects could cause false duplicate symbol errors.
This happened in two ways:
1. The pre-LTO check_duplicate_symbols flagged conflicts between LTO IR
objects and regular objects. These conflicts are handled by the LTO
backend and should not be reported.
2. After LTO compilation, archive members that were extracted pre-LTO
retained stale is_reachable flags even when the LTO output superseded
all their symbols. The post-LTO duplicate check then saw both the
archive member and the LTO output defining the same symbol.
Fix both issues:
- In check_duplicate_symbols, skip errors where one side is an LTO IR
object and the other is not. Only IR-vs-IR duplicates need catching.
- In do_lto, reset reachability for archive members before the post-LTO
resolve_symbols so that only genuinely needed members are re-extracted.
Fixes https://github.com/rui314/mold/issues/1421
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
This relocation is used by clang's -fexperimental-relative-c++-abi-vtables
to emit 32-bit GOT-relative PC-relative offsets in vtable entries (e.g. for
RTTI pointers). It resolves to G + GOT + A - P as a 32-bit value.
Fixes https://github.com/rui314/mold/issues/1561
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
This patch fixes two distinct blind spots in how `mold` handles
--no-allow-shlib-undefined checks during the final link phase:
1. The Hidden Static Provider Bug (False Negative)
Previously, if a shared library requested a symbol that was provided by a
static object file with STV_HIDDEN, `mold` would silently overwrite the
global symbol visibility to STV_DEFAULT and rubber-stamp the binary,
resulting in a guaranteed `symbol lookup error` at runtime.
Fix: We now interrogate the raw `elf_syms` array of the static object
provider. If the original visibility is hidden/internal, we flag it as an
invalid provider and throw the error, matching GNU ld/lld behavior.
2. The LTO Shadowing Bug (False Positive)
When LTO is heavily utilized, GCC's plugin may optimize away symbols that
are theoretically provided by static objects if they aren't used internally.
If a shared library still needs that symbol, `mold`'s static state
registered the provider as NULL and threw an error, ignoring the fact
that a loaded DSO (like libc.so) offered a valid public fallback.
Fix: Introduced `has_public_dso_fallback()`. Before throwing an undefined
symbol error, `mold` performs a rapid scan of `ctx.dsos`. If a public
fallback exists dynamically, the error is safely suppressed.
Originally discussed in #1535, the debug data generated by
--separate-debug-file is being corrupted. In the best case scenario,
there's many blank/nameless undefined symbols in the symtab, but in
most production scenario I've encountered, the file is corrupt / gdb
refuses to load it:
$ gdb main
Reading symbols from $PWD/main.dbg...
BFD: /$PWD/main.dbg: invalid string offset 4130582 >= 1345 for section `.strtab'
BFD: /$PWD/main.dbg: invalid string offset 16777228 >= 1345 for section `.strtab'
$ readelf -Ws main.dbg
Symbol table '.symtab' contains 139 entries:
[...]
38: 0000000000000000 0 SECTION LOCAL DEFAULT 38 .strtab
39: 0000000000000000 0 SECTION LOCAL DEFAULT 39 .symtab
40: 0018a80200010000 0xb10000003f071500 NOTYPE LOCAL DEFAULT bad section index[3232] <corrupt>
41: 0000003f07140000 0xfd02000100000cc2 NOTYPE LOCAL HIDDEN bad section index[11500] <corrupt>
42: 02000100000cd300 0x3f0712000008a3 <unknown>: 7 LOCAL PROTECTED [<other>: 3c] UND <corrupt>
43: 3f07110000056202 0x100000cf5000000 NOTYPE LOCAL DEFAULT 1 <corrupt>
This appears to be happening because the strtab size is set to be larger
than its true value, causing garbage data to be placed in it, which
breaks the symtab.
This regression occurred in 5b71029, which fixed a different form of
corruption (the PHDR symbol escaping into the section headers, shifting
the section offsets), but created this strtab corruption. The root cause
is that `compute_section_headers` was moved prior to the wiping of
irrelevant chunks. `compute_section_headers` computes the strtab size by
adding up individual chunk strtab contributions. It needs to occur _after_
the wiping so that the strtab doesn't account for the data before the
wipe.
The core of this patch is really just a partial revert of 5b71029--the
compute_section_headers call appears to have been in the correct place
prior to this change; I've added a test that catches the issue by
intentionally bloating the size of the `.plt` section with many external
symbols. This section is wiped in the dbg data, so by bloating it, the
test can inspect for corruption.
Fixes#1535
Signed-off-by: Alex Karle <[email protected]>
When mold auto-detects an unsupported target from ELF headers,
redo_main() previously called abort() with no context. This replaces
it with a descriptive error message matching the style used by the
-m flag path in cmdline.cc.
Fixes#1553
Interestingly, calling fallocate on tmpfs is very slow. It took
almost a second for a fresh ~5 GiB file, even though intuititvely
it should finish very quickly.
Make Script<E>::get_script_output_type support AS_NEEDED inside an INPUT
or GROUP group, so that the libatomic_asneeded.so linker script used by
GCC 16 works.
Fixes#1545
Signed-off-by: Jonathan Wakely <[email protected]>
This change eliminates the cost of zero-initialization on vector
resizing by using malloc/realloc instead. We also parallelize the
loops to write adddress areas and types to .gdb_index.
We read version script files in cmdline.cc, so we need to handle -C
and --chroot early. Otherwise, a version script passed by command line
can cause a "file not found" error.
By popular request, this commit adds a new cmake variable that allows
building mold with a limited set of supported targets, which can help
reduce build time.
The use of this flag is discouraged unless you build mold frequently
for personal use. Otherwise, build mold with the default settings for
simplicity and maximum compatibility. In particular, distributions
should not build mold with a limited set of supported targets.
Fixes https://github.com/rui314/mold/issues/1364
Read-only data sections containing uninitialized variables are essentially
useless because reading from them always returns 0 at any time, so if they
exist, they are there for a reason. We do not want to automatically
convert it to a BSS.
https://github.com/rui314/mold/issues/1514
Currently, when using --static inside --push-state and --pop-state, we
incorrectly mark the global state as static. This commit changes to the
correct behavior where global state is marked as static only when
--static is used at the outer most (outside of any --push-state,
--pop-state).
Quote from SH-4 CPU Core Architecture manual:
The opcode 0xFFFD is guaranteed to be defined in any SH-4
architecture revision. Other unused opcodes may be treated as
reserved in any particular SH-4 implementation.
The only undefined opcode which the architecture guarantees to cause
a General Illegal Instruction Exception is 0xFFFD.
On 64-bit ARM, mimalloc is built with the compiler flag
`-march=armv8.1-a` by default. This flag advises GCC to make use of
ARMv8.1-A-specific instructions such as `ldaddal`. Machines based on the
older ARMv8-A instruction set - for example the Raspberry Pi 4 -
experience 'Illegal instruction (core dumped)' crashes as a result.
To ensure compatibility with machines that do not support ARMv8.1-A,
disable architecture-specific optimisations by setting the CMake option
`MI_NO_OPT_ARCH` to `ON`.
As of mimalloc 2.2.3, 64-bit ARM is the only architecture affected by
the `MI_NO_OPT_ARCH` option.
Delete this message and describe the bug here. Note that we only fix bugs in the most recent release of mold, so please make sure that you are using the most recent version.
This page explains the following warning message and how to fix it.
mold emits this message when it sees an object file that may not be
compatible with mold.
```
mold: warning: foo.o: this file may cause a segmentation fault because it requires an executable stack. See https://github.com/rui314/mold/tree/main/docs/execstack.md for more info.
```
# Background
On modern computers, the stack area (to which local variables are
stored) cannot contain executable code. If the control reaches the
stack area, the CPU refuses to execute any code there and the program
is usually terminated due to segmentation fault.
This is a security measure. The stack area used to be executable (old
CPUs generally execute any code as long as it is in a readable memory
region), but that provided an easy attack vector to a malicious user.
They wrote executable code to the stack area using some buffer
overflow bug and jumped there to run arbitrary code in a remote server
process.
To prevent this type of attack, the stack area is no longer executable
since the early 2000s. On Linux, the stack's executable-ness is
controlled by a bit in an executable, and the loader respects that
bit. The bit is set by the linker.
GCC had (and still has) a feature that depends on the executable
stack, so they invented a way to tell the linker to mark the stack
executable. Specifically, if an object file contains a
`.note.GNU-stack` section with the `SHF_EXECSTACK` bit, GNU linker
silently makes the stack of an output file executable.
But the GNU linker's behavior is dangerous. If you accidentally link
an object file that has that marker section, the entire stack area
silently becomes executable, disabling the security mechanism.
Therefore, mold simply ignores that marker section. If you are using
mold, you need to explicitly pass `-z execstack` to the linker to make
@@ -48,9 +48,11 @@ Whether good or bad, you should keep these semantics in mind to understand Unix
.P
\fBmold\fR does not have any host\-specific default settings\. This is contrary to the GNU linkers, for which some configurable values, such as system\-dependent library search paths, are hard\-coded\.\fBmold\fR depends only on its command\-line arguments\.
.SH"OPTION NOTATIONS"
Multi\-letter long options may precede either a single dash or double dashes, except for those starting with the letter "o"\. For historical reasons, long options beginning with "o" must precede double dashes\.
Long options are spelled with two dashes\. Some also accept a single dash, but which ones do is somewhat arbitrary for historical reasons\.\fBmold\fR follows the conventions of other linkers for compatibility\. For example, \fB\-\-export\-dynamic\fR can be spelled \fB\-export\-dynamic\fR, while \fB\-\-execute\-only\fR requires two dashes\.
.P
For example, you can spell\fB\-\-as\-needed\fR as \fB\-as\-needed\fR, but \fB\-\-omagic\fR must not be spelled as \fB\-omagic\fR\.\fB\-omagic\fRwill be interpreted not as \fB\-\-omagic\fR but as \fB\-o magic\fR\.
This can cause unexpected behavior when constructing command\-line options in a script\.For example, you might use\fB"\-e$foo"\fR to set the entry point symbol to the value of \fB$foo\fR\. However, if \fB$foo\fR is \fBxport\-dynamic\fR, the argument becomes \fB\-export\-dynamic\fRand is interpreted as \fB\-\-export\-dynamic\fR\.
.P
To avoid such ambiguities, we recommend always using two dashes for long options and separating a single\-letter option from its argument with a space\. For example, use \fB\-e $foo\fR or \fB\-\-entry=$foo\fR to specify the entry point symbol in a script\.
.SH"MOLD\-SPECIFIC OPTIONS"
.TP
\fB\-\-chroot\fR=\fIdir\fR
@@ -132,6 +134,17 @@ Use multiple threads\. By default, \fBmold\fR uses as many threads as the number
\fB\-\-quick\-exit\fR, \fB\-\-no\-quick\-exit\fR
Use or do not use \fBquick_exit\fR to exit\.
.TP
\fB\-\-zero\-to\-bss\fR
Convert all\-zero data sections into BSS\.
.IP
When this option is enabled, \fBmold\fR scans input data sections that are not of type \fBSHT_NOBITS\fR and checks whether their contents consist solely of zero bytes\. Such sections are then converted into BSS (\fBSHT_NOBITS\fR) sections\. This reduces the size of the output file, since BSS sections occupy no space in the file image\.
.IP
This behavior is especially useful for user\-defined sections created with \fB__attribute__((section("\.sectname")))\fR that contain uninitialized global variables\. GCC and Clang do not automatically mark such sections as BSS even if their contents are entirely zero, and instead emit them as regular data sections\.
.IP
For example, consider \fB__attribute__((section("\.sectname"))) int vec[256];\fR\.
.IP
By default, this results in a \fB\.sectname\fR section of type \fBSHT_PROGBITS\fR filled with zeros\. With \fB\-\-zero\-to\-bss\fR, the linker will recognize it as empty data and convert it to a \fBSHT_NOBITS\fR section, reducing the output file size without changing runtime semantics\.
As a security measure, some CPU instruction sets have recently gained a feature to protect control flow integrity by disallowing indirect branches by default\. If the feature is enabled, the instruction that is executed immediately after an indirect branch must be an branch target marker instruction, or a CPU\-level fault will raise\. The marker instruction is also known as "landing pad" instruction, to which indirect branches can land\. This feature makes ROP attacks harder to conduct\.
.IP
@@ -141,7 +154,9 @@ By default, the compiler always emits a landing pad at the beginning of each glo
.IP
If \fB\-\-rewrite\-endbr\fR is given, mold conducts a whole program analysis to identify functions whose addresses are actually taken and rewrites landing pads with no\-ops for non\-address\-taken functions, reducing the attack surface\.
.IP
This feature is currently available only on x86\-64\.
This feature is currently available only on x86\-64 and arm64\.
.IP
On arm64, the landing pad is \fBbti c\fR\. Note that code compiled with \fB\-mbranch\-protection=standard\fR (BTI + pointer authentication) starts functions with \fBpaciasp\fR, which doubles as a landing pad, so there is no separate \fBbti c\fR to rewrite and this option has no effect on such code\.
.SH"GNU\-COMPATIBLE OPTIONS"
.TP
\fB\-\-help\fR
@@ -182,7 +197,10 @@ Omit \fB\.debug_*\fR sections from the output file\.
Read linker script from \fIfile\fR\.
.TP
\fB\-X\fR, \fB\-\-discard\-locals\fR
Discard temporary local symbols to reduce the sizes of the symbol table and the string table\. Temporary local symbols are local symbols starting with \fB\.L\fR\. Compilers usually generate such symbols for unnamed program elements such as string literals or floating\-point literals\.
Discard temporary local symbols to reduce the sizes of the symbol table and the string table\. Temporary local symbols are local symbols starting with \fB\.L\fR\. Compilers usually generate such symbols for unnamed program elements such as string literals or floating\-point literals\. This is the default; use \fB\-\-discard\-none\fR to keep temporary local symbols\.
.TP
\fB\-\-discard\-none\fR
Keep all local symbols in the symbol table, including temporary ones starting with \fB\.L\fR, except ones in mergeable sections\. This restores the default behavior of GNU ld\.\fBmold\fR and \fBld\.lld\fR discard temporary local symbols by default\.
@@ -247,6 +265,9 @@ Alias for \fB\-\-section\-start=\.data=\fR\fIaddress\fR\.
\fB\-\-Ttext\fR=\fIaddress\fR
Alias for \fB\-\-section\-start=\.text=\fR\fIaddress\fR\.
.TP
\fB\-\-Ttext\-segment\fR=\fIaddress\fR
Set the address of the first byte of the text segment\.
.TP
\fB\-\-allow\-multiple\-definition\fR
Normally, the linker reports an error if there are more than one definition of a symbol\. This option changes the default behavior so that it doesn't report an error for duplicate definitions and instead use the first definition\.
.TP
@@ -271,8 +292,8 @@ Synonym for \fB\-\-build\-id=sha256\fR\.
Compress DWARF debug info (\fB\.debug_*\fR sections) using the zlib or zstd compression algorithm\.\fBzlib\-gabi\fR is an alias for \fBzlib\fR\.\fBzlib:\fR\fIN\fR selects the zlib compression level, where \fIN\fR is between 0 and 9 (0 means no compression), and \fBzstd:\fR\fIN\fR selects the zstd compression level, where \fIN\fR is between 1 and 22\. Higher levels achieve better compression ratios but are slower\.\fBzlib\fR is equivalent to \fBzlib:1\fR and \fBzstd\fR is equivalent to \fBzstd:3\fR\.
.TP
\fB\-\-defsym\fR=\fIsymbol\fR=\fIvalue\fR
Define \fIsymbol\fR as an alias for \fIvalue\fR\.
@@ -360,10 +381,16 @@ Embed a specified string into the \fB\.note\.package\fR section\. This option is
.IP
An argument to this option is treated as percent\-encoded and decoded before being inserted into the section, allowing you to avoid the use of the comma (\fB,\fR) character in the argument\. This is useful because the compiler replaces all occurrences of commas in \fB\-Wl,\fR with spaces before forwarding them to the linker\. Note that \fBmold\fR always interprets the argument as percent\-encoded, so you also need to escape all occurrences of \fB%\fR as \fB%25\fR\.
If \fBrelr\fR is specified, all \fBR_*_RELATIVE\fR relocations are put into \fB\.relr\.dyn\fR section instead of \fB\.rel\.dyn\fR or \fB\.rela\.dyn\fR section\. Since \fB\.relr\.dyn\fR section uses a space\-efficient encoding scheme, specifying this flag can reduce the size of the output\. This is typically most effective for position\-independent executable\.
.IP
Note that a runtime loader has to support\fB\.relr\.dyn\fRto run executables or shared libraries linked with \fB\-\-pack\-dyn\-relocs=relr\fR\. As of 2022, only ChromeOS, Android and Fuchsia support it\.
If \fBandroid\fR is specified, all dynamic relocations are encoded in Android's packed relocation format (APS2) and stored in the\fB\.rel\.dyn\fRor \fB\.rela\.dyn\fR section, whose section type is changed to \fBSHT_ANDROID_REL\fR or \fBSHT_ANDROID_RELA\fR\. The dynamic table is updated to use \fBDT_ANDROID_REL\fR / \fBDT_ANDROID_RELA\fR and their \fBSZ\fR variants in place of the standard tags\.
.IP
\fBandroid+relr\fR combines both: \fBR_*_RELATIVE\fR relocations go to \fB\.relr\.dyn\fR and the rest are APS2\-encoded\. This is the configuration used by most Android system binaries\.
.IP
Note that a runtime loader has to support \fB\.relr\.dyn\fR to run executables or shared libraries linked with \fB\-\-pack\-dyn\-relocs=relr\fR\. As of 2026, glibc (since 2\.36), musl (since 1\.2\.4), Android, ChromeOS, and Fuchsia all support it\. The APS2 format, on the other hand, is only handled by Android's bionic loader; the encoder itself is target\-agnostic, but the produced binary can only be loaded on platforms where bionic (or a compatible loader) handles \fBDT_ANDROID_REL(A)\fR\.
@@ -429,6 +456,9 @@ How to handle undefined symbols\.
\fB\-\-version\-script\fR=\fIfile\fR
Read version script from \fIfile\fR\. If \fIfile\fR does not exist in the current directory, it is searched from library search paths for the sake of compatibility with GNU ld\.
.TP
\fB\-w\fR, \fB\-\-no\-warnings\fR
Suppress warnings and cancel \fB\-\-fatal\-warnings\fR\.
If one memory page contains multiple segments, the page protection bits are set in such a way that the needed attributes (writable or executable) are satisfied for all segments\. This usually happens at a boundary of two segments with two different attributes\.
.IP
\fBseparate\-loadable\-segments\fR adds paddings between segments with different attributes so that they do not share the same page\. This is the default\.
\fBseparate\-loadable\-segments\fR adds paddings between segments with different attributes so that they do not share the same page\.
.IP
\fBseparate\-code\fR adds paddings only between executable and non\-executable segments\.
.IP
\fBnoseparate\-code\fR does not add any paddings between segments\.
\fBnoseparate\-code\fR does not add any paddings between segments\. This is the default\.
.TP
\fB\-z defs\fR, \fB\-z nodefs\fR
\fB\-z defs\fR, \fB\-z undefs\fR
Report undefined symbols (even with \fB\-\-shared\fR)\.
.TP
\fB\-z shstk\fR
Enforce shadow stack by turning \fBGNU_PROPERTY_X86_FEATURE_1_SHSTK\fR bit in \fB\.note\.gnu\.property\fR output section\. Shadow stack is part of Intel Control\-flow Enforcement Technology (CET), which is available since Tiger Lake (2020)\.
If a section name is valid as a C identifier (i\.e\., it matches \fB/^[_a\-zA\-Z][_a\-zA\-Z0\-9]*$/\fR), mold creates \fB__start_SECNAME\fR and \fB__stop_SECNAME\fR symbols to mark the beginning and end of the section, where \fBSECNAME\fR is the section name\.
.IP
You can make these marker symbols visible from other ELF modules by passing \fB\-z start_stop_visibility=protected\fR\. Default is \fBhidden\fR\.
You can make these marker symbols visible from other ELF modules by passing \fB\-z start\-stop\-visibility=protected\fR\. Default is \fBhidden\fR\.
\fBmold\fR by default reports an error if dynamic relocations are created in read\-only sections\. If \fB\-z notext\fR or \fB\-z textoff\fR are given, \fBmold\fR creates such dynamic relocations without reporting an error\.\fB\-z text\fR restores the default behavior\.
Error(ctx)<<sym<<": GNU ifunc symbol is not supported on m68k";
switch(rel.r_type){
caseR_68K_16:
caseR_68K_8:
scan_absrel(ctx,sym,rel);
break;
caseR_68K_PC32:
caseR_68K_PC16:
caseR_68K_PC8:
scan_pcrel(ctx,sym,rel);
break;
caseR_68K_GOTPCREL32:
caseR_68K_GOTPCREL16:
caseR_68K_GOTPCREL8:
caseR_68K_GOTOFF32:
caseR_68K_GOTOFF16:
caseR_68K_GOTOFF8:
sym.flags|=NEEDS_GOT;
break;
caseR_68K_PLT32:
caseR_68K_PLT16:
caseR_68K_PLT8:
if(sym.is_imported)
sym.flags|=NEEDS_PLT;
break;
caseR_68K_TLS_GD32:
caseR_68K_TLS_GD16:
caseR_68K_TLS_GD8:
sym.flags|=NEEDS_TLSGD;
break;
caseR_68K_TLS_LDM32:
caseR_68K_TLS_LDM16:
caseR_68K_TLS_LDM8:
ctx.needs_tlsld=true;
break;
caseR_68K_TLS_IE32:
caseR_68K_TLS_IE16:
caseR_68K_TLS_IE8:
sym.flags|=NEEDS_GOTTP;
break;
caseR_68K_TLS_LE32:
caseR_68K_TLS_LE16:
caseR_68K_TLS_LE8:
check_tlsle(ctx,sym,rel);
break;
caseR_68K_32:
caseR_68K_TLS_LDO32:
caseR_68K_TLS_LDO16:
caseR_68K_TLS_LDO8:
break;
default:
Error(ctx)<<*this<<": unknown relocation: "<<rel;
}
}
}
}// namespace mold
Loaded 100 of 3216 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Rui Ueyama
