Running the suite on GCC Compile Farm machines found test scripts that
don't work as intended on some hosts:
- on_qemu() took KVM guests on POWER for qemu-user
- the native POWER10 target ignored CPU and shared ppc64le's directory
- readelf 2.35 appends "[<localentry>: 1]" to POWER10 function names
- compressed debug sections hid .debug_str from `readelf -p`
- some tests assumed Linux, glibc, GCC-only flags or g++
- an x86-64 executable was run without $QEMU on a non-x86 host
- old binutils lack multiple .symver and SHF_LINK_ORDER
- GCC 8's `gcc -r` links start files into relocatable output
- debug-macro-section.sh grepped for an offset format objdump doesn't print
With -g3, GCC emits a COMDAT group for the .debug_macro contents of each
header file, and other .debug_macro sections refer to it. mold doesn't
deduplicate these groups because it would resolve references to discarded
copies to 0 (https://github.com/rui314/mold/issues/438), but -r output
dropped the groups while keeping SHF_GROUP on their members. Now -r keeps
every copy as a COMDAT group. GNU ld -r deduplicates them and redirects
the references to the kept copies; without that redirection, mold leaves
deduplication to the final link, where GNU ld, lld and mold handle the
groups as they would in the original objects.
--print-dependencies reads the relocations of every input section,
including non-allocated ones whose CREL tables are not decoded when the
file is read. It read them as an array, which reinterpreted such a table
as RELA records and failed an assertion. C++ mold's get_rels() decoded
the table on demand instead.
A read-only pass doesn't need the table materialized, so iterate with
the streaming relocation iterator as --stats does, and document when
the array accessor can be used.
rustc's LoongArch targets use the medium code model, whose calls reach
±128 GiB, but the cc crate passes no -mcmodel to the C compiler unless
-Ccode-model is given explicitly, so C code got the normal code model,
whose direct calls reach only ±128 MiB. A debug build of mold has more
than 128 MiB of text, so calls from C code such as zstd, mimalloc and
lto-message.c to the PLT failed to link with R_LARCH_B26 overflows on
a native LoongArch host. We now give -Ccode-model=medium explicitly.
A section group whose flag word is 0 is not a COMDAT group. It is never
deduplicated; it only ties its members together so that they are kept or
discarded as a unit. annobin emits such groups, e.g. into crt1.o on
AlmaLinux 8. mold ignored them, but -r output kept SHF_GROUP on their
members, leaving SHF_GROUP sections that no SHT_GROUP lists, which the
gABI forbids and objdump 2.30 rejects. -r output now keeps each such
group with flag word 0, as GNU ld and lld do. Other links still treat
its members as ordinary sections.
CREL tables of non-allocated sections are not decoded when an object
file is read; their relocations are streamed when they are applied.
On ppc64v1, rewrite_opd() also rewrites relocations of non-allocated
sections such as .debug_info, and rels_mut() reinterpreted such a table
as RELA records. Linking a CREL object built with -g failed an
assertion, and a table whose size was a multiple of 24 would be misread.
A CREL table can't be rewritten in its compressed form, so rels_mut()
now decodes it into the side table first, as get_rels() in C++ mold
did. relocation_iter() then returns the rewritten records from there.
This reverts aed5a38d1, which made -e accept a number as an entry
address if no symbol has that name, as GNU ld and lld do. New features
are deferred until after 3.0.0.
This reverts f03b74595, 48428d389 and 3a1b9c6c2, which merged
GNU_PROPERTY_AARCH64_FEATURE_1_AND, emitted BTI PLTs and landing pads
for range extension thunks, and added -z force-bti, pac-plt, gcs,
bti-report and gcs-report.
GCC 16 no longer records -mbranch-protection in .note.gnu.property but
in AArch64 build attributes, which we don't read, so outputs silently
lost the BTI, PAC and GCS bits and -z force-bti warned about every
object file. We'll bring the feature back with build attribute support
after 3.0.0. Until then, ARM64 outputs have no .note.gnu.property, as
in 2.42.1.
These sorts have keys that are unique among the elements, so a stable
sort produces the same order. Using an unstable sort, as C++ mold did,
avoids the stable sort's buffer and signals that ties don't matter.
Two mapping symbols can share an offset, e.g. when `ld -r` places a
section ending with a mapping symbol before one starting with another.
The later one in the symbol table applies, as in the C++ version, which
used stable_sort. An unstable sort could leave BE8 code unswapped.
Default features are turned off because uuid's std feature refers to
optional wasm32-only crates, and Cargo puts those into Cargo.lock even
though nothing builds them. With them off, uuid is the only new package.
When run in an extracted source archive, dist.sh cloned the release tag
as a bare repository into .git to get the commit hash and timestamp.
Cargo refuses to build in a directory whose .git is a bare repository,
and the embedded hash named the tag rather than the files actually
built. A worktree doesn't work either, because its .git file points
outside the build container and the hash is silently omitted. Stop with
an error unless .git is a directory.
Non-allocated strings were merged in a background Rayon task, but Rayon
has no task priorities: a thread waiting for a foreground parallel pass
could pick up a long piece of the merge, and the pass waited up to
200 ms for it. Resolve these sections in compute_section_sizes as before
2316c952a. Within noise on large debug builds (0.94-1.04).
Instruction pairs such as ADRP+ADD on ARM64 refer to the same string
with two consecutive relocations, and each got a dummy symbol of its own.
Reuse the previous relocation's dummy symbol if it refers to the same
section symbol with the same addend. Since both relocations of a pair now
refer to one symbol, ADRP+ADD relaxation applies to them too. On
chrome-release-arm64, reattach_section_pieces 39 -> 30 ms and peak RSS
5.86 -> 5.56 GB.
Range extension thunk creation and removal lay out input sections in a
serial loop that looked up each member through its file, costing a few
dependent cache misses per section. Read sizes and alignments into an
array in parallel first, and publish the offsets in parallel. On
chrome-release-arm64, compute_section_sizes 34 -> 27 ms and
remove_redundant_thunks 25 -> 15 ms; on chrome-debug-arm64, 87 -> 81 ms
and 72 -> 50 ms.
CIE deduplication, which both ICF and .eh_frame layout run, compared each
CIE with every distinct CIE found so far in a serial loop. Compute their
SipHash digests in parallel with a random key and number the classes
through a hash table instead, assuming that digests never collide as ICF
does. FDE offsets are now assigned in parallel too. uniquify_cies 9 -> 2
ms on chrome-release-arm64; .eh_frame layout 15 -> 8 ms on
chrome-debug-x86_64.
set_visibility() is only called on exclusively owned symbols, but it
updated the bits with a compare-and-swap loop. Take &mut self and write
them directly.
merged_resolve_members visited every file on one thread. Fold the
members of each file in parallel and append the per-thread lists in
file order. About 5 ms on chrome-release-arm64.
gather reserved each shard for every recorded occurrence of a name, about
three times the number of distinct names, so the sparse table touched far
more pages than it used. Estimate the distinct names with HyperLogLog
instead. chrome-debug-arm64 peak RSS 26.76 -> 25.91 GB.
Pattern::matches called memchr::memmem::find for the literal after each
star, which builds a searcher on every call. Build them when compiling
the pattern instead. apply_version_script on tensorflow-debug 260 -> 220
ms.
compute_import_export collected updates in parallel and applied them on
one thread because Symbol's attribute bits needed &mut. Make the bits an
AtomicU16 so that file-parallel passes can set them through shared
references, testing a bit before writing it as mark() does.
parse_symbol_version now finds versioned symbols in parallel; it still
applies them in input order because one may read a version that another
has just set. compute_import_export on tensorflow-debug 167 -> 9 ms.
Assemblers emit a call to a static function in another section as a
branch to a section symbol plus an offset, but a range extension thunk
jumps to the start of a symbol. On RELA targets, give such a branch a
symbol at its destination and a zero addend, as we do for references to
section fragments. This fixes Chromium's ARM64 debug build, which
9b9162146 made an error and 2.42.1 linked into calls to wrong addresses.
mold needs Rust 1.95 for std::hint::cold_path and the atomic update
methods, while its dependencies need only 1.85. Declaring it makes Cargo
reject older toolchains with a clear message before building, and lets
the resolver avoid dependency versions that need a newer rustc.
This reverts 6f8dbb64b. mimalloc uses 64-bit atomics, which GCC
implements as calls into libatomic on hosts such as 32-bit PowerPC.
The need comes from linking mimalloc statically into mold, so the
executable is the right place to add libatomic. Our mimalloc_rust fork
is pinned to the commit before its libatomic change, so the fork
differs from upstream only by the bundled mimalloc version.
This reverts bd97ebc90. mimalloc's page map still doesn't cover the
sign-extended high half of the sparc64 address space, but mimalloc maps
its arenas at hinted addresses in the low half, and since v3.5.0 a page
outside the map fails to register instead of corrupting memory. On an
UltraSPARC T5 running Linux, mold with mimalloc 3.5.3 passes the native
test suite.
mimalloc_rust now links libatomic itself on targets without 64-bit
atomics, so cli/build.rs, which did that for 32-bit PowerPC hosts, is
no longer needed.
readelf before binutils 2.39 can't decode version 9 indices and exits
with an error, so gdb-index-dwarf5-types ignores its exit status as
check_index already does. arch-x86_64-eh-frame-hdr-order links without
the C runtime, whose objects may have FDEs of their own.
mold handles .ARM.exidx and other users of SHF_LINK_ORDER directly
rather than implementing the flag in general. This reverts 49129fbd5,
which kept SHF_LINK_ORDER sections alive with their linked sections, and
restores the ARM-specific GC edge to .ARM.exidx. 9aacb62c0 now sets
sh_link only in relocatable output, which needs it to preserve the flag.
GCC emits no pubnames for type units, so the index listed a type only
for the compilation units referring to it by signature. GDB 18 doesn't
follow such a reference when it looks a type up through the index, so
ptype failed for types defined in type units. Read the name of the type
each type unit defines from its DIEs, resolving their section offsets
with input relocations, and add it to the index for the type unit.
A non-PIC object referring to an absolute or undefined weak symbol with
a PC-relative relocation, such as AArch64's ADRP, can't be linked into
a position-independent output, but recompiling it with -fPIC fixes that
by going through the GOT. The error message lost that hint when it was
reworded, which broke abs-error on aarch64_be.
od -w is a GNU extension, so arch-x86_64-eh-frame-hdr-order failed on
FreeBSD. head and grep -q exit before reading all of their input, so
under pipefail the producer may die of SIGPIPE and fail the pipeline,
which happened to mergeable-entsize-order on FreeBSD.
Clang gives the STT_TLS type to the labels of TLSDESC code sequences,
which are in .text. With --emit-relocs they are written to .symtab,
where their values were computed as offsets from the start of the TLS
segment, which underflowed. Write the address of an STT_TLS symbol
outside a TLS section as it is.
Assemblers emit a call to a static function as a relocation against a
section symbol plus an offset. A range extension thunk jumps to the
start of a symbol, so taking such a branch through a thunk landed at
the wrong address. Report it as an error instead.
Relocations in a relocatable output may refer to local symbols such as
RISC-V's .L labels, so removing them with -x, -X or -s left relocations
pointing to symbol 0. -r now ignores -x and -X, and -s strips only debug
info as -S does.
In the extreme code model, the pcalau12i with R_LARCH_TLS_GD_PC_HI20 or
R_LARCH_TLS_LD_PC_HI20 begins a 64-bit sequence whose lu32i.d and lu52i.d
supply the bits beyond ±2 GiB, so the GOT may be farther away than that.
The other *_PC_HI20 relocations were already applied without a range
check, as in GNU ld and lld, but these two rejected such a GOT with an
out-of-range error.
In the extreme code model, lu32i.d and lu52i.d follow pcalau12i by 8 and
12 bytes, and the upper 32 bits they set are relative to the pcalau12i's
page. We used page(P - 8) for both, which is the next page for lu52i.d if
the pcalau12i is the last word of a page. The result was wrong whenever
the borrow reached bit 52; e.g. a pcalau12i at 0x100000ffc materialized
0xfff0000080000000 instead of 0x80000000.
mold crashed at startup on sparc64 Linux, which maps user memory at
sign-extended high addresses. mimalloc v3 registers pages in a map that
covers only 48-bit addresses, so the first registration wrote out of
bounds. Until the bundled mimalloc handles that layout, sparc64 hosts use
the system allocator (https://github.com/rui314/mold/issues/1639).
mold -run looked for mold-wrapper.so in /usr/local/lib/mold unless it was
next to the executable or in ../lib/mold, and install-mold.sh always put
it in $PREFIX/lib/mold, so a package using lib64 or a multiarch libdir
could not make -run find it. The C++ version took the directory from
CMake's libdir. MOLD_LIBDIR now sets it at build time and install time.
util::prefetch assumed SSE on all x86 targets, but i586-unknown-linux-gnu
has none, so mold died with SIGILL on CPUs such as the Pentium, K6,
Vortex86 and Quark. The x86 prefetch is now compiled only when the target
has SSE. Such a target simply ignores the hint, as other targets without
a prefetch instruction do.
Versions of a symbol have the same name, so they tied in the unstable sort
of exported dynamic symbols. Rayon breaks ties with a pseudo-random
generator whose width follows usize, so .dynsym and its hash and version
tables differed between 32-bit and 64-bit hosts. The input order now breaks
the tie. .eh_frame_hdr had the same problem with FDEs for one function,
which are now ordered by their addresses.
On a 32-bit host, linking an output of 2 GiB or more panicked, because
the doubled mapping length wrapped around and sizes of 4 GiB or more were
truncated to usize. Each panic also left a multi-GiB temporary file. We
now check the size before creating the file and report a fatal error.
The limit is 2 GiB, as no Rust object can be larger than isize::MAX bytes.
A range extension thunk jumps to its destination with `br x16`, which
faults under BTI if the destination doesn't begin with a landing pad, as
is the case for a static function that is only called directly. For such
a function, we now put `bti c; b <function>` at the beginning of the first
thunk after the function's section, which is always within direct branch
range, and other thunks jump there instead, as lld's landing pad thunks do.
-z force-bti and -z pac-plt force the BTI and PAC bits of the output's
GNU_PROPERTY_AARCH64_FEATURE_1_AND, and -z pac-plt also makes PLT entries
authenticate .got.plt entries with autia1716 and sets DT_AARCH64_PAC_PLT.
-z gcs=implicit|never|always controls the GCS bit. -z bti-report and
-z gcs-report diagnose input files lacking the features, and as in lld,
-z force-bti and -z gcs=always imply the warning level.
We used to merge only x86 properties in .note.gnu.property, so an ARM64
output lost GNU_PROPERTY_AARCH64_FEATURE_1_AND even if all its inputs were
compiled with -mbranch-protection. We now AND-merge it as GNU ld and lld do.
If the output is BTI-enabled, the PLT header and the PLT entries that may be
reached by an indirect branch start with `bti c` as lld's do, which makes PLT
entries 24 bytes long, and DT_AARCH64_BTI_PLT is set.
R_RISCV_ALIGN has no symbol, and its addend is the size of the NOP
padding that a relaxing link shrinks to realign the code after it.
symidx_addend returned (0, 0) for any symbol not written to the output
symbol table, including the null symbol, so `mold -r` wrote R_RISCV_ALIGN
with addend 0 and a later link could no longer realign the code. We now
keep the addend of a relocation without a symbol, as GNU ld and lld do.
LoongArch's R_LARCH_ALIGN refers to a local symbol and was not affected.
GCC 14 uses R_LARCH_TLS_DESC64_PC_LO20 and R_LARCH_TLS_DESC64_PC_HI12
with -mcmodel=extreme -mtls-dialect=desc to extend the TLSDESC sequence's
GOT access to 64 bits, and we rejected them as unknown. We now apply them
like the other 64-bit PC-relative pairs and don't rewrite the sequence's
LO12 instruction with pcaddi. As GNU ld and lld do, we don't relax the
sequence to IE or LE, except in static executables, which can't use
TLSDESC and therefore get LE.
is_got_load_pair checks that a GOT load's PCREL_LO12_I refers to the
label at its AUIPC by comparing the label's value with the AUIPC's
r_offset. Symbol values are adjusted for removed bytes before relocations
are applied, but r_offsets are not, so a GOT load after a shrunk
instruction was no longer rewritten to AUIPC+ADDI. We now compare with
the label's st_value, as find_paired_reloc does.
R_RISCV_32_PCREL, R_RISCV_PLT32 and R_RISCV_GOT32_PCREL in allocated
sections were truncated to 32 bits without a range check, so a reference
to a symbol or a GOT slot more than 2 GiB away silently got a wrong
value. We now report an out-of-range error for them, as we already do
for R_RISCV_32_PCREL in .eh_frame and as lld does.
The .riscv.attributes parser knew the value types of only the three tags
mold uses, so the value of any other tag was read as a tag. For example,
an unknown tag 40 with value 5 was mistaken for Tag_RISCV_arch. As per
the psABI, an even tag has a ULEB128 value and an odd tag has a
NUL-terminated string, so we now skip other attributes by that rule.
They are still dropped from the output, as lld does for unknown tags.
The output's e_flags were the first input's plus EF_RISCV_RVC, so an
object compiled for the TSO memory model (-march=rv64gc_ztso) lost its
EF_RISCV_TSO bit if it came after an ordinary RVWMO object. RVWMO code
runs correctly on TSO hardware but not vice versa, so the output now has
the bit if any input has it. GNU ld does the same; lld still takes the
bit from the first file.
-Map wrote address zero as "0x0" where C++ mold wrote "0", and
--print-dependencies omitted the blank line that separated the comment
header from the dependency lines. Both outputs are now byte-identical
to those of C++ mold.
With MOLD_JOBS=1 and an empty XDG_RUNTIME_DIR, mold joined "" with
"mold-lock" and created the lock file in the current directory, so
processes in different directories did not serialize. Like other XDG
variables, an empty value now means unset, and mold falls back to
/tmp/mold-lock-$USER.
-rpath-link, -fuse-ld, -O, --sort-section and --max-cache-size are
accepted only for compatibility, and their values are never used. mold
nevertheless required the values to be UTF-8, so a non-UTF-8 directory
given to -rpath-link was a fatal error. Read the values as raw bytes.
The manual said -z separate-code was the default, but mold defaults to
-z noseparate-code and adds no padding between segments unless
--section-order is given, which implies -z separate-loadable-segments.
The manual documents --no-color-diagnostics, but mold rejected it as an
unknown option; accept it as lld does. Conversely, the manual and --help
listed -z nodefs and -z start_stop_visibility, which neither mold nor
GNU ld accepts; the options are -z undefs and -z start-stop-visibility.
--help also said --color-diagnostics means always, while it means auto.
--dynamic-list-data, --lto-pseudo-probe-for-profiling and a bare
--thinlto-index-only take no argument in GNU ld and lld, but mold read
the next command line argument as their value, silently dropping an
input file placed there. GNU ld's --package-metadata takes an optional
argument, so mold now accepts its value only after an equal sign.
Rust's io::Error appends " (os error N)" to the strerror text when it is
displayed, so mold printed "cannot open foo.o: No such file or directory
(os error 2)" where the C++ version printed just the text. Diagnostics
now format I/O errors with a strerror helper that returns the text alone
and falls back to the usual formatting for errors not from the OS.
scan_pcrel, which also handles GOT-relative relocations, told the user
to recompile with -fPIC whenever it couldn't resolve a relocation, even
though code using GOTOFF is PIC already. It now reports the reason: a
symbol at a fixed address, such as an absolute or undefined weak one, in
a position-independent output, or a preemptible symbol defined in the
shared object being created, which -Bsymbolic or hidden visibility fixes.
`-e 0x401000` was looked up only as a symbol name, so the link warned
that the entry symbol was undefined and set e_entry to 0. GNU ld and lld
fall back to parsing the argument as a number (decimal, 0x hex or octal)
when no symbol has that name, and so does mold now.
When a version script named the same symbol in two version nodes, each
exact-name match overwrote the previous one, so the last node won.
GNU ld and lld give the symbol the first node's version (lld warns about
the reassignment, and GNU ld rejects a name that is both local and
global). Apply only the first exact match. Wildcards still let the last
matching node win, as in both linkers.
A version script or --default-symver on a link that exports no dynamic
symbols, such as a static executable, made verdef::construct index the
empty .dynsym and panic. Symbol versions exist only for dynamic symbols,
so write no version sections in that case, as GNU ld does.
Undefined symbols are allowed by default with -shared or with
--unresolved-symbols=ignore-all, and a non-weak hidden one was then
silently resolved to 0. A hidden symbol can't be resolved at runtime,
so report it as an error regardless of those options, as GNU ld and lld
do. Weak undefined hidden symbols still resolve to 0.
--defsym=foo=bar with bar defined by a shared library copied bar's
library-relative value into an absolute foo, so calling foo jumped to a
bogus address. Such a symbol has no link-time address, so report it the
same way as an undefined one. GNU ld and lld reject it as well.
The first file on the command line owned a common symbol, so `int buf[4]`
in one file and `int buf[64]` in another gave a 16-byte buf, and writes
to it clobbered its neighbours. As GNU ld and lld do, let the largest
common own the symbol and align it to the largest alignment among them,
for .common, .tls_common and -r output alike.
--export-dynamic and --dynamic-list export symbols even from an
executable that links against no DSO. Such an executable has no
.dynamic, but the symbols were still added to .dynsym, and .dynstr
lacked its leading empty string, which only the creation of .dynamic
adds, so entry 0 of .dynsym got a symbol's name. Like GNU ld and lld,
emit no dynamic symbol table without .dynamic.
Mergeable sections that share a name, type and flags but differ in
entry size are merged into separate output sections. Their sort keys
tied, so their order depended on which one the parallel conversion of
input files happened to create first. Break the tie with the first
member's file priority and section index, as for output sections.
add_synthetic_symbols defined __start_ and __stop_ symbols for every
allocated chunk with a C identifier name, including the ELF and program
headers named EHDR and PHDR. fix_synthetic_symbols skips headers, so
the symbols were left in .symtab with value 0, and -z
start-stop-visibility=protected exported them. Exclude headers in
start_stop_name, which both functions use.
compute_address_significance marked exported symbols' sections as
address-taken by walking .dynsym, but .dynsym is not populated until
after ICF, so the loop did nothing and --icf=safe gave identical
exported functions of a DSO the same address. Walk each object file's
defined global symbols instead, whose is_exported bits are already set.
DT_INIT and DT_FINI refer to the --init and --fini functions, but only
the entry point was a GC root. If such a function was not exported
because it was hidden or made local by a version script, --gc-sections
discarded its section, DT_INIT and DT_FINI were emitted with value 0,
and the DSO crashed on load. Make them GC roots as GNU ld and lld do.
The $a, $t and $d symbols that mold creates for PLT entries and thunks
were STT_FUNC, copied from the symbols naming the entries, while AAELF
requires mapping symbols to be STT_NOTYPE. push_mapping_symbol now
builds the symbol from a section index and an address itself.
Only a final link converts big-endian ARM code to BE8, so relocatable
output keeps the input's BE32 code, but it was marked EF_ARM_BE8 anyway.
GNU ld rejected such an object as "already in final BE8 format", and
disassemblers decoded its code in the wrong byte order. -r also failed
with "--be32 is not supported" unless --be8 was given, which GCC and
Clang never pass with -r.
A 32-bit Thumb instruction is a pair of halfwords, each in the byte
order of the input's code, but NOP.W was written as a single 32-bit
word. On big-endian targets, that swapped its halfwords, so a Thumb call
to an undefined weak symbol and a TLSDESC call relaxed to local-exec
became garbage in BE8 output. NOP.W is now written a halfword at a time.
Clang assembles ADR and LDR (literal) referring to a symbol in another
section to R_ARM_ALU_PC_G0 and R_ARM_LDR_PC_G0 in ARM code and to
R_ARM_THM_ALU_PREL_11_0 and R_ARM_THM_PC12 in Thumb code, which failed
with "unknown relocation", as did 8- and 16-bit data. The offset's sign
picks ADD or SUB, or the U bit of LDR, and the Thumb forms are relative
to the PC aligned down to 4, producing the same bytes as GNU ld.
TBZ/TBNZ to a label in another section, .2byte/.4byte data referring to
a symbol, and GOT loads compiled with -mcmodel=tiny -fPIC failed with
"unknown relocation". TSTBR14 reaches ±32 KiB and is range-checked like
GNU ld and lld do, without thunks. ABS16 and ABS32 are too narrow for
dynamic relocations, so they must be link-time constants in PIC output.
GOT_LD_PREL19 is an LDR literal that loads the symbol's GOT slot.
Relocations copied by --emit-relocs refer to symbols in .symtab, which
--strip-all omits, so the combination crashed or produced relocation
sections pointing to a nonexistent symbol table. Report an error as
lld does. (GNU ld instead keeps the symbols the relocations need.) -r
sets --emit-relocs internally and keeps accepting --strip-all.
An SHF_LINK_ORDER section's sh_link refers to the section it describes,
but mold always wrote zero. Once passed through -r, such sections lost
their links, so relinking with --gc-sections dropped .ARM.exidx and
__patchable_function_entries, and C++ programs on ARM aborted when
throwing. Set sh_link from the members' linked-to sections. Since it
can name only one section, -r also separates SHF_LINK_ORDER sections
whose linked-to sections are in different output sections, as GNU ld
and lld do.
An SHF_LINK_ORDER section such as __patchable_function_entries holds
metadata for the section that its sh_link refers to, and usually
nothing else refers to it. --gc-sections therefore discarded all of
them except .ARM.exidx, for which there was an ARM-specific edge from
a section to its unwind index. Replace that edge with a generic one
from a section to the SHF_LINK_ORDER sections linked to it, as GNU ld
and lld do.
Allocated fragments of mergeable sections start out dead under
--gc-sections and are revived by the garbage collector, but mold -r
returns before running GC. As a result, `-r --gc-sections` discarded
every .rodata.str* and .rodata.cst* piece and left the relocations
referring to them with symbol index 0. Since mold doesn't collect
garbage for -r, ignore the option there. GNU ld and lld do collect
garbage with -r, taking roots only from -e and -u.
R_SPARC_WDISP16 is used by branch-on-register instructions such as brz,
whose 16-bit word displacement is split into d16hi at bits 21:20 and
d16lo at bits 13:0. We ORed the fields into the upper halfword of the
instruction instead, corrupting its opcode, and limited the displacement
to 64 KiB instead of 128 KiB. Place the fields as GNU ld does.
R_SPARC_OLO10 carries a second addend in the upper 24 bits of r_info,
which is added after the low 10 bits of S + A are taken. RelocSection
built its output records with ElfRel::new(), which zeroes that field, so
`ld [%g1 + %lo(var) + 8]` lost its +8 in -r and --emit-relocs output.
Build each record from a copy of the input one instead.
The 8- and 16-bit GOT and TLS relocations fill the displacement of a
(d8,An,Xn) or (d16,An) operand, which the CPU sign-extends, but we checked
them against unsigned ranges. A GOT offset of 32 KiB or more was thus
silently encoded as a negative displacement, and a local-exec TLS offset,
which is usually negative because the thread pointer is biased by 0x7000,
was rejected. Check them as signed as GNU ld does.
SH4 object files keep relocation addends in the relocated places even
though the relocations are RELA, and RelocSection already follows that
convention. EhFrameRelocSection, however, put adjusted addends only into
r_addend, which SH4 readers ignore, so after `ld -r` every FDE's pc_begin
pointed at the start of its output .text and C++ exceptions terminated.
Write the addends into .eh_frame as we do for REL targets.
This enables the three -r exception tests that were skipped on SH4.
On SH4, r2 carries the address of the buffer for a struct return value.
Our PLT header loaded the link map into r2 and set r0 to 0, which glibc's
resolver accepts, but the resolver then restores r2 to its entry value, so
a lazily bound function returning a struct overwrote the link map. The
header now passes the link map in r0 as GNU ld does, borrowing r2 only
until the jump's delay slot restores it from the stack.
A section folded by ICF has no output section, so with --emit-relocs a
relocation against its section symbol was emitted with symbol 0, and one
in .rela.eh_frame, such as an LSDA pointer, hit an expect(). Symbols
defined in a folded section are not in .symtab either, so references to
them lost their symbol or, in .rela.eh_frame, got a wrong one. Emit both
relative to the leader's output section, as the relocated contents and lld
do.
When relaxing TLSDESC to initial-exec, we write an AUIPC to the
TLSDESC_ADD_LO12's slot and a load to the TLSDESC_CALL's slot, but we
computed their GOT-relative offsets from the TLSDESC_HI20's address. The
two coincide only if the instructions in between are deleted, which is not
the case for objects lacking R_RISCV_RELAX on TLSDESC relocations, such as
clang 18's, so the load read a wrong GOT entry. Use the AUIPC's address,
as lld does.
With --emit-relocs, a TLSDESC_HI20 relaxed to IE or LE was rewritten to
R_NONE as soon as it was applied. The TLSDESC_LOAD_LO12, TLSDESC_ADD_LO12
and TLSDESC_CALL that follow find it with find_paired_reloc(), which
matches by type, so such links failed with "paired relocation is missing".
Rewrite TLSDESC_HI20s after all relocations have been applied. LoongArch is
unaffected because its TLSDESC relocations refer to the symbol directly.
shrink_sections adjusted each symbol by the r_deltas of its own input
section, but a section folded by ICF is never shrunk, so its symbols kept
their pre-relaxation offsets while being resolved against the shrunk
leader. With LoongArch GCC's leading R_LARCH_ALIGN nop, calls to a folded
function landed 4 bytes past the leader's entry. A folded section was
identical to its leader before relaxation, so use the leader's deltas for
both symbol values and st_size.
GCC calls _savegpr0_N, _restgpr0_N, _savefpr_N and similar functions
from -Os prologues and epilogues and expects the linker to provide them.
We synthesized them only for ELFv2, so ELFv1 programs built with -Os
failed to link with undefined symbols. ELFv1 calls them with plain
R_PPC64_REL24 relocations, not through function descriptors, and
expects the same code, so emit the same section in target byte order
and move its instruction table from ppc64v2.rs to the section's module.
R_PPC64_GOT16, _LO, _HI and _HA take the offset of a GOT slot from the
TOC base, but we subtracted the TOC base from the slot's offset within
.got rather than from its address, so the result was off by the address
of .got. GCC addresses data through .toc instead, so only hand-written
assembly using @got was affected.
With -fpic, PPC32 code loads a GOT slot with one instruction whose
R_PPC_GOT16 relocation is a signed 16-bit offset from the GOT, and
R_PPC_GOT_TLSGD16, R_PPC_GOT_TLSLD16 and R_PPC_GOT_TPREL16 are the
same. We truncated these offsets silently, so a program whose GOT grew
past 32 KiB linked but loaded the wrong slots. Report an error instead,
as the psABI requires overflow checks for these fields.
A static executable's startup code applies the R_IRELATIVE relocations
between __rela_iplt_start and __rela_iplt_end, and we placed the start
by counting only the GOT slots of IFUNC symbols. On PPC64, a data
reference to an IFUNC symbol gets its own R_IRELATIVE instead of the
canonical PLT address, so the range began too late and skipped a GOT
slot, and a call through that slot ran the resolver. Count them too.
A global symbol defined only in a member of a discarded COMDAT group has
no owning file, so it has nothing to resolve to. The relocation scan
reports a reference to it and skips the relocation, but --noinhibit-exec
downgraded the error to a warning and the relocation was applied anyway,
which panicked on a missing GOT or TLS entry on most targets. Report the
reference as a fatal error in that case.
The relocation scan skips a relocation to an undefined symbol when that
is an error, because the link fails anyway. --noinhibit-exec reports the
error as a warning and writes the output, so relocations that needed a
GOT, PLT or TLS entry were applied without one, and mold panicked on the
missing index. Such relocations are now scanned as they are with
--warn-unresolved-symbols, so the symbol gets the entries it needs and
resolves to 0, as it does in GNU ld and lld.
Every target crashed on a -fPIC object reading an undefined variable,
an undefined TLS variable, or both.
GNU as emits R_RISCV_ALIGN and R_LARCH_ALIGN for an alignment directive
in an executable NOBITS section. With -r, RelocSection sliced the output
section's bytes at each relocation before calling write_addend, but a
NOBITS section has no bytes in the file, so the slice was out of range.
Since only REL targets and SH4 keep addends in section contents, take
the slice only on those targets.
GAS and Clang emit R_RISCV_64 and R_LARCH_64 for `.quad sym` even in
ELF32 objects. On RV32 and LA32, they are not the word-size absolute
relocation that the output section applies, so they reached
apply_reloc_alloc, which had no case for them and panicked. Handle them
like other non-word-size absolute relocations, as lld does: scan them
with scan_absrel and write S + A as a 64-bit value.
With a plugin that lacks get_symbols_v3, mold re-executes itself with
the same command line. The new process applies -C again, but it was
started in the directory the first one had already changed to, so a
relative -C failed with "chdir failed". Remember the directory before
-C and return to it before re-executing.
Input file names already have --chroot applied. Relative thin archive
members and relative names in linker scripts are resolved against such
a name, but the result was opened with --chroot applied again, so the
link failed. Open these paths as they are. Also write the --repro
archive next to the output in the chroot rather than outside of it.
--repro reopened each input by its recorded name. Response files and
--retain-symbols-file are opened before -C changes the directory, so a
relative name no longer found them and the link failed, and a name with
--chroot applied got the prefix twice. Archive the contents already in
memory instead: no input has been rewritten in place at this point.
An existing output file is reused by writing it in place. When that file
is also a memory-mapped input, as in `mold a.o -o a.o` with an a.o larger
than 32 KiB, its private mapping sees the truncation and the writes while
the input is still being read, so mold crashed with SIGBUS or produced a
broken output. Such a file is now replaced with a fresh one. Inputs are
matched by device and inode, which covers archives and thin members too.
An absolute path in a linker script is prefixed with the sysroot if
the script is in the sysroot. is_in_sysroot returned false for an
empty sysroot, so such a path fell through to the -L search, and an
incompatible file was skipped with a warning. C++ mold treats an empty
sysroot as containing every file and opens the path as is, reporting
an error if it cannot be used. GNU ld and lld don't search -L either.
--dynamic-list and --export-dynamic-symbol-list files were opened by
the driver after parse_args had already changed the directory for -C,
so a relative path was looked up in the new directory. C++ mold reads
them in the option loop, as we already do for --retain-symbols-file.
Open them there too and keep the mapped files in DynamicListSource;
the driver still parses them, since the parser needs a Context.
A panic aborts the process in release builds, skipping the cleanup that
a fatal error or a SIGSEGV/SIGBUS does, so a failed link left its
.<output>.<pid> file behind. Install a panic hook that removes the file
and then calls the default hook, so the panic message, backtrace and
abort stay as they were. The hook runs on the panicking thread, so it
also covers rayon workers and the extern "C" LTO plugin callbacks.
Arguments are passed on as C strings, so a NUL byte in the -plugin path,
a -plugin-opt or the -o name made loading the LTO plugin panic. A
response file is the only way to get a NUL byte into an argument, as
the OS passes arguments as C strings too, so reject one when reading a
response file instead of at each C string conversion.
mimalloc uses 64-bit atomics, which GCC implements as calls into
libatomic on hosts such as 32-bit PowerPC. The mimalloc crate links
libatomic only on ARM, so mold failed to link there, which CI's cargo
check did not catch. Link libatomic when the target lacks 64-bit
atomics, as the C++ version did. It is a link argument rather than a
library because rustc would place the latter before mimalloc, where
--as-needed drops it.
libz-sys declares the length parameter of adler32_combine() as
libc::off_t, which is 32 bits on 32-bit glibc hosts, but Debian and
Ubuntu now compile C code for armhf and powerpc with 64-bit off_t, so
the bundled zlib reads a 64-bit length there. Every zlib-compressed
debug section larger than one 1 MiB shard got a wrong checksum, and
readelf, objcopy and gdb rejected it. Compute the combination in Rust
instead; it is a few lines of modular arithmetic.
apply_reloc_alloc read the 4-byte instruction at every relocation to
extract its register fields, so a 1- or 2-byte data relocation such as
R_SPARC_8 or R_SPARC_DISP16 in the last bytes of a section panicked.
Read the fields only for the relocations that rewrite an instruction.
i386 aligns 64-bit integers to 4 bytes, so Symbol and NameRecord are
smaller there than on other hosts, and the assertions that they are
exactly 48 and 16 bytes failed to compile. Assert upper bounds instead,
which still catch growth on 64-bit hosts, and check the i686 build in
CI.
std has no AtomicU64 or AtomicI64 on hosts such as 32-bit PowerPC,
32-bit MIPS and pre-v6 ARM, so mold did not compile there, whereas the
C++ version fell back to libatomic. Take those two types from
portable-atomic, which uses the std types where they exist and falls
back to locks elsewhere. Also check the 32-bit PowerPC build in CI; the
existing armv7 check did not catch this because armv7 has 64-bit
atomics.
See https://github.com/rui314/mold/discussions/1643#discussioncomment-18603572.
The .tbss layout loop gained a second end-of-list check next to the
existing one, for a case that cannot happen: .tbss is never the last
chunk. DigestMap::new was rewritten with repeat_with, which took five
lines to say what one line did. Restore both.
Follow-up to https://github.com/rui314/mold/pull/1685.
FileRecord::size was size_of::<Self>(), and record sizes were written
both ways. Use size_of everywhere. Unlike the trait method, it is a
const fn, so it also works in the compile-time assertions.
https://github.com/rui314/mold/pull/1683 unified the spelling in the
other direction.
An ignored SIGCHLD is inherited across exec, e.g. from `trap '' CHLD`
in a shell. The child that fork_child creates is then reaped
automatically, so the parent's waitpid fails with ECHILD. The parent
ignored that and read the untouched zero status as a successful exit,
so a link that failed exited with status 0. Report a failure instead.
Also read and write the completion byte through File, which removes
two raw libc calls.
Based on https://github.com/rui314/mold/pull/1682.
Both callers of from_hex already check that the byte is a hex digit,
so the new assertion was redundant. The utsname rewrite traded
MaybeUninit::assume_init for mem::zeroed without removing an unsafe
block, so restore the original code.
The fix it tests is target-independent, and only its last check was
x86-64-specific. Count the dynamic relocations against foo and bar
instead of looking for R_X86_64_64. PowerPC is skipped because GCC puts
the read-only pointers in writable sections there, so nothing is
promoted.
LLVMgold calls get_symbols with a null pointer and a zero count for an
IR file that has no symbols, e.g. one compiled from an empty source
file. slice::from_raw_parts does not accept a null pointer even for a
zero length, so the debug build aborted on its precondition check.
Use an empty slice for a zero count, in add_symbols as well.
Based on https://github.com/rui314/mold/pull/1680.
Building the thread pool fails when threads cannot be created, e.g.
under a low `ulimit -u`. That hit an expect, so mold aborted with a
panic message rather than reporting the error like other system
errors.
Taken from https://github.com/rui314/mold/pull/1678, which also
added checks for malformed inputs that are not included here.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Input files are opened in parallel, and each file is appended to the
file pool when it is opened, so the pool's order changed from run to
run. --dependency-file and --repro list files in that order. Return
the pool sorted by name instead.
The dependency-file tests expected a space after a.o, i.e. that a.o
was not the last dependency. That failed intermittently on Alpine,
where libc.so is not a linker script and no dependency reliably comes
after a.o.
report_out_of_range looked the record up again through the file's
relocation table while apply_reloc_alloc held that table mutably, which
created a shared view of memory behind a live unique reference, and it
walked the CREL iterator from the start to do so. Every caller has the
record at hand, so pass it in. The loop indices that only served the
lookup are gone with it.
Cherry-picked from https://github.com/rui314/mold/pull/1674.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
CieRecord stored a shared slice over its .eh_frame relocation records,
but reattach_fragment_relocations later rewrites those records in place
through the private input mapping, and the FDE and CIE accessors read
through the stale slice afterwards. The record now keeps only the
relocation section index and reads the current table through
ObjectFile::relocations, as decoded CREL tables already did. The raw
relocation view no longer claims a 'static lifetime.
Cherry-picked from https://github.com/rui314/mold/pull/1674.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
mold accepts a multi-letter option with a single dash, but the option
chain is first-match, and some single-letter options were tried before
long options starting with the same letter. "-entry=main" was thus read
as "-e ntry=main", "-filter libf.so" as "-f ilter" followed by an input
file, and "-fix-cortex-a53-843419" left a bogus DT_AUXILIARY entry. GNU
ld and lld read all of these as the long option.
Move each single-letter option past the last long option it shadows.
Conversely, GNU ld and lld read "-execute-only" as "-e xecute-only" and
"-omagic" as "-o magic", because those names take double dashes only.
Spell such names with double dashes in the option table, which
match_option now honors, replacing the rule that all names starting
with "o" need two dashes. Where GNU ld and lld disagree, follow GNU ld.
Thanks to @lu-zero for investigating this issue and proposing a fix in
https://github.com/rui314/mold/pull/1670.
Fixes https://github.com/rui314/mold/issues/1671
Zip the glob NFA's three parallel arrays and fill a bracket range with
one slice fill, merge the two glob module headers left over from two C++
files, check SipHash's output width at compile time and split its input
into words with as_chunks, mark Timer must_use so a dropped one is a
compile-time warning, remove a Windows TODO from the C++ demangler that
never applied to this implementation, and fix a typo in a comment.
The merged glob header no longer lists qt_private_api_tag* as an
example of an Aho-Corasick pattern, since prefix patterns are matched
directly rather than by the Aho-Corasick matcher.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Each output section promoted the imported symbols of its read-only
absolute relocations and classified its relocations in one pass, while
other sections ran in parallel. Whether a writable section's relocation
against the same symbol became a dynamic relocation therefore depended
on which section ran first, so the output changed from run to run. In a
non-PIE that references an imported variable and function from both
.rodata and .data, 37 of 100 links in one measurement emitted two extra
R_X86_64_64 dynamic relocations.
Collect all sections' relocations, promote, and only then classify, so
that the result is the same in every run: the word is resolved at link
time whenever a copy relocation or canonical PLT entry makes its address
known.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The --undefined names were cloned only because get_symbol takes the
whole context mutably, so intern them through the symbol table instead.
The section-word reversal swaps whole words, the .tbss layout loop no
longer indexes past the last chunk, the has_lto_obj predicate replaces
an inline copy of itself, and the --exclude-libs ALL test is hoisted out
of the file loop. The dynamic symbol groups are extended in place, the
IRELATIVE count loses a temporary, and thunk addresses zip symbols with
their offsets.
In ICF, the digest no longer converts to little-endian on the way into
a hash whose key is random per run, the digest map is filled with
repeat_with, and HashMap is imported once.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
apply_reloc_nonalloc on ARM32, PPC32, m68k and SH-4 read the relocation
section of a non-allocated section as an array of ordinary records. A
CREL table of a non-allocated section is not decoded ahead of time, so
for an object with SHT_CREL relocations those bytes are compressed
records: linking a PPC32 or m68k object that Clang 19 built with -g and
-Wa,--crel panicked with a failed length assertion in
records_from_bytes. Use the relocation iterator, which decodes CREL
tables, as the other targets already do.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
A relocatable link keeps every COMDAT group's members in output
sections of their own, so an object with many groups of one name, such
as the .debug_macro groups GCC emits with -g3, produces output sections
whose sort keys tie. The tie fell back to the order in which the
parallel creation phase happened to make them, which changed from run
to run: 30 runs of the link in relocatable-compressed-debug-info.sh
produced 13 different outputs. Break the tie with the first member's
file priority and section index, which are in input order.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
C++ mold adds the _savegpr0_* family after the __start_ and __stop_
symbols, and the internal object's symbol order is the order in the
output .symtab. Match it so that ppc64le outputs get the same .symtab,
and hence the same build ID, as C++ mold's.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
C++ mold defines these target-specific segments after PT_GNU_RELRO, but
the port defined them before PT_GNU_STACK. ARM and RISC-V outputs
therefore had their program headers in a different order than C++
mold's for the same input, and a different build ID. Define them where
C++ mold does.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The four places that write a target word through an IS_64 branch
(.got, .got.plt, .relr.dyn and an output section's absolute
relocations) now use the existing Word::<E>::new(value).write(slot).
The padding filler walks its buffer with chunks_exact_mut; thunk
symbols zip their two parallel vectors; relr_offsets concatenates its
shards once; the .ARM.exidx lookup uses find_map instead of position
plus a second match, and text_end borrows the section headers instead
of copying them; .gnu.hash looks each symbol's hash up once;
merged-section resolution takes the timers by destructuring instead of
cloning them; OutputSectionId::new folds its two checks into one; the
.sframe writer slices each FRE block by start and length; WORD_SIZE is
derived from the word type like IS_64; and ElfDyn derives what its
siblings derive.
The doc comment on ChunkHeader's local-symbol fields no longer starts
with a stray "For --gdb-index", and a few comment typos are fixed.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
shndx_at_in was identical to shndx_at, is_discarded_comdat re-implemented
is_discarded_comdat_sym, ObjectFile's Display duplicated display_file,
rels_mut looked its table up twice, find_null re-implemented a chunked
position search, the archive member name reader re-implemented cstr_at
and memchr, the file type probe indexed the header table it was
iterating, and the linker script reader spelled slice patterns as length
checks plus indexing.
The loop in is_dwarf32 now compares the end of the next unit header with
the section size instead of subtracting its start from the size, which
wrapped around when a unit's size field pointed past the end of a
malformed section.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The command-line and result types of the per-target link functions were
spelled out in the twenty target crates, the executable and the driver.
Name them Cmdline and LinkResult in the driver and use the names
everywhere else.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The record views in elf.rs relied on a debug assertion, and therefore on
the FileRecord contract alone in release builds, for the alignment that
their pointer casts require. An inline const assertion is evaluated per
instantiation, so a record type that violates the contract no longer
compiles.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Several comments compared the code to C++ mold's arena storage and
chunks, or named functions that do not exist in this implementation
(SharedFile<E>::mark_live_objects, Symbol<E>::get_addr and
open_file_impl); point them at the Rust functions or drop the
comparison. The ICF and thunks modules repeated their description in a
second module-level summary, and the TLS module docs had a stray title
line. drop_mappings did not say that it discards the relocation records
rewritten in place, and the madvise in write_build_id did not say why
dropping written pages of the output loses nothing. Also fix three
typos.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The page-aligned start of the interior range was turned back into a
pointer with an integer cast, which leaves it without the allocation's
provenance. Derive it from the caller's pointer with with_addr instead,
as the rest of the function already reads addresses with addr().
Co-Authored-By: Claude Opus 5.5 <[email protected]>
record_comdat_owner and set_visibility_bits hand-rolled fetch_min and
update. record_comdat_owner keeps its initial load so that a file that
does not lower the priority still leaves the cache line shared. The
packed visibility and flags bytes are no longer pub, since nothing
outside symbol.rs touches them and a direct store would corrupt the
state bits they share.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
x86-64 and i386 defined their own little-endian write helpers with the
same bodies as write_ul16, write_ul32 and write_ul64 in util::endian,
and i386 read REL addends with inline from_le_bytes calls. Call the
shared functions directly, as the other fixed-endian targets do. The
ARM64 and RISC-V instruction accessors now delegate to them, like
LoongArch's already do.
Also drop an unused parameter of relax_tlsdesc_to_ie, a redundant mask
on SPARC relocation types (the record keeps the type in a byte of its
own), two TRAP comments that described C++ target structs instead of
the instruction, and the pub on two PPC64 ELFv1 functions that only
their module uses. SH-4 reports an unknown relocation with error! like
every other target, so that all of them are listed in one run.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The key length was an UnsafeCell whose reads and write needed unsafe
blocks justified by the release/acquire pair of the key pointer. A
relaxed atomic expresses the same protocol, compiles to the same loads
and stores, and takes the probe comparisons out of unsafe code. Also
check the probe invariant at compile time and reject keys longer than
32 bits instead of truncating their lengths.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
driver is a public module, so `pub` made copy_chunks part of the
library's API although only the crate itself calls it; make it
pub(crate). from_hex now asserts its precondition in debug builds
instead of silently producing garbage for a non-hex byte.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Six flags in the ignored-for-compatibility chain were unreachable
because earlier arms already consume them: color-diagnostics,
eh-frame-hdr, fatal-warnings, enable-new-dtags, disable-new-dtags and
warn-once.
returns_etxtbsy zeroed a MaybeUninit<utsname> and then asserted it
initialized after calling uname. utsname consists of byte arrays, so a
zeroed one is already a valid value; start from that and drop the
assume_init step.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The runner starts each script in its own process group, so a script
that read the terminal would stop there until the timeout. Give it
end-of-file instead.
run already returns ExitCode, so report setup errors through it
instead of exiting the process. Also drop the second sort of the
discovered scripts (make_jobs sorts the combined list), the unused
Clone on TestJob and the intermediate vector in the PATH search, and
give the unsafe kill a SAFETY comment like the other unsafe blocks.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
R_X86_64_GOTPC32, R_386_GOTPC, R_390_GOTPC and their variants compute
GOT + A - P, the distance to the start of the GOT section rather than
to an entry in it. They were grouped with the GOT-indirect relocations,
so every link that referenced the GOT base got an entry for
_GLOBAL_OFFSET_TABLE_ that nothing ever read, plus a RELATIVE
relocation to fill it in a PIE.
R_ARM_REL32 was in the bucket of relocations that need no scanning,
so a reference to a symbol defined in a DSO was silently resolved as
if the symbol were at address zero. Route it through scan_pcrel like
PREL31 and MOVT_PREL, which gives imported data a copy relocation,
an imported function a canonical PLT entry, and a shared object an
error.
R_X86_64_GOTOFF64 and its counterparts on i386, ARM32, s390x and SH4
compute S + A - GOT, which is a link-time constant only if the symbol
is in the output file. We didn't scan them at all, so a reference to a
symbol defined in a DSO was silently resolved as if the symbol were at
address zero. Route them through scan_pcrel, which already makes the
right decision: a copy relocation for imported data, a canonical PLT
entry for an imported function, and an error in a shared object.
Fixes https://github.com/rui314/mold/issues/1668
The port kept C++ idioms in which a raw pointer or an unchecked index
stood in for a slice, guarded by an assertion. Add SyncUnsafeCell, a
slice-of-cells view for parallel passes, and use it for the file
mapping, the symbol-table allocators, the symbol editor and decoded CREL
tables. Make the fields that parallel passes write atomic, which turns
allocate_aux, the dynsym pass, the gdb_index prefix scan and the COMDAT
signature slots into safe code. Write merged sections and the gdb_index
tables through split slices, and look up files and sections with plain
bounds checks.
This removes 58 unsafe blocks and 16 assertions.
check_duplicate_symbols and check_symbol_types looked every reference
up in the symbol table, a cache miss each in a large link, before checks
on the file's own entry that skip it anyway. An undefined reference is
never a duplicate, and one without a type never mismatches.
Assisted-By: Claude <[email protected]>
R_AARCH64_TLSLE_LDST{8,16,32,64,128}_TPREL_LO12_NC were unhandled, so
objects using them failed to link with "unknown relocation". The immediate
is the local-exec TLS offset scaled by the access size.
addr_with decoded the tagged origin word into its five-variant enum up
front and matched it twice, which left the enum spilled to the stack on
every relocation. It only needs to know whether the origin is a fragment
or an input section, and each is a mask and a compare.
Assisted-By: Claude <[email protected]>
The check read every referenced symbol's ELF entry from its owning file,
a second cache miss per reference, only to get the type the symbol
already caches.
Assisted-By: Claude <[email protected]>
Sizing .rela.dyn walks every GOT symbol, a cache miss each, and the
section is sized three times on the way to the final layout. The walk
was the only serial one of that size left before the copy.
Assisted-By: Claude <[email protected]>
scan_abs_relocations already collects every R_X86_64_64 in an allocated
section, and apply_abs_rels applies it. Resolving the symbol again here
only to fall through an empty match arm is wasted work.
Assisted-By: Claude <[email protected]>
Six of the fifty relocation kinds need G, but every relocation paid for
its aux-table lookup and bounds check. Move the computation into a
closure, as the other targets already do.
Assisted-By: Claude <[email protected]>
A section's address and the GOT base do not change while its relocations
are applied, but the section offset is atomic and the context carries
interior mutability, so the compiler must reload both after every store
into the output buffer. Read them before the loop instead.
Assisted-By: Claude <[email protected]>
InputSection stores its contents as a pointer and sh_size. For a
compressed section, sh_size is the Chdr's uncompressed size while the
pointer led to the compressed bytes, so reading contents() before
uncompress() made a slice far longer than its buffer. Only a debug_assert
guarded this, and a crafted SHF_COMPRESSED .eh_frame reached it through
the .eh_frame parser.
Store a null pointer for compressed and NOBITS sections instead, so that
a nonnull pointer always has sh_size bytes behind it, and pass the
compressed bytes to uncompress() from the file's section header. The gABI
forbids compressing allocated sections, so reject them with an error.
Parallel writes into an output section went through OutputBuffer, a raw
pointer whose ranges were computed by the caller and only debug-asserted.
For ARM32 BE8 byte swapping those ranges came from mapping-symbol values
in the input file, so a corrupted symbol wrote outside of its section.
Replace OutputBuffer with for_each_member, which splits the buffer at
member offsets with rayon::iter::split and split_at_mut and hands each
member its own slice. Thunks get their slices by splitting the buffer
after the member loop, and the ARM32 swap indexes mapping-symbol ranges
within the member's slice. No unsafe code remains in either file.
Word-size absolute relocations were applied after the copy pass through
a raw pointer into the output section, with only a debug_assert on the
offset. A corrupted r_offset therefore wrote past the section in release
builds instead of failing.
Apply them in the per-member copy loop instead, indexing the member's
slice so that an out-of-range r_offset panics. abs_rels is built in
member order, so each member's relocations are one run of the vector;
AbsRel now records the member index and write_to finds the run with
partition_point. This removes write_word and its SAFETY assumption that
input relocations do not overlap.
The 80k-section tests take about 2 seconds alone but 13 to 14 seconds
on every target when the suite runs 128 jobs at once, so any other load
on the machine pushed some of them past 30 seconds. The limit exists to
catch hangs, and 60 seconds still does that promptly.
Layout carried only the file format: byte order, word size and the
record forms that differ between ELF32 and ELF64. No type implemented it
without also implementing Target, so the split earned its keep only in
RISC-V and LoongArch, where the record types depend on a const generic
and could not be chosen inside one Target impl.
Those two targets now take their record types from Class<IS_64>, a
helper private to the target module that maps the const parameter to
the ELF32 or ELF64 forms. ARM32 and SH4 lose their per-byte-order impls
and read section header words through u64::from like the other 32-bit
targets.
What the trait enumerates are targets, not architectures: Ppc64V1 and
Ppc64V2 share an architecture and differ in ABI, and Arm64 and Arm64Be
differ only in byte order. The architecture-level notion already has its
own name, Family. The rest of the tree already said target, in
Arm64Target, TargetTraits, emulation_to_target and the comments, so the
trait was the one place the vocabulary broke. The workspace directory
arch/ keeps its name because target/ at the repository root is Cargo's
build directory.
common.inc sets pipefail, and grep -q exits as soon as it finds a match.
If the producer still has output to write, it dies of SIGPIPE, the
pipeline fails, and a test that gates on the match silently skips or
fails. Plain grep reads its whole input, so the pipeline's status is
grep's own.
Clippy's match_same_arms lint reports 15 pairs. Most sit in relocation
tables that follow the relocation number order, where merging would
break the order, so only the two that are not in such tables change.
Apply clippy's redundant_pub_crate suggestion. An item in a private
module is crate-visible at most, so `pub(crate)` on it says nothing
that `pub` does not.
Apply clippy's ptr_as_ptr, borrow_as_ptr, ref_as_ptr and
ptr_cast_constness suggestions. `ptr.cast()` and `ptr::from_ref()`
cannot silently change the pointee type when a declaration elsewhere
changes, and `&raw mut x` passes a pointer to C without creating an
intermediate reference.
rewrite_opd looks up the symbol of a function descriptor by its offset
with a binary search. Aliases share a descriptor, so the search could
land on any of them, and a section-relative reference to a static
function then went through one of its global or weak aliases: it became
preemptible and resolved to whichever definition won. Use the lower
bound instead; the stable sort keeps the local symbol first.
`dist.sh arm` failed in the networked fetch step: cargo could not read
its git database for the mimalloc dependency ("Value too large for
defined data type"). ext4 hands 64-bit directory offsets to the QEMU
process, the armhf glibc reports EOVERFLOW when they don't fit a 32-bit
off_t, and the libgit2 built into the 32-bit cargo uses that interface.
Only the git dependency is affected; everything else goes through Rust's
std, which uses the 64-bit calls.
Keep cargo's cache on a tmpfs inside the networked container and have it
`cargo vendor --locked` into target/dist-vendor-$arch together with the
printed config. The offline build reads the vendored sources through
`--config` and no longer needs a cargo home at all.
Tests: ./dist.sh x86_64
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_0141irCwtPmPMZa9s7pCN3Cz
Two struct-size assertions hold only with 8-byte pointers: SymbolAux ends
in a Vec and PendingComdatSignature starts with a fat pointer, so they
are 48 and 16 bytes on a 32-bit host and rustc rejected the crate for
armv7. Gate them on target_pointer_width = "64"; the other five size
assertions have no pointer-sized fields and pass on both widths.
CI now runs cargo check for armv7-unknown-linux-gnueabihf over the whole
workspace so this cannot regress unnoticed.
Tests: cargo check --release --target armv7-unknown-linux-gnueabihf --workspace --all-targets
Tests: cargo test --release (pass=5681 skip=654 fail=0)
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_0141irCwtPmPMZa9s7pCN3Cz
The bullseye targets (aarch64, arm, ppc64le, s390x) left apt on the live
deb.debian.org mirrors. Those mirrors dropped the architectures Debian LTS
does not support when bullseye entered LTS, so `dist.sh ppc64le` and
`dist.sh s390x` fail in `apt-get install` with "Unable to locate package
build-essential". The pinned snapshot the images ship commented out in
sources.list still has them, and using it also gives these builds the
reproducibility the header comment already claims.
Make the snapshot switch the default apt_setup; riscv64 (deb822 sources)
and loongarch64 (live sid) keep their own.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_0141irCwtPmPMZa9s7pCN3Cz
Retain the original versioned reference when following undefined symbols
in shared libraries. This prevents an unversioned archive definition from
causing unnecessary extraction while preserving symbol interposition.
Skip the retained undefined references in the secondary symbol type check
and port both upstream regression tests unchanged.
Validation: full workspace tests passed with 34 unit tests and 5,684
integration tests across 14 target configurations; 651 skipped, 0 failed.
Four additional targets lacked a compiler or QEMU. The archive regression
fails before the fix, and both new tests pass on all available targets.
Upstream-Commit: 294e22a86a
Retain the original versioned reference when following undefined
symbols in shared li6braries. This prevents an unversioned archive
definition from causing unnecessary extraction while preserving symbol
interposition.
Fixes https://github.com/rui314/mold/issues/1657
Append 70,000 empty sections to GCC-generated LTO assembly for a tiny
program. Preserve the extended section numbering checks for slim and
fat LTO objects and verify that the fat object actually invokes LTO.
Validation: this test passes on all 14 available target configurations.
The observed LoongArch run dropped from 19 seconds to 4.4 seconds; the
14-target run completed in 5.7 seconds.
Replace 1,000 grep invocations with one awk process while checking all
500 DWARF32 and 500 DWARF64 names. This avoids timeouts spent repeatedly
spawning grep during parallel test runs.
Validation: the updated test passes on all 14 available target
configurations. The check accepts a complete dump and rejects missing
names and empty input.
Remove the NOBITS debug-link placeholder and remap section indices so
symbols and section headers still refer to the correct sections. Port
the upstream regression test unchanged.
Validation: separate-debug-file tests across 14 target configurations:
29 passed, 13 skipped, 0 failed. The regression fails before the fix and
passes afterward.
Upstream-Commit: f4edcbd37e
Remove the malformed NOBITS debug link and adjust section indices so
symbols and section headers still refer to the correct sections.
Add a regression test for following the debug link and preserving
symbols in later sections.
Fixes https://github.com/rui314/mold/issues/1656
The relocatable output lays out all leader CIEs before the FDE blocks
of every file, but EhFrameRelocSection::copy_buf emitted the
relocations per file: a file's CIE relocations followed by its FDE
relocations. When a file after the first has a CIE with a relocation,
which is any object with a personality routine, that relocation was
written after the previous files' FDE relocations, so .rela.eh_frame
was not sorted by offset.
Consumers pair CIEs and FDEs with a contiguous run of relocations in a
single scan over .eh_frame, and parse_ehframe is one of them. On the
unsorted section the personality relocation was attached to an earlier
file's FDE with a negative in-record offset, so the final link wrote the
value into the preceding FDE and left the CIE's personality pointer
zero. libgcc treats an encoded zero as "no personality", so functions
under that CIE lost their cleanups and catch handlers: an exception
thrown inside their try block escaped uncaught. Found by dev-util/perf,
whose build partially links a Rust workload object after C objects.
Emit the CIE relocations of every file first and only then the FDE
relocations, which matches the layout, and add a test that links a C
object before a C++ object with `-r`, checks that the relocations are
sorted, and catches an exception through the result.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_0141irCwtPmPMZa9s7pCN3Cz
The relocatable output lays out all leader CIEs before the FDE blocks
of every file, but the relocations for them were emitted per file: a
file's CIE relocations followed by its FDE relocations. When a file
after the first has a CIE with a relocation, which is any object with
a personality routine, that relocation was written after the previous
files' FDE relocations, so .rela.eh_frame was not sorted by offset.
Consumers pair CIEs and FDEs with a contiguous run of relocations in a
single scan over .eh_frame, mold included. On the unsorted section the
personality relocation was attached to an earlier file's FDE with a
negative in-record offset: we panicked on the slice index, and C++ mold
silently wrote the value into the preceding FDE and left the personality
pointer zero, which libgcc treats as "no personality", dropping that
function's cleanups and catch handlers. Found by dev-util/perf in the
Gentoo campaign; a Rust workload object partially linked after C objects
triggered it.
Emit the CIE relocations of every file first and only then the FDE
relocations, which matches the layout. The C++ implementation received
the same change.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_0141irCwtPmPMZa9s7pCN3Cz
Move shell cases and the runner directly under tests/, and remove the
standalone symbol output chunk test files and their references.
Make shell scripts executable and invoke them directly, honoring their
shebangs and execute permissions.
Context already uses E through its input files and output sections, so it
does not need a PhantomData field.
Validation: workspace check across all targets.
Reserve ReaderJob for command-line inputs. Store deferred scripts and LTO
files as tuples with required mapped-file references, and remove unused
archive metadata from the command-line job. Move reader contexts into the
script queue while preserving command-line sorting.
Validation: workspace check; 28 script, archive and LTO tests passed
(two 32-bit prerequisite skips); clang, Chrome and TensorFlow debug
outputs match the baseline.
Replace the manual registration flag with std::sync::Once. Keep counter
updates atomic and skip registration when statistics are disabled.
Validation: workspace check; clang, Chrome and TensorFlow debug links
with matching counter values under --stats.
Remove PhantomData<()> from LoongArchTarget. Its const parameter needs no
marker field, and the target remains a zero-sized type.
Validation: workspace check, including both LoongArch targets.
Replace the separate shuffle mode and seed fields with ShuffleSections.
Keep the last explicit seed while parsing, so a later bare shuffle option
reuses it even after --reverse-sections. Extend the existing seed test to
cover that option sequence.
Validation: workspace check; shuffle and reverse integration tests;
the new option sequence also passed before the implementation change.
Replace SectionOrderKind and its separate fields with a SectionOrder
enum. Keep names or numbers in their variants and retain the original
token only for address diagnostics.
Validation: workspace check and existing section-order integration tests.
Allocate character tables only for bracket expressions. Literal tokens
carry their bytes, and star and question tokens carry no data. Match the
variants directly when parsing patterns and building the NFA.
Validation: workspace check; four glob unit tests; 34 glob and version
script integration tests; clang, Chrome and TensorFlow debug benchmarks.
Replace BuildIdKind and the separate value/hash_size fields with one
BuildId enum. Each parsed option sets a complete variant, and note
emission reads its payload.
Validation: workspace check; existing build-id test; 10 checks covering
absent IDs, fixed byte values and option overrides.
Remove the redundant struct and constructors. Keep the iterator opaque
and retain always-inline dispatch for ordinary and deferred CREL records.
Validation: workspace check; 15 relocation and GDB-index tests; three
large debug links with byte-for-byte identical output.
Replace Arm32InputSectionExtra with u32, using u32::MAX for an absent
.ARM.exidx section. Add InputSectionExtra::empty so ARM32 starts at the
sentinel instead of u32's default zero.
Implement InputSectionExtra directly for Box<[RelocDelta]> and use it on
RISC-V and LoongArch. Remove the wrapper that only forwarded access to
its boxed slice.
Replace the section display wrapper and both file-name display wrappers
with formatting closures. Share the file-name formatter between input
parsing and merged-section diagnostics.
Use Rayon's recursive splitter and split_at_mut to give each task a
separate slice of the preallocated edge array. Preserve edge order and
the per-vertex count checks while removing EdgeBuffer and its unsafe
Sync implementation.
Use owned vectors for parallel thunk symbol collection, removing
ThunkSymbolBins and its unsafe Sync implementation. Atomic symbol marking
still deduplicates entries, and the final sort preserves output order.
Collect files and scripts in per-worker vectors, locking only to append
completed results. Consume the bins after the scoped jobs join and restore
command line order as before.
Keep one cache per Rayon worker and lock it once per input file. Remove the
custom UnsafeCell wrapper and its unsafe Sync implementation while
preserving cache reuse across jobs.
Report a parse error instead of silently defining address zero when a
decimal --defsym value exceeds u64::MAX, as for hexadecimal values.
Test the largest valid decimal and hexadecimal values, zero, and values
beyond the limit. The new regression fails with the original Rust linker;
all seven defsym integration tests pass with this fix.
Keep generated names as bytes and replace each non-ASCII-alphanumeric byte
with an underscore, matching C++ mold for valid and invalid UTF-8 input.
Add a regression that links references to ordinary start/stop symbols.
It passes on all 14 available target configurations and with C++ mold,
and fails with the original Rust linker.
Keep linker-script assignments, --defsym definitions and targets, retained
names, and forced undefined symbols as byte vectors through lookup and GC.
Accept raw command-line symbol names and trim retain-file entries like C++ mold.
Add a regression with distinct invalid UTF-8 and replacement-character names,
raw alias names, archive extraction, and GC roots. It passes with the Rust and
C++ linkers and fails with the original Rust linker. Existing defsym and
retain-symbols tests also pass.
Store the temporary winning file priority in an actual AtomicU32 instead of
casting an immutable u32 field to an atomic reference. Keep ordinary
symbol-index access behind getters and an exclusive setter.
Add a concurrent ownership regression verified under Miri with both Stacked
Borrows and Tree Borrows.
Store files in Vec<Box<T>> and live entries as u32 pool indices. Resolve
access through the pool so indexed access and live iteration do not rely
on cached pointers across Box borrows.
Use disjoint pool slices for mutable and parallel iteration, preserving
stable file indices while letting Box and Vec manage ownership.
Use the abort panic strategy for release builds to reduce generated
code. A panic now terminates the process without unwinding the stack.
An isolated five-run comparison across three workloads reduced .text
size by 12.7% on the local x86-64 machine and 13.6% on Fedora AArch64.
The measured link-time differences were small relative to run-to-run
variation.
Validated release builds for all 20 targets and successful links for
all three workloads on both machines.
Avoid clock reads, getrusage calls and shared record locks during ordinary
links. Explicit --perf keeps the existing timing output. Cover nested timers,
background handles and repeated stops in a unit test.
Match C++ output growth by passing the old length and appended size to
fallocate. Revisiting the already-written prefix flushes dirty extents on btrfs
and adds about a second to Chromium debug links on Fedora. The GDB-index suite
passes across available targets, and the change is neutral on ext4.
Inspect ELF symbol type and definition with readelf instead of requiring nm to
label foo as text. PPC64 ELFv1 function symbols name descriptors in .opd and
correctly appear as data to nm. This lets the existing path and reproduction
tests run across the full target suite.
Initialize reachability when creating object, LTO, and shared inputs so
mandatory objects outrank unextracted archive members during initial resolution.
Process every --undefined-glob match, including multiple matches in reachable
files, so GC retains each matching definition. Port both upstream regressions
verbatim.
Upstream-Commit: af7d056853
Mark dead fragments, copy relocations, and dead input sections as cold paths,
matching the upstream branch hints while retaining ICF leader resolution and
discarded-section diagnostics.
Upstream-Commit: 5bdd5e6284
Collect strong undefined references during the existing import/export traversal
and apply binding changes together with the import flags. Remove the later full
object-symbol scan, preserving weak binding when every undefined reference is
weak.
Upstream-Commit: a4bcbb45ae
Remember up to 32 merged-section keys and IDs in each conversion worker,
avoiding the shared lock on cache hits. Copy immutable keys rather than
retaining pointers into the growable section vector, and discard the cache when
conversion finishes.
Upstream-Commit: 4a83b5d856
Record shared-library names and default-version aliases directly into worker
bins during parsing. Reserve both symbol vectors first so their recorded
slots stay stable until the existing parallel gather fills them. FileList
retains the backing files through SONAME deduplication. This removes the
temporary name arrays and the later registration pass.
Upstream-Commit: 20eb664a7e
Load flag bytes before setting bits, skipping the read-modify-write when all
requested bits are already present. Apply the shared helper to symbol and
input-section setters; keep fetch_or for operations that need the previous
value.
Upstream-Commit: c72d19dcca
Initialize reachability when creating object, LTO, and shared inputs so
mandatory objects outrank unextracted archive members during the first
symbol resolution pass.
Process all --undefined-glob matches regardless of file reachability so
--gc-sections retains every matching definition. Cover archive ordering,
directly linked objects, and multiple matches in one object.
MergedSection::get_instance took a shared lock for every mergeable
input section, though nearly all of them resolve to the same few
output sections. Each worker now remembers up to 32 sections it has
already resolved, keyed on the same name, flags, type and entry size,
and checks them before taking the lock. The cache lives only for the
conversion pass, during which those fields are immutable.
Geometric-mean link time over the 21-program suite fell 0.87% at 32
threads on a Threadripper 7980X and 3.88% at 20 threads on an M1
Ultra, where create_merged_sections on clang-release went from 24 ms
to 10 ms.
Shared libraries inserted their symbol names into the symbol map one
at a time on the parsing thread, leaving a serial tail behind large
DSOs. Their unversioned names now go through the same per-thread bins
as object-file symbols, so gather_symbols creates the canonical
Symbols in parallel across hash shards. The symbol vector is reserved
up front, so the recorded slots stay valid until the gather fills them.
Geometric-mean link time over the 21-program suite fell 1.41% at 32
threads on a Threadripper 7980X, almost all of it from blender-release
(0.177 s to 0.142 s), which links against many shared libraries.
Relocation scanning requests GOT, PLT and TLS entries for the same
symbols repeatedly, and each request took exclusive ownership of the
symbol's cache line even when the bits were already set. Atomic's
operator|= now loads first and skips the read-modify-write when no
bit would change. fetch_or stays unconditional for callers that need
it.
Measured before the check moved from a symbol-specific helper into
the wrapper: geometric-mean link time over the 21-program suite fell
1.95% at 32 threads on a Threadripper 7980X and 2.71% on an M1 Ultra.
Use OsString for arguments and PathBuf for filesystem names so valid Unix paths
survive parsing and reach the filesystem unchanged. Preserve their bytes in
loader metadata, LTO callbacks, reproduction archives and dependency rules as
well.
Add a regression using compiler-generated objects with non-UTF-8 bytes and
spaces in their names. Cover direct arguments, nested response files, library
search, linker scripts, thin archives and reproduction replay.
Store compact chunk IDs in symbol origins instead of pointers with erased
lifetimes and target types. Resolve each ID through an append-only context
registry, preserving the 48-byte symbol layout and removing the raw pointer
dereference.
Add runtime tests for context moves, chunk reordering and 32-bit layouts, plus
compile-fail tests for dangling and incorrectly typed references. The unit
tests, API tests, doctests and 15 related native linker tests pass.
Keep SONAME bytes intact during library deduplication, dependency checks and
dynamic string and version-table output. Distinct non-UTF-8 names must retain
distinct library identities.
Add two versioned libraries whose names previously collapsed under lossy UTF-8
decoding. Verify both names in .dynstr and run the linked program. All 43 native
SONAME, dependency and version-script tests pass.
Make archive member paths absolute without resolving symlinks, matching the
names retained in response.txt. Reopen each input to copy its original contents
as before.
Add a regression that extracts and replays archives for a relative symlink and
an absolute path through a symlinked directory. Both native reproduction tests
pass.
Use the reader context to determine the target of a nested script before
accepting it. This lets an incompatible script beside its parent be skipped in
favor of a compatible script in a later library directory.
Add an x86-64 regression with competing i386 and x86-64 nested scripts. It fails
before the fix and passes in C++ mold; all 15 native linker-script tests pass.
Stop decompression when the requested output buffer is full, allowing the
twelve-byte DWARF classification probe to read a prefix of a larger frame.
Preserve the premature-end diagnostic for short output.
Add a test with verified Zstd-compressed DWARF32 and DWARF64 objects and check
their output order. The new test fails before the fix and passes in C++ mold.
All seven native Zstd and debug-sorting tests pass.
Recognize global:foo and local:* in the shared version-command parser so VERSION
blocks behave like standalone version scripts. Keep colons intact in input
filenames.
Add a regression covering local and global visibility and colon-containing input
names. Discover the new test/ directory alongside the existing shell suite. All
32 native version-script tests pass.
Record how all 27 upstream commits are represented in Rust, including
existing equivalents and C++-only changes. Describe the Rust-specific
optimizations and the completed build, integration, unit-test and static
checks.
Document the 21-workload release benchmark, build settings and noise
rejection criteria. Include raw samples supporting the measured 6.1%
improvement over the original Rust build and the remaining 2.9%
geometric-mean gap to C++.
Inline addr_with and the small auxiliary-index accessors used during
relocation and symbol-table writes. Move rare discarded-section and
.eh_frame diagnostics into a cold, non-inlined helper while retaining
ICF leader handling in the common address path.
Read the auxiliary record once when checking for PLT entries. These
changes reduce call and lookup overhead without changing computed
addresses or diagnostic behavior.
Force AhoCorasick::walk into its caller because it runs once per input
byte while matching symbol names. Keeping the state index and current
priority in the caller's registers removes per-byte call overhead. The
transition logic and highest-priority match rules stay unchanged.
Use mark_reachable for shared-library edges so repeatedly visiting an
already reachable DSO does not keep writing its reachability flag. This
avoids unnecessary cache-line traffic during parallel traversal.
Allocate child-work buffers only when a batch discovers new sections,
and transfer full buffers with mem::take. Exported symbols can already
make most sections roots, leaving many batches with no children; those
batches no longer need a heap allocation.
Feed ICF digest words directly to SipHash when its byte buffer is empty,
avoiding temporary byte arrays and buffering work during propagation.
Keep the byte-update fallback for partial buffers, preserve the native
digest byte order and wrapping length counter, and inline the hot update
and finalization paths.
Compare word updates with byte updates across partial prefixes, scalar
tails and length wraparound to verify that the digest is unchanged.
Replace the eight Option<u32> fields in SymbolAux with four-byte indices
using u32::MAX for absence. This removes separate discriminants and
reduces each auxiliary record from 96 to 64 bytes. Use u32::MAX - 1 for
the provisional dynamic-symbol index so it remains distinct from the
missing-entry sentinel.
Update table insertion and lookup helpers, assert the record size, and
test absent indices, index zero, the provisional value and reuse.
Upstream changes temporary-output-name construction to avoid a GCC
-Wrestrict warning from C++ string concatenation. Rust already builds
these names through formatting and has no matching expression to fix.
This empty commit records that no Rust change is needed.
Upstream-Commit: dad2695f3e
Allocate decoded_crel for all section indices on the first CREL section.
This avoids repeatedly growing and copying the table during section
parsing. Files without CREL allocate no table.
Upstream-Commit: ff0cb501b0
Parsing already omits losing COMDAT members and marks retained sections
alive, so skip redundant state updates for newly parsed files. Keep the
member walk when reusing parsed files, when LTO can change ownership, or
with --gdb-index.
Upstream-Commit: 91252d8aa5
Upstream increases per-thread arena blocks from 64 KiB to 1 MiB to
reduce global reservations, retaining the local-allocation cutoff at one
quarter of a block. Rust uses vectors and mimalloc rather than that
arena, so there is no corresponding block-size setting to change. Record
the C++ change with an empty commit.
Upstream-Commit: 4a11b7849d
Populate name-length caches during symbol registration. Scan global
names up to '@' or NUL, then scan the version suffix only when present,
retaining the raw length before version normalization or --wrap. This
removes a separate cache-population pass for both regular and LTO
objects.
Upstream-Commit: 53715c07ee
Mark the .dynsym, .plt and .plt.got add_symbol functions inline so the
compiler can expand their calls in the serial table-allocation loop.
This exposes the small insertion helpers to optimization while
preserving table insertion order and index assignment.
Upstream-Commit: 41d2f376c3
Reuse the version-suffix flag set during global-symbol registration
instead of scanning COMDAT signature names for '@' again. Handle a bare
trailing '@' separately because it does not set the flag.
Upstream-Commit: 3a58944ff9
Check dynamic-symbol version conflicts in parallel, using existing
symbol-map locking and synchronized diagnostics. Keep the error
checkpoint after all checks finish.
Upstream-Commit: ec7c803207
Create fragment records as section-symbol relocations are reattached,
collecting only the records each file needs in relocation order. Reserve
exactly their final slots in the central symbol vector, then construct
symbols and append their IDs to each file in parallel. This removes the
preliminary relocation-counting pass and unused dummy symbols. Retain
the cheap CREL-header capacity bound to avoid moving existing symbols.
Upstream-Commit: f9bdf4b4de
Rust already runs the synthesized-symbol, object-file and shared-file
symbol-table writers concurrently. Their symbol, string and
extended-section-index ranges are disjoint, so independent writers need
no shared append point. This empty commit records that the upstream
parallelization is already present.
Upstream-Commit: e9de4def18
Keep each dynamic symbol's name and string offset after sizing .dynstr,
reusing cached GNU hash sort keys when available. Both output writers
can then use independent parallel loops without repeating prefix scans
or name lookups. The cache costs 24 bytes per dynamic symbol.
Upstream-Commit: 18d173697c
Rust already caches exact u32 symbol name lengths while keeping Symbol
at 48 bytes. Its name accessors use that cache without rescanning long
names, so there is no corresponding layout or lookup change to apply.
This empty commit records the upstream equivalence.
Upstream-Commit: fa1e8edc39
Rust already uses flat hashbrown tables for symbol-map shards. Compare
cached hashes before strings and fold higher hash bits into bucket
placement so fixed shard-selection bits do not concentrate keys in the
same buckets. Preserve entropy in the high-bit lookup tags and keep
symbol storage stable while reducing lookup overhead.
Upstream-Commit: 841880212f
Resolve non-allocated merged sections and assign their offsets in a
background Rayon task while symbol processing, GC and relocation
scanning continue. Give the task owned metadata and immutable input
bytes, create .comment before starting it, and publish the result before
section sorting and sizing. Keep allocated-fragment layout after GC, and
reassign offsets when debug sections are reordered.
Upstream-Commit: f585fee6ca
Fill .gnu.version slots in parallel after allocating the array. Each
dynamic symbol owns a distinct slot, so these writes are independent.
Upstream-Commit: 95137b235e
Rust already collects newly marked GC roots in local Rayon worker
vectors and combines them before graph traversal. This avoids contention
on a shared concurrent vector, matching the upstream change. No source
change is needed; this empty commit records that equivalence.
Upstream-Commit: 95dbca6708
Give consecutive version-script patterns with the same version index one
priority. Stop matching when no higher-priority result is possible, and
check simple substring patterns before glob patterns.
Upstream-Commit: f8cb3daf00
Compute fragment hashes and cardinality estimates during the string or
record scan, avoiding a second traversal of the input contents and
fragment offsets.
Upstream-Commit: 0f582b0a23
Debug relocations often visit consecutive merged strings. Cache the last
symbol and mergeable-section metadata in each relocation stream, so
input sections do not share mutable lookup state. Try the next fragment
before binary search and prefetch upcoming entries.
Upstream-Commit: 2c402f707f
Process four unsigned bytes per iteration and factor the DJB recurrence
into pairs to shorten its dependency chain. Hash the remaining bytes
with the scalar loop.
Upstream-Commit: 046a644906
Add a stable partition helper that evaluates predicates and scatters
elements into disjoint output ranges in parallel. Use it for the
local/global and import/export partitions in sort_dynsyms, preserving
the relative order within both groups. Evaluate predicates sequentially
for small ranges.
Upstream-Commit: 8566358143
Prefetch symbol records ahead of their auxiliary records to hide the two
dependent pointer loads in the serial table-allocation loop.
Upstream-Commit: 9887339f72
Keep dynamic symbols grouped by their owning input file so auxiliary
records can be initialized and indexed in parallel without globally
sorting symbol IDs. File ownership makes writes independent. Preserve
first-use order within each file, suppress repeated IDs, and retain
existing auxiliary data before the serial table-allocation loop.
Upstream-Commit: 0706198355
Cache GNU hash bucket indices and name views before sorting dynamic
symbols. This avoids repeated pointer chasing, name-length lookups and
bucket calculations in the comparison function.
Upstream-Commit: cbcf79c0ef
Compute string offsets with parallel scans and write dynamic symbols and
strings in parallel. This removes serial work on large dynamic symbol
tables while preserving output order.
Upstream-Commit: 6060657622
Allocate decoded_crel for all section indices on the first CREL section.
This avoids repeatedly growing and copying the table during section
parsing. Files without CREL allocate no table.
Parsing already omits losing COMDAT members and marks retained sections
alive, so skip redundant state updates for newly parsed files. Keep the
member walk when reusing parsed files, when LTO can change ownership, or
with --gdb-index.
Populate name-length caches during symbol registration. Scan global
names up to '@' or NUL, then scan the version suffix only when present,
retaining the raw length before version normalization or --wrap. This
removes a separate cache-population pass for both regular and LTO
objects.
Reuse the version-suffix flag set during global-symbol registration
instead of scanning COMDAT signature names for '@' again. Handle a bare
trailing '@' separately because it does not set the flag.
Check dynamic-symbol version conflicts in parallel, using existing
symbol-map locking and synchronized diagnostics. Keep the error
checkpoint after all checks finish.
Create fragment symbols in the arena as section-symbol relocations are
reattached. Collect pointers locally and reserve the file's symbol
vector once before appending them in relocation order, eliminating the
preliminary counting pass.
Run synthesized-symbol, object-file and shared-file symbol-table writers
concurrently. They write disjoint symbol, string and
extended-section-index ranges; initialize section symbols before
starting these writers.
Keep each dynamic symbol's name and string offset after sizing .dynstr,
reusing cached GNU hash sort keys when available. Both output writers
can then use independent parallel loops without repeating prefix scans
or name lookups. The cache costs 24 bytes per dynamic symbol.
Store a signed 32-bit name length in Symbol's existing padding to avoid
rescanning long names. Saturate at INT32_MAX and scan the suffix only
for saturated lengths.
Replace per-key unordered_map nodes with linear-probing slots containing
cached hashes and pointers to arena entries. This reduces lookup
overhead while keeping symbol addresses stable.
Resolve non-allocated merged sections and assign their offsets in a
low-priority TBB task while symbol processing, GC and relocation
scanning continue. Create .comment before starting the task and join
before section sorting and sizing consume the results. Keep
allocated-fragment layout after GC, and reassign offsets if debug
sections are reordered.
Collect newly marked root sections in per-thread vectors, then flatten
them before graph traversal. This avoids contention on the shared
concurrent vector.
Give consecutive version-script patterns with the same version index one
priority. Stop matching when no higher-priority result is possible, and
check simple substring patterns before glob patterns.
Compute fragment hashes and cardinality estimates during the string or
record scan, avoiding a second traversal of the input contents and
fragment offsets.
Debug relocations often visit consecutive merged strings. Cache the last
symbol and mergeable section per thread, keyed by input section and
symbol index. Try the next fragment before binary search and prefetch
upcoming entries.
Process four unsigned bytes per iteration and factor the DJB recurrence
into pairs to shorten its dependency chain. Hash the remaining bytes
with the scalar loop.
Add a stable partition helper that evaluates predicates and scatters
elements in parallel. Use it for the local/global and import/export
partitions in sort_dynsyms, keeping the standard algorithm for small
ranges.
Initialize SymbolAux in the per-file collection pass, where symbol
ownership makes the writes independent. This reduces work in the serial
allocation loop.
Cache GNU hash bucket indices and name views before sorting dynamic
symbols. This avoids repeated pointer chasing, name-length lookups and
bucket calculations in the comparison function.
Compute string offsets with parallel scans and write dynamic symbols and
strings in parallel. This removes serial work on large dynamic symbol
tables while preserving output order.
Deferred LTO jobs already own archive names, but claiming each member
allocated a second copy. Move the name into the object and use the object's
archive name when setting as_needed.
Validation: all 27 native LTO tests pass. Workspace Clippy passes with
warnings as errors. All 545 upstream test files match C++ mold exactly.
Remove five Rust-only shell tests so the test cases and common support file
match the C++ suite exactly. The Cargo runner still provides test discovery and
execution.
Validation: compare every tracked test file and Git file mode against C++ mold
at 92eb17a07c60a88b3a1a8c9cef014ac84eaf0cf9; no missing, extra or modified test
cases.
Port C++ mold 92eb17a07c. Record IR files from
direct inputs, regular and thin archives, and linker scripts, then claim them
serially after sorting by their command line positions.
Remove the redundant claim mutex and once-only initialization guard. Require
mutable context access when claiming an IR file, and retain the simple
loaded-state check used by the serial plugin lifecycle.
Copy the upstream regression test unchanged. It reproduces an undefined
__cxx_global_var_init before the fix. All 27 native LTO tests pass afterward.
Port C++ mold 450832db6b. Apply the
internal-object binding rule after handling input-local symbols and relocatable
output, preserving global binding for exported synthetic symbols.
Copy the upstream symtab-binding test unchanged. It fails before the fix and
passes afterward. Native symbol-table, synthesized-symbol, start/stop, export
and relocatable-output checks: 24 passed, 1 skipped.
Port the test from C++ mold d3cf4a23e0 unchanged.
The Rust symbol and section arenas were already replaced with vectors, so there
is no large writable arena reservation to change.
Validation: the native rlimit-data test passes with the current allocator.
Port C++ mold 32626eba5d. Ignore empty SFrame
input before reading its header, as required for startup objects assembled by
GNU assembler 2.45.
Copy the upstream regression test unchanged. It failed with a corrupted-section
diagnostic before the fix and passes afterward. Native SFrame tests: 1 passed, 2
skipped because the assembler lacks SFrame v3 support.
The parallel file reader handed IR files to the LTO plugin in
whatever order it found them. LLVM's LTO assumes that the module
holding the prevailing copy of a COMDAT group arrives before the
modules holding the other copies, as it does with a serial linker;
otherwise the losing copies' internal symbols, such as
__cxx_global_var_init.N, become undefined references in the LTO
result. The reader now only records IR files and claims them after
all input files have been found, sorted by command line position.
Claims were already serialized, so no parallelism is lost, and the
serializing mutex and once_flag in lto-unix.cc are no longer needed.
Fixes https://github.com/rui314/mold/issues/1649
Co-Authored-By: Claude Fable 5.1 <[email protected]>
35f4a334 changed is_local() to derive a symbol's binding in .symtab from
its visibility. Linker-synthesized symbols never have their visibility
merged from the internal object's ELF symbols, so they went from
STB_LOCAL to STB_GLOBAL by accident. That adds _DYNAMIC, __bss_start,
_end and the like to the set of global symbols, which confuses ABI
checkers that read .symtab.
Restore the old rule for the internal object's symbols: they are local
unless we export them. The exported ones, such as __global_pointer$ on
RISC-V or __start_*/__stop_* under -z start-stop-visibility=protected,
stay global as before.
Reported by Yao Zi in https://github.com/rui314/mold/pull/1650
Fixes https://github.com/rui314/mold/issues/1651
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Keep Endian, byte order markers, and integer wrappers together in
util::endian. Add fixed-endian read/write helpers and use them from
Endian and architecture code.
Use one macro to provide new, get, and set consistently for all five
standard integer wrappers.
Make create_phdr's define and append helpers accept &ElfShdr directly.
Reuse section headers at their call sites and use short osec bindings
for optional sections.
Validation:
- cargo build --locked --workspace
- cargo test --locked --workspace: 5,487 integration tests passed,
637 skipped, no failures; all 14 library tests passed. Four cross
targets lacked a compiler or QEMU.
Rename the module directory and update its declaration, imports, and
call sites to use crate::chunks.
Validation:
- cargo check --locked --workspace
- cargo test --locked --workspace: 5,487 integration tests passed,
637 skipped, no failures; all library tests passed. Four cross targets
were unavailable because their compiler or QEMU was missing.
Give each section a sibling module containing its type and functions,
and update callers to use the direct paths. Remove the misc and version
grouping modules and redundant nested wrappers.
Validation:
- cargo check --locked --workspace
- cargo test --locked --workspace: 5,488 integration tests passed,
636 skipped, no failures; all library tests passed. Four cross targets
were unavailable because their compiler or QEMU was missing.
ConcurrentMap exposed lookup and traversal while insertion was still
possible. Move those entry points to FrozenMap and keep the construction
API limited to allocation, insertion, prefetch, and freezing.
Freeze the GDB name map before collecting sorted entry references.
Extend the existing map test to verify lookup by the original entry IDs
after freezing, including the associated keys and values.
ConcurrentMap::prefetch only issued a cache hint on x86-64. Move the hint
into a shared utility and use stable inline assembly for x86, ARM32/64,
RISC-V32/64, PowerPC32/64, s390x, and LoongArch64.
Assume SSE on both x86 variants. Encode RISC-V PREFETCH.R as its ORI hint
so it remains a no-op without Zicbop. Keep unsupported architectures as
no-ops and preserve the map's empty-table guard.
Move the Linux and Android MADV_HUGEPAGE calls into shared utilities,
with no-op implementations on other targets. Reuse them for output
mappings, concurrent-map storage, and symbol allocations, keeping the
symbol advice limited to complete pages within the allocation.
Extract platform-specific allocation and deallocation helpers for the
concurrent map, and guard page-alignment rounding against overflow.
The 8 GiB arena was mapped read-write, so on hosts with overcommit disabled
(vm.overcommit_memory=2) Linux ignored MAP_NORESERVE and charged the whole
reservation against the commit limit, and fork() charged it again for the
child. Even a hello-world link then failed with "fork: Cannot allocate
memory". Reserve the range with PROT_NONE instead and make it writable in
64 MiB chunks as the bump pointer advances, as the Windows code already does
with MEM_COMMIT. Only the chunks in use are charged, by mmap and by fork()
alike, and transparent huge pages still back them.
Under RLIMIT_DATA, which counts writable private mappings the same way, a
trivial link now needs 0.5 GiB instead of more than 8 GiB.
Fixes https://github.com/rui314/mold/issues/1644
Co-Authored-By: Claude Fable 5.1 <[email protected]>
InputSectionId stored an already packed integer while FragmentRef kept
its component indices. Store a private ObjId and zero-based u32 index in
InputSectionId, with direct field access in file() and index(). Confine
the packed representation to raw() and from_raw().
Keep the eight-byte size and the existing Origin bit layout. Use the
reserved u32::MAX index for NONE and apply the index-plus-one encoding
at the packing boundary so its payload remains zero. Extend the existing
round-trip test to cover zero indices, maximum supported indices, and
the placeholder value.
Origin was assembling and splitting FragmentRef's two indices itself,
while InputSectionId already supplied raw and from_raw methods. Add the
same encoding interface to FragmentRef and use it from Origin::new and
Origin::get.
Keep FragmentRef's typed fields and preserve the existing packed layout.
Origin still checks the 30-bit merged-section limit before shifting the
payload to make room for its tag.
ObjectFile section access and output-section construction already know
the owning file ID, but building a section ID reloads it from each input
section. Construct IDs from the known owner and the decoded dense index,
and pass the owner once into regular_ids_mut.
Move paired ID/reference lookup to ObjectFile so it resolves the ELF
section index once. Reuse the validated dense-index accessors for shared
and mutable section access, avoiding redundant owner loads and checks.
InputSectionIds are assigned when a section is inserted into a file's
dense vector, and their file indices refer to the stable file pool.
Resolving these IDs currently repeats bounds checks for both indices on
hot section-access paths.
Add internal unchecked accessors for the pool and dense section vector,
and use them in Context::input_section. Document their insertion-based
preconditions and retain debug assertions on both bounds. Keep the
unsafe operations within these accessors and the ID resolution method.
The early fragment estimate scans ordinary relocation tables before
archive extraction, including members that may never be selected. That
adds work to large links such as ClickHouse just to predict capacity.
Restrict the early estimate to cheap CREL header counts from direct
inputs and already reachable archive members. Leave ordinary relocation
counting to the existing preparation of selected files after COMDAT
selection. This retains the early reservation needed by CREL-heavy links
while avoiding an extra scan of ordinary relocation records.
Rewriting mergeable-section relocations can append millions of dummy
symbols after global symbols have been constructed. Reserving only local
symbols allows this later append to grow the vector and copy the already
populated table, which is particularly costly for CREL-heavy links.
Estimate fragment demand before gathering globals, using relocation
counts from CREL headers and inspecting ordinary relocations that refer
to mergeable section symbols. Include that estimate in the initial
reservation and compute it across files in parallel. Reapply huge-page
advice only when vector capacity actually changes.
The ClickHouse benchmark contains C++ symbol names longer than the u16
limit, causing valid input to panic in Symbol::new. Restore u32 storage
for the cached name length and retain the checked conversion into that
representation.
Change the boundary test to construct a 65536-byte name successfully and
verify that Symbol::name returns all of its bytes. This removes the
restriction introduced by narrowing the length field.
Replace obsolete arena terminology with the actual storage and lifetime
rules: symbols and section members live in vectors, input files belong
to a stable pool, and parallel construction reserves disjoint ranges
before exposing writable slots.
Remove stale ArenaPtr distance restrictions and describe background
debug-info work in terms of tasks and concurrency. These are comment
updates; allocation, ownership, and linker behavior are unchanged.
Reserve an upper bound for local symbols when gathering globals so that
appending locals does not immediately move a populated symbol vector.
Request transparent huge pages for whole pages inside that allocation,
leaving partial boundary pages that may contain allocator metadata
alone.
Let per-file section vectors grow with the inputs actually inserted;
reserving for every ELF header wastes space on relocation and metadata
headers that never become InputSections. Validate the packed file-index
limit when inserting files into the pool, and use debug assertions for
the redundant checks when encoding and decoding validated section IDs.
Section lookups were constructing an InputSectionId and decoding it
again even when the caller only needed a reference. Factor the ELF
section table decoding into input_index and use the resulting dense
index directly for shared, mutable, and paired ID/reference access.
Keep regular entries and entries with merge metadata on the same lookup
path. Extend the Origin round-trip test to cover the full supported
32-bit section-index range.
Have InputSectionExtra expose r_deltas_mut, returning a mutable
reference to the owned delta array. InputSection::set_r_deltas can then
perform the assignment itself while retaining its nonempty-array
assertion.
RISC-V and LoongArch return their embedded field. Targets without
relaxation metadata retain the unreachable guard for attempts to mutate
it, keeping the target-specific implementation limited to field access.
Rename MergeableSection to MergeInfo and update the section-table flags
and accessors to describe its role as metadata for an ordinary input
section. Update merging, symbol attachment, and debug-info callers to
use the new names.
Preserve the existing ownership model: eligible SHF_MERGE inputs remain
in their dense slots and are marked dead while their contents are
represented by fragments. Store merge metadata only for those inputs,
without adding a merge-info field to every InputSection.
Logical section IDs and directly owned target state allow input sections
to move when their storage grows. Store InputSection values in each
SectionList's dense vector and resolve IDs by indexing that vector. Keep
merged inputs in their slots so existing references remain valid.
Remove SectionArena, its thread-local allocation blocks, and the manual
section destruction code. Vector ownership now supplies allocation and
destruction, and the shared virtual-memory reservation module can be
deleted because neither symbols nor input sections use it anymore.
The self-relative pointer to InputSectionExtras prevents input sections
from moving and requires a separate arena allocation for target state.
Make InputSection generic over Arch and embed an associated
InputSectionExtra value directly in each section.
ARM32 stores its exception-index section link, and RISC-V and LoongArch
own their relaxation-delta arrays. Other targets use a zero-sized extra
type. Propagate the target type through section owners and callers, and
remove the self-relative pointer encoding and extras allocator.
Arena-relative section IDs tie callers to the location of an allocation.
Represent InputSectionId as an object-file ID and a dense index within
that file, packed into 64 bits. Reserve two bits for Origin's tag and
keep zero available as the placeholder value.
Add a dense per-file slot table that still resolves to the existing
arena during this transition. Update symbol origins, output-section
members, relaxation, and layout lookups to resolve the logical ID
through its owner. This allows section storage to change independently
of the IDs used throughout the linker.
Symbols are already referenced through SymbolId, so their addresses only
need to remain stable while a construction or mutation phase borrows
them. Replace the 8 GiB SymbolArena reservation with Vec<Symbol> and let
the vector own and release the initialized symbols.
Keep the existing parallel construction scheme: reserve capacity before
publishing writable slots, assign disjoint ranges to workers, and set
the initialized length after they finish. Remove the custom virtual
memory allocation, slice access, and destruction code for symbols.
Use u16 for Symbol::name_len, reducing the width of the cached length
field while keeping the existing thin pointer to the name bytes. This
representation imposes a maximum symbol name length of 65535 bytes.
Check the conversion in Symbol::new so that longer names produce an
explicit diagnostic. Add a regression test for a 65536-byte name to
exercise that boundary.
Let the compiler decide whether to inline private and generic helpers that do
not correspond to explicitly inline C++ code. Keep forced inlining on the hot
ELF and relocation paths identified by the C++ implementation and perf
profiles.
Firefox-debug performance remained neutral in both single-threaded and
32-thread runs. The full test suite and release clippy checks pass.
Merge duplicated C++ and Rust comment blocks into single explanations that
describe the Rust implementation while retaining the original rationale.
Remove stale C++ type references and restore the missing rustdoc summary for
sorted concurrent-map entries.
Remove helpers, accessors, and constants that are unreachable from the mold
command and its tests. This includes wrapper methods whose only callers were
other dead wrappers, unused ELF integer operations, and unused concurrent-map
APIs.
Keep the remaining internal len methods crate-visible so Clippy does not treat
them as incomplete public collection APIs.
Keep Origin eight bytes and use its low two bits to distinguish input
sections, output chunks, merged-section fragments, and symbols. Remove the
separate origin-kind bits from Symbol’s visibility byte.
This follows C++ mold’s tagged-pointer representation without the auxiliary
tables and per-variant codecs required by the discarded 32-bit experiment.
The experiment reduced Symbol from 48 to 40 bytes, but it required a
manual codec for every ChunkId variant, a separate namespace for allocated
merged sections, and context-dependent origin decoding. It added 434 net
lines.
It also made the previous Rust linker about 2% slower on Firefox debug and
10% slower on TensorFlow debug, even though it remained faster than C++ mold.
Prefer the simpler 64-bit origin until the memory saving justifies those
costs.
Use unary low-bit tags so fragment origins have a 31-bit payload. Pack
allocated merged-section IDs and fragment bucket IDs into that payload,
and use compact encodings for input sections, chunks, and symbols.
Also compact the remaining symbol metadata so that Symbol shrinks from 48
to 40 bytes.
Split the output-section and symbol-table borrows so
gather_thunk_addresses can record addresses while traversing thunks. This
matches the C++ loop and avoids a temporary allocation for each section.
Replace the callback-based internal parser with an Iterator built using
std::iter::from_fn. Keep the eager Vec results and member creation order
unchanged.
Return the original io::Error from the shared file-opening path instead of
discarding it and reading errno later. Keep optional file lookup semantics by
mapping NotFound to None, and remove errno_string.
Rewrite comments copied verbatim from C++ so they describe the Rust
implementation and terminology. Remove notes that have no Rust counterpart,
and update source-file references to their Rust names.
Keep C++ comparisons when they document an intentional implementation
difference or a remaining performance gap.
GNU assembler 2.45 emits a zero-size .sframe section for an input file
that needs no unwind info, such as glibc's Scrt1.o on Ubuntu 25.10.
parse_sframe() read a header past the end of such a section and reported
it as corrupted.
Fixes https://github.com/rui314/mold/issues/1646
Co-Authored-By: Claude Fable 5 <[email protected]>
Mold performs one link per process, so keep diagnostic settings, error state,
and output serialization in a process-wide singleton. This removes
diagnostic plumbing from contexts and helper APIs while preserving the mutex
that keeps messages from worker threads from interleaving.
Port mold dist.sh to build Rust release binaries in old distro
containers. Download a pinned official Rust toolchain as gzip, fetch
Cargo.lock dependencies before disabling the network, and normalize
package timestamps.
Allow install-mold.sh to consume an external Cargo target directory.
path_clean("../../a/b") returned "a/b": the ParentDir branch popped a
trailing ".." from the output instead of appending another one, so any
path with two or more leading ".." components collapsed into a wrong
path. C++ mold uses std::filesystem::path::lexically_normal, which
keeps leading ".." components intact.
Library search paths are cleaned with this function, so mold failed
with "library not found" whenever a Makefile referenced an in-tree
library as -L../../<dir>, which is a very common pattern. Found by the
Gentoo all-package campaign within its first hour: sys-fs/lvm2,
sys-fs/xfsprogs, sci-astronomy/montage, dev-libs/OpenNI2,
dev-erlang/fast_yaml, and app-backup/spideroak-bin all failed with
mold-rust and built with mold-c++.
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_0118ERbwNepcG1Z9gh5cm6Ut
Benchmark record, 2026-09-01 (unrelated to this change; this tree is
what was measured, built as 6becb4da before this message was amended):
C++ reference: mold at e322a3cc, i.e. the translation base fbaaa75f
plus 15 fixes that do not touch hot paths.
Protocol: ~/bench/tab4-bench.sh as-is — run 1 warms the page cache,
median of runs 2-6 reported; all runs of one linker complete before
the other linker starts; sync + dirty-writeback wait between runs;
--no-fork; default thread count (min(ncores, 32) = 32 on this
128-core machine); frequency boost off, performance governor,
THP madvise, idle machine.
Workload C++ (s) Rust (s) Rust vs C++
blender-debug 0.94 0.82 -12.8%
blender-release 0.20 0.17 -15.0%
chrome-debug-arm64 1.88 1.97 +4.8%
chrome-debug-x86_64 1.75 1.80 +2.9%
chrome-release-arm64 0.77 0.79 +2.6%
chrome-release-x86_64 0.66 0.69 +4.5%
clang-debug 1.49 1.28 -14.1%
clang-release 0.11 0.10 -9.1%
clickhouse-debug 1.14 1.09 -4.4%
clickhouse-release 0.43 0.43 +0.0%
firefox-debug-arm64 1.07 0.98 -8.4%
firefox-debug 0.87 0.87 +0.0%
firefox-release 0.21 0.20 -4.8%
godot-debug 0.51 0.47 -7.8%
godot-release 0.08 0.08 +0.0%
libmergedlo.so-debug 0.46 0.47 +2.2%
libmergedlo.so-release 0.20 0.18 -10.0%
pytorch-debug 0.93 0.84 -9.7%
pytorch-release 0.15 0.14 -6.7%
tensorflow-debug 3.16 2.89 -8.5%
tensorflow-release 0.69 0.63 -8.7%
Geomean -5.1% (Rust faster); 13 faster, 3 even, 5 slower. No workload
regressed against the import-time table in the initial commit; the
open item remains chrome's +2-5%.
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_0118ERbwNepcG1Z9gh5cm6Ut
Apply relocation deltas, adjust symbols and recompute executable output
section layouts in parallel, matching the three file and chunk parallel
loops in C++ mold.
Process object files in parallel as C++ mold does. Use the existing
disjoint-range output-buffer abstraction so each worker can reverse its
input sections in place.
Process object files in parallel as C++ mold does. Serialize symbol access
with the existing byte-sized symbol lock and rewrite each file's relocation
records in place.
Use stock zlib's low-level API to reproduce the C++ Z_BLOCK, deflatePending,
optional deflatePrime and Z_SYNC_FLUSH sequence. This avoids the old
libbacktrace boundary bug and preserves C++ mold's compressed bytes.
Install the Unix handlers with SA_SIGINFO and track the mapped output range
as it is created, extended or moved. Report a possible full disk only when
SIGBUS or SIGSEGV identifies an address in that range.
Port the C++ Counter facility and collect the same parser, section, symbol,
ICF, relocation, COMDAT, thunk and merged-section statistics. Keep counter
updates dormant unless --stats is requested.
get_api_version handed the plugin the buffer of a stack-local
std::string, which dies when the function returns. Make it static.
Co-Authored-By: Claude Fable 5 <[email protected]>
The thin archive reader skipped a __.SYMDEF symbol table without
advancing the read position, so it kept parsing the same member header
forever. Skip past the member contents like the other symbol table
branch does.
Co-Authored-By: Claude Fable 5 <[email protected]>
R_RISCV_TLSDESC, R_RISCV_GOT32_PCREL, R_SPARC_SIZE32 and R_SPARC_JMP_IREL
were printed as "unknown (n)" in diagnostics.
Co-Authored-By: Claude Fable 5 <[email protected]>
parse_defsym_value returned a default-constructed variant on a bad hex
string such as 0xz, which holds a null Symbol pointer. The null pointer
was then dereferenced when marking GC roots, crashing the linker instead
of printing a diagnostic.
Co-Authored-By: Claude Fable 5 <[email protected]>
reattach_section_pieces rewrote every symbol whose local definition is in
a mergeable section, including global symbols that lost resolution to a
definition in another file. The loser then redirected the symbol to its
own section piece. Since the pass runs on all files in parallel, it was
also a data race on the shared Symbol.
Co-Authored-By: Claude Fable 5 <[email protected]>
R_390_12, R_390_PC12DBL, R_390_PLT12DBL, R_390_GOT12 and R_390_GOTPLT12
OR'd their 12-bit value into the instruction through a little-endian
16-bit accessor. s390x is big-endian, so the value landed in the wrong
bytes, corrupting both the displacement and the base register field.
Co-Authored-By: Claude Fable 5 <[email protected]>
create_phdrs walks a filtered, address-sorted copy of ctx.chunks, but the
loop that merges adjacent note sections into one PT_NOTE examined
ctx.chunks with an index into that copy. The two vectors differ e.g. when
the ELF and program headers are not memory-mapped, in which case every
note section got its own PT_NOTE segment.
Co-Authored-By: Claude Fable 5 <[email protected]>
When a merged section is laid out in two regions, 32-bit-addressable
fragments followed by the rest, the loop that rebases per-shard offsets to
section-global ones started at shard 1. That is fine for the 32-bit region,
whose shard 0 needs no adjustment, but it left shard 0 of the 64-bit region
unadjusted, so its fragments overlapped the beginning of the section. Start
at shard 0; the 32-bit fragments of that shard just get zero added.
Co-Authored-By: Claude Fable 5 <[email protected]>
The alloc relocation path incremented the local pointer loc instead of the
byte it points to, so the relocated byte was left untouched. The result was
wrong whenever an 8-bit label difference spanned code whose size changed
during relaxation. The non-alloc path already did this correctly.
Co-Authored-By: Claude Fable 5 <[email protected]>
The non-alloc relocation paths of i386, ARM64, s390x, SPARC64 and PPC64V1
passed the relocation value and the relocation index to check_range in the
wrong order. The intended range check was thus never performed, and if the
relocation index happened to fall outside the value range, the error path
indexed the relocation table with the relocation value, reading out of
bounds.
Co-Authored-By: Claude Fable 5 <[email protected]>
Use the same KERN_PROC_PATHNAME sysctl as C++ mold when locating the
current executable on FreeBSD, where /proc may not be mounted. Run one
shell test at a time under sanitizers so large cases do not overlap,
while leaving two CPUs available to the linker under TSan.
Use Cargo per-target features in native and platform jobs so they do not
rebuild all 20 linker instantiations. Tests that require an i386 linker
already skip when that target is disabled. Keep test-all-targets using
defaults for complete QEMU coverage.
If an object file has SHN_LORESERVE (0xff00) or more sections, e_shnum
is zero and the actual number of sections is stored in the first section
header's sh_size field. is_gcc_lto_obj() used the raw e_shnum, so it
scanned no sections and misclassified such LTO objects as regular
object files.
Fixes https://github.com/rui314/mold/issues/1642
Co-Authored-By: Claude Fable 5 <[email protected]>
Give ThreadSanitizer extra time for the many-output-sections test.
Disable incremental artifacts and debug info in the disk-constrained FreeBSD VM.
Also suppress the non-x86 unused prefetch parameter warning.
Add the four platform jobs from C++ mold and make the Rust implementation
compile on each native platform.
Use the corresponding Windows virtual-memory, output-file, subprocess, LTO
and process-timer facilities. Keep the Unix paths for macOS and FreeBSD while
applying platform-specific system calls only where C++ mold does.
Tested with cargo build --release, cargo test and cargo clippy --release
--all-targets. The MinGW executable also links against dlfcn-win32.
The all-target CI job exhausted its runner disk because it retained the
output of every completed shell test. Remove each work directory after a
passing or skipped test while preserving logs and all failed-test files for
diagnosis.
The multi-architecture build generated incremental and debug artifacts on
an ephemeral runner, leaving too little room for the test outputs and
causing the one-hour timeout. Disable both for this job.
In Ubuntu 24.04 Podman, the clean build used 693 MB and the completed test
tree used 7.4 GB. All targets passed with pass=6782 skip=895 fail=0.
ARM BE8 instructions are little-endian, but data embedded in PLT and
thunk entries uses target byte order. Match C++ mold's ul32 instruction
templates and U32<E> literal stores.
CI-equivalent Ubuntu 24.04 Podman:
all targets: pass=6782 skip=895 fail=0
armeb: pass=368 skip=53 fail=0
All workspace crates are built before testing, but the target crates and CLI
binary contain no unit tests. Select the library and integration test targets
so Cargo does not exhaust the runner disk linking empty harnesses.
musl uses an unsigned type for statfs.f_type while libc exposes
TMPFS_MAGIC as a signed value. Convert the constant to the field's type so
the comparison builds with both musl and glibc.
Match C++ mold by attempting fallocate when fstatfs fails.
The job-level environment did not populate the container configuration, so
the Rust toolchain action still lost the system directories when it extended
GITHUB_PATH. Set PATH through container.env instead.
The Tumbleweed image does not declare PATH in its container
configuration. GitHub's implicit default disappears when rust-toolchain
adds Cargo's bin directory through GITHUB_PATH. Define the normal system
path explicitly for the distro matrix so Bash remains available.
Run install-test-deps.sh with /bin/sh so minimal container images can
invoke it. Expand the Debian cross-toolchain package list without Bash
brace expansion, install Bash on Tumbleweed, and remove the Alpine
bootstrap step from CI.
The Alpine base image does not include Bash, so install it before
running install-test-deps.sh. Keep the dependency script Bash-based so it
can use brace expansion.
Run the complete multi-architecture suite and native tests across the
same Linux distribution matrix as C++ mold. Install the extra toolchains
used by the upstream tests and fail CI if any expected target is
unavailable.
Build and test ASan and TSan configurations with nightly Rust. Use the
system allocator for sanitizer builds and instrument the two C adapters as
well. Limit TSan to two CPUs to avoid false reports from Rayon while still
exercising parallel execution.
Make install-test-deps.sh portable across the distribution containers.
Tested with cargo test: pass=5360 skip=636 fail=0. ASan passed 477
native tests with 9 skips, and TSan passed 454 native tests with 32 skips.
Match C++ mold's relaxation behavior by updating input relocation records
at the point where instructions are rewritten. Store decoded CREL records
in typed mutable side tables so emitted relocation sections observe the
same updates, and remove the relocation type reconstruction hooks.
Use I386, X86_64, M68k, and Sparc64 as representative layouts when
classifying GCC LTO objects, matching C++ mold. Use real target layouts for
the compile-time record assertions and remove Elf32Le, Elf64Le, Elf32Be, and
Elf64Be.
Also remove the obsolete Record trait. Direct file records use
FileRecord, and Record had no implementations or callers.
Checked with cargo check --workspace. The test suite was not rerun as
requested.
Rename ProgramHeaderRecord and CompressionHeaderRecord to the shorter
PhdrRecord and ChdrRecord names. Expand the four plain layout definitions
directly and remove the plain_layout macro.
Infer the ELF class from the layout word size and whether relocations
carry addends from RelRecord. This removes the duplicated IS_64 and IS_RELA
declarations from every target.
Tests: cargo test (pass=5360 skip=636 fail=0)
Lint: cargo clippy --release --all-targets
Keep input section headers and symbols in their endian-aware on-file
representation instead of decoding them into target-independent mirrors.
Carry that representation through output chunks and expose fields through
record traits, so reads and writes remain target-endian.
Remove SymbolEntry, SectionHeader, SymTable, ShdrTable, and RecordLayout
now that no converted record mirrors remain. Move the SFrame constants to
elf/consts.rs.
On the 21-workload, 32-thread benchmark, the geometric-mean time changed
by -1.1% versus 801fc. Excluding tensorflow-debug's unusually large
improvement, it was effectively flat at -0.1%.
Tests: cargo test (pass=5360 skip=636 fail=0)
Lint: cargo clippy --release --all-targets
Represent uniform ELF records with public target-endian fields and select
class-dependent symbol, program-header, and compression-header layouts
through Layout. Convert differing layouts as complete records instead of
adding field-by-field forwarding accessors.
Remove the raw st_info API and the old offset-based record readers.
Tests: cargo test
Tests: cargo clippy --release --all-targets
Define byte-backed integer and relocation types that mirror the C++ ElfRel
specializations, including the SPARC64 type-data field.
Select each relocation record through Layout and view input and output
tables as ordinary typed slices. This removes the host-native conversion
wrappers and the synthetic r_info representation.
Preserve target byte order independently of the host.
Tests: cargo test
Tests: cargo clippy --release --all-targets
Define relocation constants and diagnostic-name functions for each
architecture from a single macro invocation. Keep each macro input to the
relocation name and value, and use stringify! to produce the diagnostic
spelling.
Remove the separate relnames module so relocation names and values can no
longer drift apart.
Validation passes with cargo build --release and cargo clippy --release
--all-targets. Cargo test reports 5360 passed, 636 skipped, and no failures.
Replace the callback-only relocation traversal API with streaming Rust
iterators. Make CrelReader and RelocationIter implement Iterator while
preserving deferred CREL decoding without allocation.
Use ordinary for loops in the architecture backends and gdb-index reader.
Keep a specialized fold implementation so Iterator::for_each can select the
ordinary or CREL representation once.
The full test suite passes with 5360 cases passed, 636 skipped, and no
failures. Clippy is clean.
Five-run firefox-debug medians improved from 0.798 s to 0.765 s at 32
threads and from 11.135 s to 11.002 s with one thread. The C++ reference
at fbaaa75f measured 0.792 s and 11.999 s, respectively.
Configure Rayon's process-global pool after target detection and process
forking. Register the main thread as worker zero so --threads=N limits the
linker to N active threads.
Run the linker directly instead of wrapping its entire body in
ThreadPool::install. This follows mold's command-oriented execution model
and its process-wide TBB parallelism limit.
Rename Rust modules to match their C++ source counterparts where
practical. Use archive_file, cmdline, error, output_chunks,
shrink_sections, and perf while keeping mapped_file, output_file, lto,
and subprocess unchanged.
Remove filename-only comments added during the port. They were not copied
documentation and became redundant once the filenames were aligned.
Update all module paths and target entry points for the renamed modules.
Carry over mold's distribution-specific dependency installer as
install-test-deps.sh. Drop CMake packages that the Rust build no longer uses.
Keep test setup out of the user build instructions. Explain the stable Rust
and native C compiler requirements directly in README.
Copy the current README, manual, logos, and license from mold. Adapt the
build and installation instructions for the Rust implementation.
Add install-mold.sh to install the binary, preload library, manual, license,
and linker symlinks under a configurable PREFIX with DESTDIR support.
Move the two remaining C source files from csrc/ to c/ and update the
build script and LTO documentation to use the new paths. The files are
otherwise byte-identical.
Tests: cargo build --release
Tests: cargo test (pass=5360 skip=636 fail=0)
Tests: cargo clippy --release --all-targets
Tests: cargo fmt --all -- --check
Replace the local allocator wrapper and vendored mimalloc sources with the
rui314/mimalloc_rust repository pinned at:
cc58b72775b8b5bc89ee37752af61fcb4f7c2909
The dependency contains mimalloc 3.5.0 and its overlapping upstream files
are byte-identical to the deleted copy. Update the workspace, allocator
import, lockfile, and README.
Tests: cargo build --release
Tests: cargo test (pass=5360 skip=636 fail=0)
Tests: cargo clippy --release --all-targets
Tests: cargo fmt --all -- --check
Register a custom integration-test target for mold-cli and move the Rust
runner into the mold-tests workspace crate. Keep all 537 shell-test files
byte-for-byte unchanged under tests/cases, run every available target and
QEMU job by default, and stage Cargo-built test artifacts in the active
profile directory.
Remove run-tests.sh and make every workspace crate a default member so
plain cargo test is the complete test entry point.
Use explicit wrapping arithmetic for unsigned C++ operations exposed by
the debug-profile test suite.
Tested with:
cargo test
cargo build --release
cargo clippy --release --all-targets
cargo fmt --all -- --check
Full shell-suite result: 5,996 scheduled, pass=5360, skip=636, fail=0.
Treat every C++ implementation comment at mold commit
fbaaa75f9f as a first-class porting
asset. Restore all 2,179 non-namespace comment occurrences at their
Rust counterparts, including file-level explanations, ABI notes,
examples, TODOs, original typos, inline instruction annotations, and
repeated comments. Add provenance where one Rust implementation
consolidates multiple C++ implementations. Closing namespace labels
are the sole comments without a Rust construct.
Verify that all 537 upstream test files remain byte-for-byte identical
to the pinned C++ tree.
The audit also exposed a few implementation-fidelity gaps: reopen
input mappings before writing a repro archive, reset discarded-COMDAT
symbol indices after owner selection, mirror C++ GOT
dynamic-relocation accounting branches, and reserve version-section
output storage as C++ does. These are faithful translations, not new
algorithms.
Validated with cargo build --release, ./run-tests.sh --all (pass=5360
skip=636 fail=0), cargo clippy --release --all-targets, cargo fmt
--check, and git diff --check.
Mirror C++ ConcurrentMap sorted-entry handling: reserve each shard vector, use unstable sorting, retain direct stable entry references, and test occupancy through the atomic key pointer alone.
Store direct stable NameEntry references in indexed-name records so the table writer does not translate a bucket ID for every public-name occurrence.
Reference C++ mold: fbaaa75f9f
Validation:
- cargo build --release
- ./run-tests.sh --all: pass=5360 skip=636 fail=0
- cargo clippy --release --all-targets
32-thread benchmark, median of five, with sync before and sync plus sleep 1 after every linker invocation:
- chrome-debug-arm64: C++ 1.736 s, Rust 1.834 s (+5.65%)
- chrome-debug-arm64 --no-gdb-index: C++ 1.576 s, Rust 1.592 s (+1.02%)
- chrome-release-arm64: C++ 0.715 s, Rust 0.738 s (+3.22%)
The remaining Chrome debug difference is isolated to TBB low-priority arena behavior; Rayon has no corresponding task-priority feature.
mold demangles with the platform's __cxa_demangle, and FreeBSD's libcxxrt
spells `operator delete[]` as `operator delete []`, so the quoted name in
the test never matched there. Test literal matching of brackets with
`S::operator[](int)` instead, which every demangler spells the same way.
Co-Authored-By: Claude Fable 5 <[email protected]>
So far -z rewrite-endbr was implemented only for x86-64, where it
rewrites unnecessary ENDBR64 landing pads (Intel CET) with NOPs for
functions whose addresses are not taken. arm64 has an equivalent
control-flow-integrity feature, BTI, whose landing pad is `bti c`.
This commit generalizes rewrite_endbr into a template shared by both
targets: the landing pad and NOP byte patterns are selected per-arch
(ENDBR64/NOP on x86-64, `bti c`/`nop` on arm64). The whole-program
analysis that preserves landing pads for address-taken functions is
unchanged.
Unlike x86-64, arm64 needs range extension thunks, and a thunk reaches
its target with an indirect branch (`br x16`). A function that is only
called directly can therefore still be entered indirectly once its call
is routed through a thunk, so we must keep its landing pad. We preserve
the landing pad of every thunked symbol; the thunk symbol set has
already been narrowed to the ones that actually need a thunk by
remove_redundant_thunks().
Note that on arm64 with recent glibc, `main`'s landing pad is
legitimately removed: the CRT passes a local __wrap_main stub (whose
landing pad is kept) to __libc_start_main and reaches main through a
direct branch, which does not require a landing pad.
Also note that code built with -mbranch-protection=standard (BTI +
pointer authentication) starts functions with `paciasp`, which doubles
as a landing pad, so there is no separate `bti c` to rewrite and this
option has no effect on such code.
This was locally tested on an Apple M2 machine running Asahi Linux.
Fixes#1552.
The relocation has a 16-bit displacement from the TOC pointer, but the
ELFv2 backend applied it without a range check and silently truncated
displacements to .toc entries beyond 32 KiB, producing a load from the
wrong address. The ELFv1 backend already checked it; do the same here.
Co-Authored-By: Claude Fable 5 <[email protected]>
FDEs refer to their functions with 32-bit PC-relative relocations, and
.eh_frame_hdr stores 32-bit offsets, but mold applied them without any
range check. A function more than 2 GiB away from .eh_frame, which
happens with a huge text section or with -Ttext or --section-start
placing code far from the default layout, silently got a wrapped-around
FDE that the unwinder cannot use, while GNU ld and lld reject such
links. Those relocations are now range-checked like all others, and so
are the .eh_frame_hdr entries.
The aarch64 long thunk test placed code with unwind tables 4 GiB away
from .eh_frame and relied on the bogus FDEs going unnoticed. It now
compiles the far functions without unwind tables and keeps the far
section fixed while swapping the functions between the sections, so
calls in both directions are still covered.
Co-Authored-By: Claude Fable 5 <[email protected]>
GNU ld takes a quoted name in a version script or a dynamic list as a
literal string, while an unquoted one is a glob pattern. mold dropped
the quotes and treated everything as a pattern, so `"operator
delete[](void*)"` in an `extern "C++"` block never matched anything
because `[]` parsed as an empty character class. Glob metacharacters in
a quoted name are now escaped before the name is handed to the matcher.
Co-Authored-By: Claude Fable 5 <[email protected]>
The tokenizer keeps colons within a token for the C++ scope operator,
so `local:*;` lexed as a single token that the parser then took for a
global symbol pattern named `local:*`, and nothing was localized. The
label is now split off such a token.
Co-Authored-By: Claude Fable 5 <[email protected]>
mold took the instruction set of a branch target from the LSB of the
symbol value alone, but only STT_FUNC symbols encode the instruction
set that way. A Thumb BL to a plain label in hand-written assembly,
which is NOTYPE and has an even address, was rewritten to BLX and
crashed with SIGILL, and a Thumb B.W to it went through a mode-switching
thunk. Like GNU ld, we now switch the instruction set only for function
symbols and for PLT entries, which are always ARM code.
Co-Authored-By: Claude Fable 5 <[email protected]>
With -Os, GCC calls linker-synthesized routines not only to save and
restore general-purpose registers but also floating-point and vector
registers. mold only provided the former, so such links failed with
undefined symbol errors. The new routines use the same layout and
encodings as GNU ld; the section that holds them is renamed to
.save_restore_regs accordingly.
Co-Authored-By: Claude Fable 5 <[email protected]>
With -r, common symbols are not converted to .bss, but to_output_esym()
had no case for them and wrote them out as absolute symbols whose value
was their alignment, so linking the -r output again resolved them to a
bogus fixed address. They are now emitted as SHN_COMMON symbols with the
alignment in st_value, as in the input files.
Co-Authored-By: Claude Fable 5 <[email protected]>
The addend of R_RISCV_ALIGN is the number of NOP bytes the assembler
emitted, which is the requested alignment minus the minimum instruction
size, so the alignment is the smallest power of two greater than the
addend rather than the addend rounded up to a power of two. `.balign 4`
in RVC code (addend 2) and `.balign 8` in non-RVC code (addend 4) were
therefore treated as 2- and 4-byte alignments; in a static link that
left glibc's __sigsetjmp at a 2 mod 4 address.
Co-Authored-By: Claude Fable 5 <[email protected]>
An unresolved weak function resolves to address 0. In a huge executable,
a call site can be more than 2 GiB away from it, so a call relocation
such as R_X86_64_PLT32 cannot encode the displacement and mold reported
an out-of-range error. Such a call is never executed because the program
has to null-check the function first, so we now skip the range check for
call relocations against such symbols and write the truncated value.
This covers x86-64, RISC-V, s390x, SPARC64 and LoongArch; ARM already
rewrites such calls to NOPs, and PowerPC calls go through thunks. Data
references keep their range checks because their results feed the null
check.
This fixes linking Godot's sanitizer build, whose .text ends 70 MB past
the 2 GiB mark.
Fixes https://github.com/rui314/mold/issues/1640
Co-Authored-By: Claude Fable 5 <[email protected]>
Without .sdata, we defined RISC-V's __global_pointer$ as absolute
address 0. Since crt1.o initializes gp with a PC-relative
`lla gp, __global_pointer$`, such an executable could not be linked
more than 2 GiB away from address 0, for example with -Ttext-segment.
We now define the symbol at the beginning of the image, like GNU ld
which defines it relative to .data or .bss in that case. The exact
value doesn't matter because we don't relax gp-relative accesses.
Co-Authored-By: Claude Fable 5 <[email protected]>
reattach_section_pieces() traversed every relocation of every alloc
section once just to count the references to mergeable sections, so
that it could allocate their symbols at once, and then traversed them
again to create the symbols. The count needs only ELF data, so it is
now taken while the sections are parsed, where a CREL relocation table
has just been decoded and is still in cache.
Co-Authored-By: Claude Fable 5 <[email protected]>
The tokenizer tested each byte against six whitespace characters and
three quoting characters in turn. A table makes the scan a single
load and compare per byte, and halves the time to tokenize Chromium's
1.9 MiB response file.
Co-Authored-By: Claude Fable 5 <[email protected]>
The loop looked up every undefined symbol in the --wrap set even when
no --wrap option was given, hashing 5.1 million long C++ names on a
Chromium link, and fetched the thread's ShardedMap bin through TBB's
thread-local lookup once per symbol rather than once per file.
Co-Authored-By: Claude Fable 5 <[email protected]>
create_output_sections() copied each output section's members from its
per-file vectors into the final array on one thread and then freed the
vectors one by one, which took about 10 ms on a Chromium link. Copy
and free per file in parallel instead, computing the alignment in the
same pass.
Co-Authored-By: Claude Fable 5 <[email protected]>
The pass took each symbol's lock once per undefined reference, 5.1
million times on a Chromium link, although nearly every reference is
to a defined symbol, which the pass leaves alone. Popular symbols are
referenced from thousands of files, so their locks bounced between
cores. Find the references to symbols without a definition first,
which needs no lock because the pass does not change defined symbols,
and take the locks only for those.
Chromium release on a Threadripper 7980X at 32 threads: the pass goes
from 21 ms to 2 ms. Output files are bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
When a symbol with hidden visibility resolved to a DSO, symbol
resolution started over from scratch for all symbols with that DSO
definition excluded, and Chromium has three such symbols. Resolution is
independent per symbol, so only those symbols are resolved again, and
if one of them moves to an archive member that is not reachable yet,
liveness is propagated from that member alone. The final resolution
pass after section parsing recomputes everything else as before.
Chromium release on a Threadripper 7980X at 32 threads: resolve_symbols
goes from 143 ms to 115 ms. Output files are bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
Sorting the 44,683 files of a Chromium link into command line order
took 7 ms on one thread at the end of the input reading phase.
Co-Authored-By: Claude Fable 5 <[email protected]>
scan_abs_relocations() gathered each member's absolute relocations into
a vector of its own and then flattened the vectors on one thread. For
.data.rel.ro of a Chromium link that was 147,000 vectors holding 1.1
million entries, and the flattening and freeing took about 10 ms of
serial time. Count the entries per member first and write them to
their final positions in parallel instead.
Co-Authored-By: Claude Fable 5 <[email protected]>
The job that read a thin archive opened all of its members before
feeding them to the reader loop, so a large archive's members were
opened one after another on one thread. Chromium's libblink_core.a
has 2,424. The archive job now only lists the member paths, and each
member's job opens its file, in parallel with the others.
Co-Authored-By: Claude Fable 5 <[email protected]>
Each MergeableSection added itself to its MergedSection's member list
under a mutex from the parallel conversion loop. An output has only a
few merged sections, so on a Chromium link, 32 threads took the same
mutex 150,000 times and spent most of the pass in futex calls. Collect
the members from the files after the loop instead.
Chromium release on a Threadripper 7980X at 32 threads:
create_merged_sections goes from 55 ms to 34 ms and its system time
from 0.74 s to 0.07 s. Output files are bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
mmap(2) takes the process's address space lock, so with tens of
thousands of input files, the calls serialize at a few microseconds
each no matter how many threads make them. Linking Chromium maps 44,683
files, and that serialization alone bounds the input reading phase at
about 220 ms, more than a third of the link. Files up to 32 KiB, which
are 64% of Chromium's inputs by count but 16% by size, are now read
into malloc'ed buffers instead; read(2) takes no such lock, and larger
files are still mmap'ed so that their pages are shared with the page
cache rather than copied.
Chromium release on a Threadripper 7980X at 32 threads: the input
reading phase goes from 228 ms to 108 ms and the whole link from 0.69 s
to 0.56 s, with peak RSS unchanged. Output files are bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
The tokenizer pushed every byte through an ostringstream and copied
each token twice. Tokens without quotes or backslashes, which is nearly
all of them, are now substrings of the mapped file, and whitespace is
tested inline instead of through isspace().
Expanding Chromium's 1.9 MiB response file of 19,000 arguments took
33 ms of single-threaded startup on a Threadripper 7980X; it now takes
5 ms. Output files are bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
scan_abs_relocations() classified an output section's absolute
relocations and copy_buf() applied them in serial loops, although
write_dynrels() already processes them in parallel shards. A PIE
Chromium has 1.1 million of them in .data.rel.ro alone, so each loop
ran for about 30 ms on one thread while the others waited.
Chromium release on a Threadripper 7980X at 32 threads: scan_relocations
goes from 84 ms to 50 ms, copy from 89 ms to 75 ms, and the whole link
from 0.90 s to 0.86 s. Output files are bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
Most relocations refer to a section that another thread has already
visited, yet visit() always did an atomic fetch_or, so popular sections
bounced between cores on every reference. Check the bit with a plain
load first, as Atomic::test_and_set() does; it is never cleared during
marking, so a stale positive answer is still correct.
Chromium release on a Threadripper 7980X at 32 threads: the mark phase
goes from 65 ms to 25 ms, the whole link from 0.96 s to 0.92 s. Output
files are bit-identical.
Co-Authored-By: Claude Fable 5 <[email protected]>
GDB 10 aborts on GCC's DWARF 5 type units even when the executable
contains no .gdb_index section, which broke CI on Debian 11. Probe GDB
with an index-less executable first and skip the test if it cannot
print a type from it.
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01CKdZWtcaghcU7m5zoZtMfN
Emit version 9 indexes when type units are present so GDB 12 falls
back instead of interpreting DWARF 5 type-unit offsets as
.debug_types. Keep version 7 for indexes without type units.
Allow the regression test to run with older readelf versions and skip
it under TSAN, where relocatable links hit known oneTBB races.
With -gdwarf-5 -fdebug-types-section, GCC moves each type into its own
type unit and emits it as a one-section COMDAT .debug_info contribution,
so an object file contains several .debug_info sections. We assumed one
.debug_info section per file containing nothing but compilation units:
type units were missing from the index, and a pubnames offset could be
interpreted relative to the wrong contribution.
Read every live .debug_info contribution instead, classifying units by
their header's unit_type. Type units are collected into the .gdb_index
type-unit table along with their signatures, and each pubnames set is
matched to its contribution through the relocation on the set header's
debug_info_offset field. COMDAT group selection has already discarded
duplicate type units by the time we read them, so exactly one copy of
each type unit is indexed.
Fixes https://github.com/rui314/mold/issues/1634
Co-Authored-By: Claude Fable 5 <[email protected]>
If an IR file claims a COMDAT group, the LTO result provides the group's
contents. However, post-LTO COMDAT group selection started from a clean
slate, so an archive member extracted only after LTO could win a claimed
group, resurrecting a copy that collides with the LTO result.
Remember the claim on the signature symbol, and let only LTO-generated
files win such a group after LTO. LLVM keeps claimed groups in its
output; GCC emits their contents without a group, so no file wins.
Fixes https://github.com/rui314/mold/issues/1637
Co-Authored-By: Claude Fable 5 <[email protected]>
shrink_section() fills r_deltas once, and the later passes only read it,
so it does not need to be a growable vector afterwards. Build the deltas
in a local vector and copy them into the arena. This shrinks InputSection
by eight bytes on RISC-V and LoongArch and makes the class trivially
destructible on every target, so InputSection objects can live in the
arena without an owner that runs their destructors.
Co-Authored-By: Claude Fable 5 <[email protected]>
Several InputSection booleans were stored separately even though one
atomic byte is sufficient. Pack liveness, GC visitation, address
significance, decompression, and ICF-removal state into flags, and add
relaxed fetch_or/fetch_and operations for updating them.
Also remove fde_end by marking the final FDE in each section range in
existing FdeRecord padding. This reduces InputSection<X86_64> from 56 to
48 bytes without increasing FdeRecord or Symbol.
The output offset, dense ICF index, and arena-relative ICF leader are
needed at disjoint points in the link. Keeping separate members wastes
space and requires a separate eligibility flag.
Store them in a union, using icf_idx == -1 to represent ineligibility.
Process the gathered ICF sections directly and retain icf_leader only for
eliminated sections after ICF, allowing leaders to restore their offsets.
Counting distinct ICF classes costs about as much as a digest
propagation round, but it was done after every round. Since refinement
only splits classes, convergence does not need to be checked that often.
Run three propagation rounds between class counts and move digest-buffer
swapping into propagate(). This reduces counting overhead while preserving
the final partition.
When an ODR violation leaves a relocation referring to a discarded COMDAT
section, ObjectFile::scan_relocations() reports an error, but
scan_relocations() continues with the absolute-relocation pass. That pass
assumes symbols are resolved and may dereference a null Symbol::file,
causing mold to crash instead of exiting with the diagnostic.
Add a checkpoint between the two relocation passes so mold exits before
inconsistent symbol state reaches the absolute-relocation scan.
Fixes https://github.com/rui314/mold/issues/1635
create_comdat_group_sections assumed that a relocation section in a
comdat group is always SHT_RELA or SHT_REL. If an input file uses the
CREL format, the relocation sections are SHT_CREL instead, so we took
the branch for ordinary member sections and dereferenced a null
pointer, as mold does not create an InputSection for a relocation
section. mold -r had been crashing on such files since CREL input
support was added in mold 2.38.0.
Recognize SHT_CREL the same way as SHT_RELA/SHT_REL so that the
member's relocation section is propagated to the output group.
Co-Authored-By: Claude Fable 5 <[email protected]>
Commit 35f4a334 changed Symbol::is_local() to check visibility and
version-script localization, but get_output_sym_idx() also relies on it
to decide whether a symbol's slot offset is relative to the local or the
global partition of .symtab. An input-local symbol has default
visibility and no version, so is_local() returned false for it, and a
relocation emitted by --emit-relocs against a named local symbol
referred to an arbitrary global symbol.
This went unnoticed on most targets because GAS usually redirects
relocations against local symbols to section symbols, which take a
different code path. On LoongArch and RISC-V, the assembler keeps them
as-is because linker relaxation may change section-relative offsets, so
relocations against symbols like .LC0 were visibly wrong.
Fixes https://github.com/rui314/mold/issues/1631
Compilers may emit a meaningless FDE covering a zero-length address
range, e.g. for a function fragment that hot/cold splitting has
entirely optimized away. Such an FDE begins at the same address as the
FDE of the next real function, giving the binary search table in
.eh_frame_hdr duplicate keys. If a search lands on the empty FDE
instead of the real one, the unwinder fails to find an existing
exception handler and aborts the process.
We now write a tombstone value to .eh_frame_hdr for such an FDE. The
tombstone compares greater than any real address, so a binary search
never returns it. The FDE itself is harmless because an empty address
range doesn't match any program counter.
AArch64 range extension thunks are small and unlikely to benefit
measurably from aligning every entry to a 16-byte boundary. Padding
every entry with brk instructions wastes meaningful space when an
output contains many thunks.
Keep each thunk group 8-byte aligned, but pack its entries tightly so
the ARM64 short and long forms occupy 12 and 24 bytes instead of 16
and 32 bytes.
2026-08-12 20:02:27 +09:00
3733 changed files with 51547 additions and 804860 deletions
This page explains the following warning message and how to fix it.
mold emits this message when it sees an object file that may not be
compatible with mold.
```
mold: warning: foo.o: this file may cause a segmentation fault because it requires an executable stack. See https://github.com/rui314/mold/tree/main/docs/execstack.md for more info.
```
# Background
On modern computers, the stack area (to which local variables are
stored) cannot contain executable code. If the control reaches the
stack area, the CPU refuses to execute any code there and the program
is usually terminated due to segmentation fault.
This is a security measure. The stack area used to be executable (old
CPUs generally execute any code as long as it is in a readable memory
region), but that provided an easy attack vector to a malicious user.
They wrote executable code to the stack area using some buffer
overflow bug and jumped there to run arbitrary code in a remote server
process.
To prevent this type of attack, the stack area is no longer executable
since the early 2000s. On Linux, the stack's executable-ness is
controlled by a bit in an executable, and the loader respects that
bit. The bit is set by the linker.
GCC had (and still has) a feature that depends on the executable
stack, so they invented a way to tell the linker to mark the stack
executable. Specifically, if an object file contains a
`.note.GNU-stack` section with the `SHF_EXECSTACK` bit, GNU linker
silently makes the stack of an output file executable.
But the GNU linker's behavior is dangerous. If you accidentally link
an object file that has that marker section, the entire stack area
silently becomes executable, disabling the security mechanism.
Therefore, mold simply ignores that marker section. If you are using
mold, you need to explicitly pass `-z execstack` to the linker to make
@@ -48,9 +48,11 @@ Whether good or bad, you should keep these semantics in mind to understand Unix
.P
\fBmold\fR does not have any host\-specific default settings\. This is contrary to the GNU linkers, for which some configurable values, such as system\-dependent library search paths, are hard\-coded\.\fBmold\fR depends only on its command\-line arguments\.
.SH"OPTION NOTATIONS"
Multi\-letter long options may precede either a single dash or double dashes, except for those starting with the letter "o"\. For historical reasons, long options beginning with "o" must precede double dashes\.
Long options are spelled with two dashes\. Some also accept a single dash, but which ones do is somewhat arbitrary for historical reasons\.\fBmold\fR follows the conventions of other linkers for compatibility\. For example, \fB\-\-export\-dynamic\fR can be spelled \fB\-export\-dynamic\fR, while \fB\-\-execute\-only\fR requires two dashes\.
.P
For example, you can spell\fB\-\-as\-needed\fR as \fB\-as\-needed\fR, but \fB\-\-omagic\fR must not be spelled as \fB\-omagic\fR\.\fB\-omagic\fRwill be interpreted not as \fB\-\-omagic\fR but as \fB\-o magic\fR\.
This can cause unexpected behavior when constructing command\-line options in a script\.For example, you might use\fB"\-e$foo"\fR to set the entry point symbol to the value of \fB$foo\fR\. However, if \fB$foo\fR is \fBxport\-dynamic\fR, the argument becomes \fB\-export\-dynamic\fRand is interpreted as \fB\-\-export\-dynamic\fR\.
.P
To avoid such ambiguities, we recommend always using two dashes for long options and separating a single\-letter option from its argument with a space\. For example, use \fB\-e $foo\fR or \fB\-\-entry=$foo\fR to specify the entry point symbol in a script\.
.SH"MOLD\-SPECIFIC OPTIONS"
.TP
\fB\-\-chroot\fR=\fIdir\fR
@@ -152,7 +154,9 @@ By default, the compiler always emits a landing pad at the beginning of each glo
.IP
If \fB\-\-rewrite\-endbr\fR is given, mold conducts a whole program analysis to identify functions whose addresses are actually taken and rewrites landing pads with no\-ops for non\-address\-taken functions, reducing the attack surface\.
.IP
This feature is currently available only on x86\-64\.
This feature is currently available only on x86\-64 and arm64\.
.IP
On arm64, the landing pad is \fBbti c\fR\. Note that code compiled with \fB\-mbranch\-protection=standard\fR (BTI + pointer authentication) starts functions with \fBpaciasp\fR, which doubles as a landing pad, so there is no separate \fBbti c\fR to rewrite and this option has no effect on such code\.
.SH"GNU\-COMPATIBLE OPTIONS"
.TP
\fB\-\-help\fR
@@ -513,20 +517,20 @@ If one memory page contains multiple segments, the page protection bits are set
.IP
\fBseparate\-loadable\-segments\fR adds paddings between segments with different attributes so that they do not share the same page\.
.IP
\fBseparate\-code\fR adds paddings only between executable and non\-executable segments\. This is the default\.
\fBseparate\-code\fR adds paddings only between executable and non\-executable segments\.
.IP
\fBnoseparate\-code\fR does not add any paddings between segments\.
\fBnoseparate\-code\fR does not add any paddings between segments\. This is the default\.
.TP
\fB\-z defs\fR, \fB\-z nodefs\fR
\fB\-z defs\fR, \fB\-z undefs\fR
Report undefined symbols (even with \fB\-\-shared\fR)\.
.TP
\fB\-z shstk\fR
Enforce shadow stack by turning \fBGNU_PROPERTY_X86_FEATURE_1_SHSTK\fR bit in \fB\.note\.gnu\.property\fR output section\. Shadow stack is part of Intel Control\-flow Enforcement Technology (CET), which is available since Tiger Lake (2020)\.
If a section name is valid as a C identifier (i\.e\., it matches \fB/^[_a\-zA\-Z][_a\-zA\-Z0\-9]*$/\fR), mold creates \fB__start_SECNAME\fR and \fB__stop_SECNAME\fR symbols to mark the beginning and end of the section, where \fBSECNAME\fR is the section name\.
.IP
You can make these marker symbols visible from other ELF modules by passing \fB\-z start_stop_visibility=protected\fR\. Default is \fBhidden\fR\.
You can make these marker symbols visible from other ELF modules by passing \fB\-z start\-stop\-visibility=protected\fR\. Default is \fBhidden\fR\.
\fBmold\fR by default reports an error if dynamic relocations are created in read\-only sections\. If \fB\-z notext\fR or \fB\-z textoff\fR are given, \fBmold\fR creates such dynamic relocations without reporting an error\.\fB\-z text\fR restores the default behavior\.
Loaded 100 of 3733 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Rui Ueyama
