[libs] Simplify `unchecked_{shl,shr}`
There's no need for the `const_eval_select` dance here. And while I originally wrote the `.try_into().unwrap_unchecked()` implementation here, it's kinda a mess in MIR -- this new one is substantially simpler, as shown by the old one being above the inlining threshold but the new one being below it in the `mir-opt/inline/unchecked_shifts` tests.
We don't need `u32::checked_shl` doing a dance through both `Result` *and* `Option` 🙃
There's no need for the `const_eval_select` dance here. And while I originally wrote the `.try_into().unwrap_unchecked()` implementation here, it's kinda a mess in MIR -- this new one is substantially simpler, as shown by the old one being above the inlining threshold but the new one being below it.
Fix codegen test suite for bare-metal-like targets
For Ferrocene I needed to run the test suite for custom target with no unwinding and static relocation. Running the tests uncovered ~20 failures due to the test suite not accounting for these options. This PR fixes them by:
* Fixing `CHECK`s to account for functions having extra LLVM IR attributes (in this case `nounwind`).
* Fixing `CHECK`s to account for the `dso_local` LLVM IR modifier, which is [added to every item when relocation is static](https://github.com/rust-lang/rust/blob/2d62f71710b39960039820136a0be19ac4003d45/compiler/rustc_codegen_llvm/src/mono_item.rs#L139-L142).
* Fixing `CHECK`s to account for missing `uwtables` attributes.
* Added the `needs-unwind` attributes for tests that are designed to check unwinding.
There is no part of Rust CI that checks this unfortunately, and testing whether the PR works locally is kinda hard because you need a target with std enabled but no unwinding and static relocations. Still, this works in my local testing, and if future PRs accidentally break this Ferrocene will take care of sending followup PRs.
Enable ScalarReplacementOfAggregates in optimized builds
Like MatchBranchSimplification, this pass is known to produce significant runtime improvements in Cranelift artifacts, and I believe based on the perf runs here that the primary effect of this pass is to empower MatchBranchSimplification. ScalarReplacementOfAggregates on its own has little effect on anything, but when this was rebased up to include https://github.com/rust-lang/rust/pull/112001 we started seeing significant and majority-positive results.
Based on the fact that we see most of the regressions in debug builds (https://github.com/rust-lang/rust/pull/112002#issuecomment-1566270144) and some rather significant ones in cycles and wall time, I'm only enabling this in optimized builds at the moment.
These tend to have special handling in a bunch of places anyway, so the variant helps remember that. And I think it's easier to grok than non-Scalar Aggregates sometimes being `Immediates` (like I got wrong and caused 109992). As a minor bonus, it means we don't need to generate poison LLVM values for them to pass around in `OperandValue::Immediate`s.
Only rewrite valtree-constants to patterns and keep other constants opaque
Now that we can reliably fall back to comparing constants with `PartialEq::eq` to the match scrutinee, we can
1. eagerly try to convert constants to valtrees
2. then deeply convert the valtree to a pattern
3. if the to-valtree conversion failed, create an "opaque constant" pattern.
This PR specifically avoids any behavioral changes or major cleanups. What we can now do as follow ups is
* move the two remaining call sites to `destructure_mir_constant` off that query
* make valtree to pattern conversion infallible
* this needs to be done after careful analysis of the effects. There may be user visible changes from that.
based on https://github.com/rust-lang/rust/pull/111768
Optimize scalar and scalar pair representations loaded from ByRef in llvm
in https://github.com/rust-lang/rust/pull/105653 I noticed that we were generating suboptimal LLVM IR if we had a `ConstValue::ByRef` that could be represented by a `ScalarPair`. Before https://github.com/rust-lang/rust/pull/105653 this is probably rare, but after it, every slice will go down this suboptimal code path that requires LLVM to untangle a bunch of indirections and translate static allocations that are only used once to read a scalar pair from.
`[T; N]::zip` is "eager" but most zips are mapped.
This causes poor optimization in generated code.
This is a fundamental design issue and "zip" is
"prime real estate" in terms of function names,
so let's free it up again.
Adds support for LLVM [SafeStack] which provides backward edge control
flow protection by separating the stack into two parts: data which is
only accessed in provable safe ways is allocated on the normal stack
(the "safe stack") and all other data is placed in a separate allocation
(the "unsafe stack").
SafeStack support is enabled by passing `-Zsanitizer=safestack`.
[SafeStack]: https://clang.llvm.org/docs/SafeStack.html
Support #[global_allocator] without the allocator shim
This makes it possible to use liballoc/libstd in combination with `--emit obj` if you use `#[global_allocator]`. This is what rust-for-linux uses right now and systemd may use in the future. Currently they have to depend on the exact implementation of the allocator shim to create one themself as `--emit obj` doesn't create an allocator shim.
Note that currently the allocator shim also defines the oom error handler, which is normally required too. Once `#![feature(default_alloc_error_handler)]` becomes the only option, this can be avoided. In addition when using only fallible allocator methods and either `--cfg no_global_oom_handling` for liballoc (like rust-for-linux) or `--gc-sections` no references to the oom error handler will exist.
To avoid this feature being insta-stable, you will have to define `__rust_no_alloc_shim_is_unstable` to avoid linker errors.
(Labeling this with both T-compiler and T-lang as it originally involved both an implementation detail and had an insta-stable user facing change. As noted above, the `__rust_no_alloc_shim_is_unstable` symbol requirement should prevent unintended dependence on this unstable feature.)
CFI: Fix encode_region: unexpected ReEarlyBound(0, 'a)
Fixes#111515 and complements #106547 by adding support for encoding early bound regions and also excluding projections when transforming trait objects' traits into their identities before emitting type checks.
Fixes#111515 and complements #106547 by adding support for encoding
early bound regions and also excluding projections when transforming
trait objects' traits into their identities before emitting type checks.
Fix some issues with folded AArch64 features
In #91608 the `fp` feature was removed for AArch64 and folded into the `neon` feature, however disabling the `neon` feature doesn't actually disable the `fp` feature. If my understanding on that thread is correct it should do.
While doing this, I also noticed that disabling some features would disable features that it shouldn't. For instance enabling `sve` will enable `neon`, however, when disabling `sve` it would then also disable `neon`, I wouldn't expect disabling `sve` to also disable `neon`.
cc `@workingjubilee`
When compiling with panic=abort (or using a target that doesn't have
unwinding support), the compiler adds the "nounwind" attribute to
functions. This results in a different LLVM IR, which results in a #NNN
added after the function name:
tail call void @bar() #13, !dbg !467
attributes #13 = { nounwind }
...instead of:
tail call void @bar(), !dbg !467
This commit changes the matchers to swallow the #NNN, as it's not needed
for these specific tests.
Rollup of 6 pull requests
Successful merges:
- #111461 (Fix symbol conflict diagnostic mistakenly being shown instead of missing crate diagnostic)
- #111579 (Also assume wrap-around discriminants in `as` MIR building)
- #111704 (Remove return type sized check hack from hir typeck)
- #111853 (Check opaques for mismatch during writeback)
- #111854 (rustdoc: clean up `settings.css`)
- #111860 (Don't ICE if method receiver fails to unify with `arbitrary_self_types`)
r? `@ghost`
`@rustbot` modify labels: rollup
[rustc_ty_utils] Treat `drop_in_place`'s *mut argument like &mut when adding LLVM attributes
This resurrects PR #103614, which has sat idle for a while.
This could probably use a new perf run, since we're on a new LLVM version now.
r? `@oli-obk`
cc `@RalfJung`
---
LLVM can make use of the `noalias` parameter attribute on the parameter to `drop_in_place` in areas like argument promotion. Because the Rust compiler fully controls the code for `drop_in_place`, it can soundly deduce parameter attributes on it.
In #103957, Miri was changed to retag `drop_in_place`'s argument as if it was `&mut`, matching this change.
Fix duplicate `arcinner_layout_for_value_layout` calls when using the uninit `Arc` constructors
What this fixes is the duplicate calls to `arcinner_layout_for_value_layout` seen here: https://godbolt.org/z/jr5Gxozhj
The issue was discovered alongside #111603 but is otherwise unrelated to the duplicate `alloca`s, which remain unsolved. Everything I tried to solve said main issue has failed.
As for the duplicate layout calculations, I also tried slapping `#[inline]` and `#[inline(always)]` on everything in sight but the only thing that worked in the end is to dedup the calls by hand.
Rather than returning an array of features from to_llvm_features, return a structure that contains
the dependencies. This also contains metadata on how the features depend on each other to allow for
the correct enabling and disabling.
Some features that are tied together only make sense to be folded
together when enabling the feature. For example on AArch64 sve and
neon are tied together, however it doesn't make sense to disable neon
when disabling sve.
CFI: Fix encode_ty: unexpected Param(B/#1)
Fixes#111510 and complements #106547 by adding support for encoding type parameters and also by transforming trait objects' traits into their identities before emitting type checks.
The CHECK, -NOT, -SAME pattern ensures that the `CHECK-NOT: noalias`
is limited to only one line, and won't match unrelated lines further
down in the file.
Explicit drop call added to preserve the `foo` argument name, since
names of unused arguments are not preserved.
We've done measurements with Miri and have determined that `noalias` shouldn't
break code. The requirements that allow us to add dereferenceable and align
have been long documented in the standard library documentation.
LLVM can make use of the `noalias` parameter attribute on the parameter to
`drop_in_place` in areas like argument promotion. Because the Rust compiler
fully controls the code for `drop_in_place`, it can soundly deduce parameter
attributes on it. In the case of a value that has a programmer-defined Drop
implementation, we know that the first thing `drop_in_place` will do is pass a
pointer to the object to `Drop::drop`. `Drop::drop` takes `&mut`, so it must be
guaranteed that there are no pointers to the object upon entering that
function. Therefore, it should be safe to mark `noalias` there.
With this patch, we mark `noalias` only when the type is a value with a
programmer-defined Drop implementation. This is possibly overly conservative,
but I thought that proceeding cautiously was best in this instance.
Fixes#111510 and complements #106547 by adding support for encoding
type parameters and also by transforming trait objects' traits into
their identities before emitting type checks.
bors