RPCSX/llvm
1
0
Fork 0
mirror of https://github.com/RPCSX/llvm.git synced 2024-11-24 12:19:53 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

[instsimplify] Fix incorrect folding of an ordered fcmp with a vector of all NaN.

Browse Source 
This patch fixes a crash caused by an incorrect folding of an ordered comparison
between a packed floating point vector and a splat vector of NaN.

An ordered comparison between a vector and a constant vector of NaN, should
always be folded into a constant vector where each element is i1 false.

Since revision 266175, SimplifyFCmpInst folds the ordered fcmp into a scalar
'false'. Later on, this would cause an assertion failure, since the value type
of the folded value doesn't match the expected value type of the uses of the
original instruction: "Assertion failed: New->getType() == getType() &&
"replaceAllUses of value with new value of different type!".

This patch fixes the issue and adds a test case to the already existing test
InstSimplify/floating-point-compares.ll.

Differential Revision: https://reviews.llvm.org/D24143


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280488 91177308-0d34-0410-b5e6-96231b3b80d8
This commit is contained in:
Andrea Di Biagio 2016-09-02 14:47:43 +00:00
parent 997fee1acc
commit 74f53f3812
2 changed files with 9 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
lib/Analysis/InstructionSimplify.cpp
View File

@ -3235,7 +3235,7 @@ static Value *SimplifyFCmpInst(unsigned Predicate, Value *LHS, Value *RHS,
// If the constant is a nan, see if we can fold the comparison based on it.
if (CFP->getValueAPF().isNaN()) {
if (FCmpInst::isOrdered(Pred)) // True "if ordered and foo"
return ConstantInt::getFalse(CFP->getContext());
return ConstantInt::get(GetCompareTy(LHS), 0);
assert(FCmpInst::isUnordered(Pred) &&
"Comparison must be either ordered or unordered!");
// True if unordered.

8
test/Transforms/InstSimplify/floating-point-compare.ll
View File

@ -123,3 +123,11 @@ define i1 @nonans2(double %in1, double %in2) {
%cmp = fcmp nnan ord double %in1, %in2
ret i1 %cmp
}
define <2 x i1> @orderedCompareWithNaNVector(<2 x double> %A) {
; CHECK-LABEL: @orderedCompareWithNaNVector(
; CHECK: ret <2 x i1> zeroinitializer
%cmp = fcmp olt <2 x double> %A, <double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF>
ret <2 x i1> %cmp
}