15 Commits

Author SHA1 Message Date
Stanislav Mekhanoshin
cb2a2f79ae [AMDGPU] Do not combine dpp mov reading physregs
We cannot be sure physregs will stay unchanged.

Differential Revision: https://reviews.llvm.org/D69065

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@375033 91177308-0d34-0410-b5e6-96231b3b80d8
2019-10-16 19:28:25 +00:00
Stanislav Mekhanoshin
139b3c56f6 [AMDGPU] Do not combine dpp with physreg def
We will remove dpp mov along with the physreg def otherwise.

Differential Revision: https://reviews.llvm.org/D69063

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@375030 91177308-0d34-0410-b5e6-96231b3b80d8
2019-10-16 18:48:54 +00:00
Stanislav Mekhanoshin
e7c9c5bebd [AMDGPU] Support mov dpp with 64 bit operands
We define mov/update dpp intrinsics as overloaded but do not
support i64, which is a practically useful type. Fix the
selection and lowering.

Differential Revision: https://reviews.llvm.org/D68673

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374910 91177308-0d34-0410-b5e6-96231b3b80d8
2019-10-15 16:41:15 +00:00
Stanislav Mekhanoshin
325a4c2456 [AMDGPU] Allow DPP combiner to work with REG_SEQUENCE
Differential Revision: https://reviews.llvm.org/D68828

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374908 91177308-0d34-0410-b5e6-96231b3b80d8
2019-10-15 16:17:50 +00:00
Stanislav Mekhanoshin
a485f5b7ea [AMDGPU] Handle undef old operand in DPP combine
It was missing an undef flag.

Differential Revision: https://reviews.llvm.org/D68813

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374455 91177308-0d34-0410-b5e6-96231b3b80d8
2019-10-10 21:32:41 +00:00
Stanislav Mekhanoshin
7d6a3303df [AMDGPU] Fixed dpp combine of VOP1
If original instruction did not have source modifiers they were
not added to the new DPP instruction as well, even if needed.

Differential Revision: https://reviews.llvm.org/D68729

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@374241 91177308-0d34-0410-b5e6-96231b3b80d8
2019-10-09 22:02:58 +00:00
Jay Foad
7f18dddc3d [AMDGPU] Fix DPP combiner check for exec modification
Summary:
r363675 changed the exec modification helper function, now called
execMayBeModifiedBeforeUse, so that if no UseMI is specified it checks
all instructions in the basic block, even beyond the last use. That
meant that the DPP combiner no longer worked in any basic block that
ended with a control flow instruction, and in particular it didn't work
on code sequences generated by the atomic optimizer.

Fix it by reinstating the old behaviour but in a new helper function
execMayBeModifiedBeforeAnyUse, and limiting the number of instructions
scanned.

Reviewers: arsenm, vpykhtin

Subscribers: kzhuravl, nemanjai, jvesely, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, hiraditya, kbarton, MaskRay, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64393

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@365910 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-12 15:59:40 +00:00
Jay Foad
9ec87924ff [AMDGPU] DPP combiner: recognize identities for more opcodes
Summary:
This allows the DPP combiner to kick in more often. For example the
exclusive scan generated by the atomic optimizer for a divergent atomic
add used to look like this:

        v_mov_b32_e32 v3, v1
        v_mov_b32_e32 v5, v1
        v_mov_b32_e32 v6, v1
        v_mov_b32_dpp v3, v2  wave_shr:1 row_mask:0xf bank_mask:0xf
        s_nop 1
        v_add_u32_dpp v4, v3, v3  row_shr:1 row_mask:0xf bank_mask:0xf bound_ctrl:0
        v_mov_b32_dpp v5, v3  row_shr:2 row_mask:0xf bank_mask:0xf
        v_mov_b32_dpp v6, v3  row_shr:3 row_mask:0xf bank_mask:0xf
        v_add3_u32 v3, v4, v5, v6
        v_mov_b32_e32 v4, v1
        s_nop 1
        v_mov_b32_dpp v4, v3  row_shr:4 row_mask:0xf bank_mask:0xe
        v_add_u32_e32 v3, v3, v4
        v_mov_b32_e32 v4, v1
        s_nop 1
        v_mov_b32_dpp v4, v3  row_shr:8 row_mask:0xf bank_mask:0xc
        v_add_u32_e32 v3, v3, v4
        v_mov_b32_e32 v4, v1
        s_nop 1
        v_mov_b32_dpp v4, v3  row_bcast:15 row_mask:0xa bank_mask:0xf
        v_add_u32_e32 v3, v3, v4
        s_nop 1
        v_mov_b32_dpp v1, v3  row_bcast:31 row_mask:0xc bank_mask:0xf
        v_add_u32_e32 v1, v3, v1
        v_add_u32_e32 v1, v2, v1
        v_readlane_b32 s0, v1, 63

But now most of the dpp movs are combined into adds:

        v_mov_b32_e32 v3, v1
        v_mov_b32_e32 v5, v1
        s_nop 0
        v_mov_b32_dpp v3, v2  wave_shr:1 row_mask:0xf bank_mask:0xf
        s_nop 1
        v_add_u32_dpp v4, v3, v3  row_shr:1 row_mask:0xf bank_mask:0xf bound_ctrl:0
        v_mov_b32_dpp v5, v3  row_shr:2 row_mask:0xf bank_mask:0xf
        v_mov_b32_dpp v1, v3  row_shr:3 row_mask:0xf bank_mask:0xf
        v_add3_u32 v1, v4, v5, v1
        s_nop 1
        v_add_u32_dpp v1, v1, v1  row_shr:4 row_mask:0xf bank_mask:0xe
        s_nop 1
        v_add_u32_dpp v1, v1, v1  row_shr:8 row_mask:0xf bank_mask:0xc
        s_nop 1
        v_add_u32_dpp v1, v1, v1  row_bcast:15 row_mask:0xa bank_mask:0xf
        s_nop 1
        v_add_u32_dpp v1, v1, v1  row_bcast:31 row_mask:0xc bank_mask:0xf
        v_add_u32_e32 v1, v2, v1
        v_readlane_b32 s0, v1, 63

Reviewers: arsenm, vpykhtin

Subscribers: kzhuravl, nemanjai, jvesely, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, hiraditya, kbarton, MaskRay, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64207

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@365211 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-05 14:52:48 +00:00
Jay Foad
16da25febf Fix typos in comments and debug output.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@365146 91177308-0d34-0410-b5e6-96231b3b80d8
2019-07-04 15:04:29 +00:00
Matt Arsenault
9f1f314a53 AMDGPU: Change API for checking for exec modification
Invert the name and return value to better reflect the imprecise
nature.

Force passing in the DefMI, since it's known in the 2 users and could
possibly fail for an arbitrary vreg.

Allow specifying a specific user instruction. Scan through use
instructions, instead of use operands. Add scan thresholds instead of
searching infinitely.

Stop using a set to track seen uses. I didn't understand this usage,
or why it would not check the last use. I don't think the use list has
any particular order.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@363675 91177308-0d34-0410-b5e6-96231b3b80d8
2019-06-18 12:48:36 +00:00
Valery Pykhtin
3a178e01c4 [AMDGPU] Fix DPP combiner
Differential revision: https://reviews.llvm.org/D55444

dpp move with uses and old reg initializer should be in the same BB.
bound_ctrl:0 is only considered when bank_mask and row_mask are fully enabled (0xF). Otherwise the old register value is checked for identity.
Added add, subrev, and, or instructions to the old folding function.
Kill flag is cleared for the src0 (DPP register) as it may be copied into more than one user.

The pass is still disabled by default.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@353513 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-08 11:59:48 +00:00
Chandler Carruth
6b547686c5 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@351636 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-19 08:50:56 +00:00
Valery Pykhtin
8daf7f9815 Revert "[AMDGPU] Fix DPP combiner"
This reverts commit e3e2923a39, svn revision rL350721

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@350730 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-09 15:21:53 +00:00
Valery Pykhtin
e3e2923a39 [AMDGPU] Fix DPP combiner
Fixed issue with identity values and other cases, f32/f16 identity values to be added later. fma/mac instructions is disabled for now.
Test is fully reworked, added comments. Other fixes:

1. dpp move with uses and old reg initializer should be in the same BB.
2. bound_ctrl:0 is only considered when bank_mask and row_mask are fully enabled (0xF). Othervise the old register value is checked for identity.
3. Added add, subrev, and, or instructions to the old folding function.
4. Kill flag is cleared for the src0 (DPP register) as it may be copied into more than one user.

Differential revision: https://reviews.llvm.org/D55444

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@350721 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-09 13:43:32 +00:00
Valery Pykhtin
d339265d52 [AMDGPU] Combine DPP mov with use instructions (VOP1/2/3)
Introduces DPP pseudo instructions and the pass that combines DPP mov with subsequent uses.

Differential revision: https://reviews.llvm.org/D53762

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@347993 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-30 14:21:56 +00:00