This patch adds initial support for TMR_ABORT_TASK ops for se_cmd
descriptors using se_sess->sess_cmd_list and se_cmd->cmd_kref counting.
It will perform an explict abort for all outstanding se_cmd ops based
upon tmr->ref_task_tag that have not been set CMD_T_COMPLETE.
It will cancel se_cmd->work and wait for backing I/O to complete before
attempting to send SAM_STAT_TASK_ABORTED and perform
target_put_sess_cmd() to release the referenced descriptor.
It also adds a CMD_T_ABORTED check into transport_complete_task() to
catch the completion from backend I/O that has been aborted, and
updates transport_wait_for_tasks() to allow CMD_T_ABORTED usage with
core_tmr_abort_task() context.
Reported-by: Roland Dreier <roland@purestorage.com>
Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
This patch changes target_release_cmd_kref() to make TFO->release_cmd()
call when list_empty(&se_cmd->se_cmd_list) is TRUE. This is required
for TMR_ABORT_TASK operation where the referenced tag descriptor may
have already been pulled of the session command list.
Reported-by: Roland Dreier <roland@purestorage.com>
Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
When TARGET_SCF_ACK_KREF is in use with target_submit_cmd() for
setting the extra acknowledgement reference to se_cmd->cmd_kref,
go ahead and set SCF_ACK_KREF in order to be used later by
abort task.
Reported-by: Roland Dreier <roland@purestorage.com>
Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
transport_generic_request_failure() is a wrapper around calling
transport_send_check_condition_and_sense() that is required once
an se_cmd->cmd_kref has been obtained via target_submit_cmd() ->
target_get_sess_cmd().
tcm_qla2xxx currently requires this, and since it's necessary for
other callers using target_submit_cmd() make it exportable now.
Reported-by: Roland Dreier <roland@purestorage.com>
Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
This code isn't broken per se, but it's scary to look at! It looks like
in the t_data_nents==1 case we're doing both a kunmap and a vunmap,
what's saving us is that t_data_vmap in this case is 0, so vunmap
doesn't do anything.
Return after kunmap, so the handling of the three cases does not overlap.
Signed-off-by: Andy Grover <agrover@redhat.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Now that we use a workqueue for I/O submission there is no need to use
transport_generic_handle_cdb_map any more.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Apply the qla2xxx model of submitting all commands from a workqueue.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
This function makes little sense as a separate abstraction as it's deeply
interwinded with the control flow of its only caller. Merged it into
tcm_loop_queuecommand after factoring out a helper to convert the task
attribute representation.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Change ft_send_tm() make use of the new target_submit_tmr helper
Signed-off-by: Andy Grover <agrover@redhat.com>
Cc: Kiran Patil <kiran.patil@intel.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Similar to target_submit_cmd, this function lets fabrics call one function
(albeit with a lot of parameters) instead of 3 or more.
(nab: Add missing return for transport_lookup_tmr_lun failure)
Signed-off-by: Andy Grover <agrover@redhat.com>
Cc: Kiran Patil <kiran.patil@intel.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
transport_generic_free_cmd will end up calling ft_sess_put, so it should
work just the same.
Signed-off-by: Andy Grover <agrover@redhat.com>
Cc: Kiran Patil <kiran.patil@intel.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Don't see a reason to differentiate, so drop the fabric specific
switch statement in ft_send_tm() ahead of conversion to use
target_submit_tmr().
Signed-off-by: Andy Grover <agrover@redhat.com>
Cc: Kiran Patil <kiran.patil@intel.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
No dependencies on rest of code, let's get tm_func set asap.
Signed-off-by: Andy Grover <agrover@redhat.com>
Cc: Kiran Patil <kiran.patil@intel.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Change the test for if a cmd is a tmr request to checking if
SCF_SCSI_TMR_CDB (a new flag) is set in cmd->se_cmd_flags.
Also remove se_tmr_req_cache usage in favor of kzalloc usage,
and make core_tmr_alloc_req() return int + setup se_cmd->se_tmr_req
directly and fix up various fabric module usages
Cc: Andy Grover <agrover@redhat.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
It's used only for debug output. Debug output may want to make use of
fcp->fc_cdb directly.
Signed-off-by: Andy Grover <agrover@redhat.com>
Cc: Kiran Patil <kiran.patil@intel.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Check fc_tm_flags early and call ft_send_tm() right away. Don't need to
set local vars for tm case.
data_len local var now unneeded, remove.
Signed-off-by: Andy Grover <agrover@redhat.com>
Cc: Kiran Patil <kiran.patil@intel.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
This allows us to use scsilun_to_int without an ugly cast.
Fix up places that use scsilun_to_int on fcp->fc_lun accordingly.
In fc target, this leaves ft_cmd.lun unused, so remove it.
Signed-off-by: Andy Grover <agrover@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Kiran Patil <kiran.patil@intel.com>
Cc: James Bottomley <JBottomley@Parallels.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
It's in SBC-3.
Signed-off-by: Andy Grover <agrover@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: James Bottomley <JBottomley@Parallels.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Instead of
static struct list_head foo;
static struct mutex bar;
...
INIT_LIST_HEAD(&foo);
mutex_init(&bar);
just do
static LIST_HEAD(foo);
static DEFINE_MUTEX(bar);
Also remove some superfluous struct list_head and spinlock_t
initialization calls where the variables are already defined using
macros that initialize them.
This saves a decent amount of compiled code too:
add/remove: 0/0 grow/shrink: 0/3 up/down: 0/-178 (-178)
function old new delta
target_core_init_configfs 898 850 -48
core_scsi3_emulate_pro_preempt 1742 1683 -59
iscsi_thread_set_init 159 88 -71
Signed-off-by: Roland Dreier <roland@purestorage.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
There is no real limit for task sizes in the iblock driver given that we
can chain bios. Increase the maximum size to UINT_MAX, and change the
code to submit bios in a smaller batch size to avoid deadlocks when
having more bios in flight than the pool supports. Also increase the
pool size to always allow multiple tasks to be in flight.
I also had to change the task refcounting to include one reference for
the submission task, which is a standard practice in this kind of code
in Linux (e.g. XFS I/O submission). This was wrong before, but couldn't
be hit easily.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
There is no reason to allocate a struct just to store the host number for
a debug printk in the detach path. I've simply removed the verbose
debugging given that the calling code thinks the number passed in is
something different from a host ID anyway.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
There is no reason to have a flag telling if a command is on the per-lun list,
we can simply do a list_empty check before removing it as long as we're careful
to always use list_del_init.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Replace various atomic_ts used as flags in struct se_cmd with a single
transport_state bitmap that requires t_state_lock to be held for modifications.
In the target core that assumption generally is true, but some recently added
code in the SRP target had to grow new lock calls. I can't say I like the way
how it messes with the command state directly, but let's leave that for later.
(Re-add missing ib_srpt.c changes that nab dropped..)
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
The majority of them are regression fixes for stuff that broke during
the merge 3.3 window.
The notable ones are:
* The at91 ata drivers both broke because of an earlier cleanup patch that
some other patches were based on. Jean-Christophe decided to remove
the legacy at91_ide driver and fix the new-style at91-pata driver while
keeping the cleanup patch. I almost rejected the patches for being too
late and too big but in the end decided to accept them because they
fix a regression.
* A patch fixing build breakage from the sysdev-to-device conversion
colliding with other changes touches a number of mach-s3c files.
* b0654037 "ARM: orion: Fix Orion5x GPIO regression from MPP cleanup"
is a mechanical change that unfortunately touches a lot of lines
that should up in the diffstat.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iQIVAwUATztTM2CrR//JCVInAQJu3Q/+KN4npDjjJbRm1FR4J+z7dEy3631gt7Ku
M64JuC2259da0AtXlHXoc8XB7ZrBkMR2k1n+Q42FqUFVILOXcrHSTId6osPQ8WYE
TGWR0E2APP6/w4YH3dz0aTUauX0HhnWNP4ShWalWxw2Zsc1nhPNcMO3k57E/PNnp
nUHb2ZR+Huqk9Eje6/Vkr7OQq7dhl0KJvITJKCT1H93vVYZc5l2O5ZytcOC3dsFg
yMP/btmu9JlCenOwoKcQFv6ug0tWAYiY4ALqQujLN0kcf7rmjLLOG2HQrnycmeh3
gv9jwK04gYxHkhPbCLCgO/bg906LVcYIl9/TY7jXK8oE4kR0vVxdQOWEzKIX5+KO
dyAuwy3uGi4szG8f1DKnz1h7vR1MEyBVgQ+yRqnfhLh7mFmuZcOlGTzziD3csDXG
qd5B2xf9WvLupfpbvgnHUUKEIJVfWPDoJeN3jGCOjd4+j8OzPR6yeAtU85TDQzIx
IKs2x+0zrYMBre3R+m5vb9v3IhPb1wZU29eXXRzDmLuHJDM00Qc8LmpiWUoeu3cX
DwuLstYLm8EhWN+LnjAABd3mKeR5tyBojK3EsDFRxIfz3mKHVNEAPE6Iky60Lfwr
Pq+LgBBftFfcct70UyXWSK7UI92suavDgCHVejIxpbvIWF1UVY7S1mgmflZ1WZAL
R5tdx6oe5Y4=
=QYVn
-----END PGP SIGNATURE-----
Merge tag 'fixes-3.3-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc
These are the bug fixes that have accumulated since 3.3-rc3 in arm-soc.
The majority of them are regression fixes for stuff that broke during
the merge 3.3 window.
The notable ones are:
* The at91 ata drivers both broke because of an earlier cleanup patch that
some other patches were based on. Jean-Christophe decided to remove
the legacy at91_ide driver and fix the new-style at91-pata driver while
keeping the cleanup patch. I almost rejected the patches for being too
late and too big but in the end decided to accept them because they
fix a regression.
* A patch fixing build breakage from the sysdev-to-device conversion
colliding with other changes touches a number of mach-s3c files.
* b0654037 "ARM: orion: Fix Orion5x GPIO regression from MPP cleanup"
is a mechanical change that unfortunately touches a lot of lines
that should up in the diffstat.
* tag 'fixes-3.3-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc: (28 commits)
ARM: at91: drop ide driver in favor of the pata one
pata/at91: use newly introduced SMC accessors
ARM: at91: add accessor to manage SMC
ARM: at91:rtc/rtc-at91sam9: ioremap register bank
ARM: at91: USB AT91 gadget registration for module
ep93xx: fix build of vision_ep93xx.c
ARM: OMAP2xxx: PM: fix OMAP2xxx-specific UART idle bug in v3.3
ARM: orion: Fix USB phy for orion5x.
ARM: orion: Fix Orion5x GPIO regression from MPP cleanup
ARM: EXYNOS: Add cpu-offset property in gic device tree node
ARM: EXYNOS: Bring exynos4-dt up to date
ARM: OMAP3: cm-t35: fix section mismatch warning
ARM: OMAP2: Fix the OMAP2 only build break seen with 2011+ ARM tool-chains
ARM: tegra: paz00: fix wrong UART port on mini-pcie plug
ARM: tegra: paz00: fix wrong SD1 power gpio
i2c: tegra: Add devexit_p() for remove
ARM: EXYNOS: Correct M-5MOLS sensor clock frequency on Universal C210 board
ARM: EXYNOS: Correct framebuffer window size on Nuri board
ARM: SAMSUNG: Fix missing api-change from subsys_interface change
ARM: EXYNOS: Fix "warning: initialization from incompatible pointer type"
...
1) VETH_INFO_PEER netlink attribute needs to have it's size validated,
from Thomas Graf.
2) 'poll' module option of bnx2x driver crashes the machine, just remove
it. From Michal Schmidt.
3) ks8851_mll driver reads the irq number from two places, but only
initializes one of them, oops. Use only one location and fix this
problem, from Jan Weitzel.
4) Fix buffer overrun and unicast sterring bugs in mellanox mlx4 driver,
from Eugenia Emantayev.
5) Swapped kcalloc() args in RxRPC and mlx4, from Axel Lin.
6) PHY MDIO device name regression fixes from Florian Fainelli.
7) If the wake event IRQ line is different from the netdevice one, we
have to properly route it to the stmmac interrupt handler. From
Francesco Virlinzi.
8) Fix rwlock lock initialization ordering bug in mac80211, from
Mohammed Shafi Shajakhan.
9) TCP lost_cnt can get out of sync, and in fact go negative, in certain
circumstances. Fix the way we specify what sequence range to operate
on in tcp_sacktag_one() to fix this bug. From Neal Cardwell.
* git://git.kernel.org/pub/scm/linux/kernel/git/davem/net: (27 commits)
net/ethernet: ks8851_mll fix irq handling
veth: Enforce minimum size of VETH_INFO_PEER
stmmac: update the driver version to Feb 2012 (v2)
stmmac: move hw init in the probe (v2)
stmmac: request_irq when use an ext wake irq line (v2)
stmmac: do not discard frame on dribbling bit assert
ipheth: Add iPhone 4S
mlx4: add unicast steering entries to resource_tracker
mlx4: fix QP tree trashing
mlx4: fix buffer overrun
3c59x: shorten timer period for slave devices
netpoll: netpoll_poll_dev() should access dev->flags
RxRPC: Fix kcalloc parameters swapped
bnx2x: remove the 'poll' module option
tcp: fix tcp_shifted_skb() adjustment of lost_cnt_hint for FACK
ks8851: Fix NOHZ local_softirq_pending 08 warning
bnx2x: fix bnx2x_storm_stats_update() on big endian
ixp4xx-eth: fix PHY name to match MDIO bus name
octeon: fix PHY name to match MDIO bus name
fec: fix PHY name to match fixed MDIO bus name
...
method for register cache initialisation is used. Only affects a fairly
small proportion of users that both don't use explicit register defaults
and do use the cache.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iQIcBAABAgAGBQJPO94HAAoJEBus8iNuMP3dL+kP/2mfXyC8Nrxp9ussST9cElJA
5S6lnaKVKoA7jSa4MB5QEuMX2C4P9lgKhfZoSOmvbkdgSSU+q18nSYXDDMhgjgWt
4JUxgmXEeloj17DBpi1iEkJaUBPfmpr5HR8Bu7eFsUbXVd3cYKFVmmPBe+QC6Hf8
2MvbkcFrOL7TSA1jwYO/O9oZooE1gEkBDpmo/YgwX0u6hWU2Jc/axGhc2ZXWLTmM
SfPrfAmLq/P6q/RGJX/NPfbLxcPn1nrgv6O8ptgrMeqLjvnWUKfs26tVgiqmzEGi
SQr5oUZKxwRalHd06Q4P1tBB/tErqnYCC7I3ys3b8zKBcDudRfSIMrwOkhn9XObI
ePBMwZb6tjWpzCf2NyKwSDhONEf5puE/n36N9vuyyVMNJCnjiCbYcJ2OrRttqOtw
UDa6ohdH3QzHbIhujvyyE7WM5JiJz+Qtx8/3ZJS6kTEb7oAVPB2luF9e8P7lGfRv
+TypFrYGkAJhPhafkk5Qx29NX24fvIaErlNLPdgKHr+6O+oAizFXx8XkR7xbIibM
8RY3ZKrQNvDSTE3lI1sne1GQdJX/4XlVjgZxv6AS/jIKRpI8FMkOmL8hHK2Ttgaq
lFYeWW8xRD5gtU/ViFbGrFNytMyro/qIVmJ1qVV2IeCz1tdg16oa/3E6F7ubvjHM
a3IvzDF5njnoKpeWFwiA
=pU4C
-----END PGP SIGNATURE-----
Merge tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/broonie/regmap
Fixes a bootstrapping issue for some registers when a less commonly used
method for register cache initialisation is used. Only affects a fairly
small proportion of users that both don't use explicit register defaults
and do use the cache.
* tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/broonie/regmap:
regmap: Fix cache defaults initialization from raw cache defaults
and also fixes stale inode mode bits on eCryptfs inodes after a POSIX ACL was
set on the lower filesystem's inode.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iQIcBAABCgAGBQJPPZzXAAoJENaSAD2qAscKNjsP/0KEqvVV7SXFXq/4yXEij/02
+wHS8/4SsTXcE2fkzQM2Jx/osoZfO0Tse8VMADwpX+aNyYlfnsdI4RLYm0y5X1JL
kKdSKNuuEQaA0+YEge4KicsjMQ6nVmYIHGWs3kpEpkk3dKeafVKgh5/+k+NCOs7U
tZricmWrLV/Rn9pb7S1tp4hOo4cVQEJQGPqaFJ0QwORcdmBWTw904yFsZV+TojFf
1kEV6ue/mFav7ZPOhz8Cx2xIukUHk8EUykbScTWx3kpdkffF3INsf9b3mt34lEFB
chuLdcJZuABBBe8RL4g/qRcDOHNUmZx01uX4eHwejs7sYFDfnmNz+tjomr1MRaCv
JEUgE31n407HR3ChA8IxX1bqHOfjN0mgVnOwnmK4nbnLKX0WvmogaIDV7hZK3Z9Q
lGeKN0qwU3Cr1N9K/+JLHB3xB/yieE/53tlksHDjRgJAEXy7SV0fcftmgDLxyx5m
DX4u+eFKdFt+3YBFzTBWRtDO5GVgbylT31JVnKJUi+RDFiOWUf0PzAOXUp5nmr8w
Md8zZ/kCZY/pbnrETNVcXPrEP9eDb6nY5b16ZP5nADMRtZLQdKiKRRdHtwnKg5N6
7nU/cS1bpG7bUekJkkRgzrAz+ziDielzVvkhwb7sq3sac9fegjW67K3/SEiJiQJC
XTBZCrTW3RdEAt6WCTVV
=hwGP
-----END PGP SIGNATURE-----
Merge tag 'ecryptfs-3.3-rc4-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tyhicks/ecryptfs
Fixes maximum filename length and filesystem type reporting in statfs() calls
and also fixes stale inode mode bits on eCryptfs inodes after a POSIX ACL was
set on the lower filesystem's inode.
* tag 'ecryptfs-3.3-rc4-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tyhicks/ecryptfs:
ecryptfs: remove the second argument of k[un]map_atomic()
eCryptfs: Copy up lower inode attrs after setting lower xattr
eCryptfs: Improve statfs reporting
Here are a few more fixes for powerpc. Some are regressions, the rest
is simple/obvious/nasty enough that I deemed it good to go now.
Here's also step one of deprecating legacy iSeries support: we are
removing it from the main defconfig.
Nobody seems to be using it anymore and the code is nasty to maintain,
(involves horrible hacks in various low level areas of the kernel) so we
plan to actually rip it out at some point. For now let's just avoid
building it by default. Stephen will proceed to do the actual removal
later (probably 3.4 or 3.5).
* 'merge' of git://git.kernel.org/pub/scm/linux/kernel/git/benh/powerpc:
powerpc/perf: power_pmu_start restores incorrect values, breaking frequency events
powerpc/adb: Use set_current_state()
powerpc: Disable interrupts early in Program Check
powerpc: Remove legacy iSeries from ppc64_defconfig
powerpc/fsl/pci: Fix PCIe fixup regression
powerpc: Fix kernel log of oops/panic instruction dump
One regression fix for SR-IOV on PPC and a couple of misc fixes from
Yinghai.
* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jbarnes/pci:
PCI: Fix pci cardbus removal
PCI: set pci sriov page size before reading SRIOV BAR
PCI: workaround hard-wired bus number V2
3 radeon fixes, I have some exynos fixes to push later but I'll queue
them separately once I've looked them over a bit.
* 'drm-fixes' of git://people.freedesktop.org/~airlied/linux:
drm/radeon/kms: fix MSI re-arm on rv370+
drm/radeon/kms/atom: bios scratch reg handling updates
drm/radeon/kms: drop lock in return path of radeon_fence_count_emitted.
After all the FPU state cleanups and finally finding the problem that
caused all our FPU save/restore problems, this re-introduces the
preloading of FPU state that was removed in commit b3b0870ef3 ("i387:
do not preload FPU state at task switch time").
However, instead of simply reverting the removal, this reimplements
preloading with several fixes, most notably
- properly abstracted as a true FPU state switch, rather than as
open-coded save and restore with various hacks.
In particular, implementing it as a proper FPU state switch allows us
to optimize the CR0.TS flag accesses: there is no reason to set the
TS bit only to then almost immediately clear it again. CR0 accesses
are quite slow and expensive, don't flip the bit back and forth for
no good reason.
- Make sure that the same model works for both x86-32 and x86-64, so
that there are no gratuitous differences between the two due to the
way they save and restore segment state differently due to
architectural differences that really don't matter to the FPU state.
- Avoid exposing the "preload" state to the context switch routines,
and in particular allow the concept of lazy state restore: if nothing
else has used the FPU in the meantime, and the process is still on
the same CPU, we can avoid restoring state from memory entirely, just
re-expose the state that is still in the FPU unit.
That optimized lazy restore isn't actually implemented here, but the
infrastructure is set up for it. Of course, older CPU's that use
'fnsave' to save the state cannot take advantage of this, since the
state saving also trashes the state.
In other words, there is now an actual _design_ to the FPU state saving,
rather than just random historical baggage. Hopefully it's easier to
follow as a result.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This moves the bit that indicates whether a thread has ownership of the
FPU from the TS_USEDFPU bit in thread_info->status to a word of its own
(called 'has_fpu') in task_struct->thread.has_fpu.
This fixes two independent bugs at the same time:
- changing 'thread_info->status' from the scheduler causes nasty
problems for the other users of that variable, since it is defined to
be thread-synchronous (that's what the "TS_" part of the naming was
supposed to indicate).
So perfectly valid code could (and did) do
ti->status |= TS_RESTORE_SIGMASK;
and the compiler was free to do that as separate load, or and store
instructions. Which can cause problems with preemption, since a task
switch could happen in between, and change the TS_USEDFPU bit. The
change to TS_USEDFPU would be overwritten by the final store.
In practice, this seldom happened, though, because the 'status' field
was seldom used more than once, so gcc would generally tend to
generate code that used a read-modify-write instruction and thus
happened to avoid this problem - RMW instructions are naturally low
fat and preemption-safe.
- On x86-32, the current_thread_info() pointer would, during interrupts
and softirqs, point to a *copy* of the real thread_info, because
x86-32 uses %esp to calculate the thread_info address, and thus the
separate irq (and softirq) stacks would cause these kinds of odd
thread_info copy aliases.
This is normally not a problem, since interrupts aren't supposed to
look at thread information anyway (what thread is running at
interrupt time really isn't very well-defined), but it confused the
heck out of irq_fpu_usable() and the code that tried to squirrel
away the FPU state.
(It also caused untold confusion for us poor kernel developers).
It also turns out that using 'task_struct' is actually much more natural
for most of the call sites that care about the FPU state, since they
tend to work with the task struct for other reasons anyway (ie
scheduling). And the FPU data that we are going to save/restore is
found there too.
Thanks to Arjan Van De Ven <arjan@linux.intel.com> for pointing us to
the %esp issue.
Cc: Arjan van de Ven <arjan@linux.intel.com>
Reported-and-tested-by: Raphael Prevost <raphael@buro.asia>
Acked-and-tested-by: Suresh Siddha <suresh.b.siddha@intel.com>
Tested-by: Peter Anvin <hpa@zytor.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
The AMD K7/K8 CPUs don't save/restore FDP/FIP/FOP unless an exception is
pending. In order to not leak FIP state from one process to another, we
need to do a floating point load after the fxsave of the old process,
and before the fxrstor of the new FPU state. That resets the state to
the (uninteresting) kernel load, rather than some potentially sensitive
user information.
We used to do this directly after the FPU state save, but that is
actually very inconvenient, since it
(a) corrupts what is potentially perfectly good FPU state that we might
want to lazy avoid restoring later and
(b) on x86-64 it resulted in a very annoying ordering constraint, where
"__unlazy_fpu()" in the task switch needs to be delayed until after
the DS segment has been reloaded just to get the new DS value.
Coupling it to the fxrstor instead of the fxsave automatically avoids
both of these issues, and also ensures that we only do it when actually
necessary (the FP state after a save may never actually get used). It's
simply a much more natural place for the leaked state cleanup.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Yes, taking the trap to re-load the FPU/MMX state is expensive, but so
is spending several days looking for a bug in the state save/restore
code. And the preload code has some rather subtle interactions with
both paravirtualization support and segment state restore, so it's not
nearly as simple as it should be.
Also, now that we no longer necessarily depend on a single bit (ie
TS_USEDFPU) for keeping track of the state of the FPU, we migth be able
to do better. If we are really switching between two processes that
keep touching the FP state, save/restore is inevitable, but in the case
of having one process that does most of the FPU usage, we may actually
be able to do much better than the preloading.
In particular, we may be able to keep track of which CPU the process ran
on last, and also per CPU keep track of which process' FP state that CPU
has. For modern CPU's that don't destroy the FPU contents on save time,
that would allow us to do a lazy restore by just re-enabling the
existing FPU state - with no restore cost at all!
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
After passing through a ->setxattr() call, eCryptfs needs to copy the
inode attributes from the lower inode to the eCryptfs inode, as they
may have changed in the lower filesystem's ->setxattr() path.
One example is if an extended attribute containing a POSIX Access
Control List is being set. The new ACL may cause the lower filesystem to
modify the mode of the lower inode and the eCryptfs inode would need to
be updated to reflect the new mode.
https://launchpad.net/bugs/926292
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Reported-by: Sebastien Bacher <seb128@ubuntu.com>
Cc: John Johansen <john.johansen@canonical.com>
Cc: <stable@vger.kernel.org>
statfs() calls on eCryptfs files returned the wrong filesystem type and,
when using filename encryption, the wrong maximum filename length.
If mount-wide filename encryption is enabled, the cipher block size and
the lower filesystem's max filename length will determine the max
eCryptfs filename length. Pre-tested, known good lengths are used when
the lower filesystem's namelen is 255 and a cipher with 8 or 16 byte
block sizes is used. In other, less common cases, we fall back to a safe
rounded-down estimate when determining the eCryptfs namelen.
https://launchpad.net/bugs/885744
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Reported-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: John Johansen <john.johansen@canonical.com>
This creates three helper functions that do the TS_USEDFPU accesses, and
makes everybody that used to do it by hand use those helpers instead.
In addition, there's a couple of helper functions for the "change both
CR0.TS and TS_USEDFPU at the same time" case, and the places that do
that together have been changed to use those. That means that we have
fewer random places that open-code this situation.
The intent is partly to clarify the code without actually changing any
semantics yet (since we clearly still have some hard to reproduce bug in
this area), but also to make it much easier to use another approach
entirely to caching the CR0.TS bit for software accesses.
Right now we use a bit in the thread-info 'status' variable (this patch
does not change that), but we might want to make it a full field of its
own or even make it a per-cpu variable.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Touching TS_USEDFPU without touching CR0.TS is confusing, so don't do
it. By moving it into the callers, we always do the TS_USEDFPU next to
the CR0.TS accesses in the source code, and it's much easier to see how
the two go hand in hand.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Commit 5b1cbac377 ("i387: make irq_fpu_usable() tests more robust")
added a sanity check to the #NM handler to verify that we never cause
the "Device Not Available" exception in kernel mode.
However, that check actually pinpointed a (fundamental) race where we do
cause that exception as part of the signal stack FPU state save/restore
code.
Because we use the floating point instructions themselves to save and
restore state directly from user mode, we cannot do that atomically with
testing the TS_USEDFPU bit: the user mode access itself may cause a page
fault, which causes a task switch, which saves and restores the FP/MMX
state from the kernel buffers.
This kind of "recursive" FP state save is fine per se, but it means that
when the signal stack save/restore gets restarted, it will now take the
'#NM' exception we originally tried to avoid. With preemption this can
happen even without the page fault - but because of the user access, we
cannot just disable preemption around the save/restore instruction.
There are various ways to solve this, including using the
"enable/disable_page_fault()" helpers to not allow page faults at all
during the sequence, and fall back to copying things by hand without the
use of the native FP state save/restore instructions.
However, the simplest thing to do is to just allow the #NM from kernel
space, but fix the race in setting and clearing CR0.TS that this all
exposed: the TS bit changes and the TS_USEDFPU bit absolutely have to be
atomic wrt scheduling, so while the actual state save/restore can be
interrupted and restarted, the act of actually clearing/setting CR0.TS
and the TS_USEDFPU bit together must not.
Instead of just adding random "preempt_disable/enable()" calls to what
is already excessively ugly code, this introduces some helper functions
that mostly mirror the "kernel_fpu_begin/end()" functionality, just for
the user state instead.
Those helper functions should probably eventually replace the other
ad-hoc CR0.TS and TS_USEDFPU tests too, but I'll need to think about it
some more: the task switching functionality in particular needs to
expose the difference between the 'prev' and 'next' threads, while the
new helper functions intentionally were written to only work with
'current'.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
perf on POWER stopped working after commit e050e3f0a7 (perf: Fix
broken interrupt rate throttling). That patch exposed a bug in
the POWER perf_events code.
Since the PMCs count upwards and take an exception when the top bit
is set, we want to write 0x80000000 - left in power_pmu_start. We were
instead programming in left which effectively disables the counter
until we eventually hit 0x80000000. This could take seconds or longer.
With the patch applied I get the expected number of samples:
SAMPLE events: 9948
Signed-off-by: Anton Blanchard <anton@samba.org>
Acked-by: Paul Mackerras <paulus@samba.org>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: <stable@kernel.org>
Program Check exceptions are the result of WARNs, BUGs, some
type of breakpoints, kprobe, and other illegal instructions.
We want interrupts (and thus preemption) to remain disabled
while doing the initial stage of testing the reason and
branching off to a debugger or kprobe, so we are still on
the original CPU which makes debugging easier in various cases.
This is how the code was intended, hence the local_irq_enable()
right in the middle of program_check_exception().
However, the assembly exception prologue for that exception was
incorrectly marked as enabling interrupts, which defeats that
(and records a redundant enable with lockdep).
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
