Commit Graph

2931 Commits

Author SHA1 Message Date
Hugh Dickins
4c21e2f244 [PATCH] mm: split page table lock
Christoph Lameter demonstrated very poor scalability on the SGI 512-way, with
a many-threaded application which concurrently initializes different parts of
a large anonymous area.

This patch corrects that, by using a separate spinlock per page table page, to
guard the page table entries in that page, instead of using the mm's single
page_table_lock.  (But even then, page_table_lock is still used to guard page
table allocation, and anon_vma allocation.)

In this implementation, the spinlock is tucked inside the struct page of the
page table page: with a BUILD_BUG_ON in case it overflows - which it would in
the case of 32-bit PA-RISC with spinlock debugging enabled.

Splitting the lock is not quite for free: another cacheline access.  Ideally,
I suppose we would use split ptlock only for multi-threaded processes on
multi-cpu machines; but deciding that dynamically would have its own costs.
So for now enable it by config, at some number of cpus - since the Kconfig
language doesn't support inequalities, let preprocessor compare that with
NR_CPUS.  But I don't think it's worth being user-configurable: for good
testing of both split and unsplit configs, split now at 4 cpus, and perhaps
change that to 8 later.

There is a benefit even for singly threaded processes: kswapd can be attacking
one part of the mm while another part is busy faulting.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:42 -07:00
Hugh Dickins
92dc6fcc84 [PATCH] mm: parisc pte atomicity
There's a worrying function translation_exists in parisc cacheflush.h,
unaffected by split ptlock since flush_dcache_page is using it on some other
mm, without any relevant lock.  Oh well, make it a slightly more robust by
factoring the pfn check within it.  And it looked liable to confuse a
camouflaged swap or file entry with a good pte: fix that too.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:42 -07:00
Hugh Dickins
deceb6cd17 [PATCH] mm: follow_page with inner ptlock
Final step in pushing down common core's page_table_lock.  follow_page no
longer wants caller to hold page_table_lock, uses pte_offset_map_lock itself;
and so no page_table_lock is taken in get_user_pages itself.

But get_user_pages (and get_futex_key) do then need follow_page to pin the
page for them: take Daniel's suggestion of bitflags to follow_page.

Need one for WRITE, another for TOUCH (it was the accessed flag before:
vanished along with check_user_page_readable, but surely get_numa_maps is
wrong to mark every page it finds as accessed), another for GET.

And another, ANON to dispose of untouched_anonymous_page: it seems silly for
that to descend a second time, let follow_page observe if there was no page
table and return ZERO_PAGE if so.  Fix minor bug in that: check VM_LOCKED -
make_pages_present ought to make readonly anonymous present.

Give get_numa_maps a cond_resched while we're there.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:41 -07:00
Hugh Dickins
c34d1b4d16 [PATCH] mm: kill check_user_page_readable
check_user_page_readable is a problematic variant of follow_page.  It's used
only by oprofile's i386 and arm backtrace code, at interrupt time, to
establish whether a userspace stackframe is currently readable.

This is problematic, because we want to push the page_table_lock down inside
follow_page, and later split it; whereas oprofile is doing a spin_trylock on
it (in the i386 case, forgotten in the arm case), and needs that to pin
perhaps two pages spanned by the stackframe (which might be covered by
different locks when we split).

I think oprofile is going about this in the wrong way: it doesn't need to know
the area is readable (neither i386 nor arm uses read protection of user
pages), it doesn't need to pin the memory, it should simply
__copy_from_user_inatomic, and see if that succeeds or not.  Sorry, but I've
not got around to devising the sparse __user annotations for this.

Then we can eliminate check_user_page_readable, and return to a single
follow_page without the __follow_page variants.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:41 -07:00
Hugh Dickins
c0718806cf [PATCH] mm: rmap with inner ptlock
rmap's page_check_address descend without page_table_lock.  First just
pte_offset_map in case there's no pte present worth locking for, then take
page_table_lock for the full check, and pass ptl back to caller in the same
style as pte_offset_map_lock.  __xip_unmap, page_referenced_one and
try_to_unmap_one use pte_unmap_unlock.  try_to_unmap_cluster also.

page_check_address reformatted to avoid progressive indentation.  No use is
made of its one error code, return NULL when it fails.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:41 -07:00
Hugh Dickins
508034a32b [PATCH] mm: unmap_vmas with inner ptlock
Remove the page_table_lock from around the calls to unmap_vmas, and replace
the pte_offset_map in zap_pte_range by pte_offset_map_lock: all callers are
now safe to descend without page_table_lock.

Don't attempt fancy locking for hugepages, just take page_table_lock in
unmap_hugepage_range.  Which makes zap_hugepage_range, and the hugetlb test in
zap_page_range, redundant: unmap_vmas calls unmap_hugepage_range anyway.  Nor
does unmap_vmas have much use for its mm arg now.

The tlb_start_vma and tlb_end_vma in unmap_page_range are now called without
page_table_lock: if they're implemented at all, they typically come down to
flush_cache_range (usually done outside page_table_lock) and flush_tlb_range
(which we already audited for the mprotect case).

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:41 -07:00
Hugh Dickins
663b97f7ef [PATCH] mm: flush_tlb_range outside ptlock
There was one small but very significant change in the previous patch:
mprotect's flush_tlb_range fell outside the page_table_lock: as it is in 2.4,
but that doesn't prove it safe in 2.6.

On some architectures flush_tlb_range comes to the same as flush_tlb_mm, which
has always been called from outside page_table_lock in dup_mmap, and is so
proved safe.  Others required a deeper audit: I could find no reliance on
page_table_lock in any; but in ia64 and parisc found some code which looks a
bit as if it might want preemption disabled.  That won't do any actual harm,
so pending a decision from the maintainers, disable preemption there.

Remove comments on page_table_lock from flush_tlb_mm, flush_tlb_range and
flush_tlb_page entries in cachetlb.txt: they were rather misleading (what
generic code does is different from what usually happens), the rules are now
changing, and it's not yet clear where we'll end up (will the generic
tlb_flush_mmu happen always under lock?  never under lock?  or sometimes under
and sometimes not?).

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:40 -07:00
Hugh Dickins
705e87c0c3 [PATCH] mm: pte_offset_map_lock loops
Convert those common loops using page_table_lock on the outside and
pte_offset_map within to use just pte_offset_map_lock within instead.

These all hold mmap_sem (some exclusively, some not), so at no level can a
page table be whipped away from beneath them.  But whereas pte_alloc loops
tested with the "atomic" pmd_present, these loops are testing with pmd_none,
which on i386 PAE tests both lower and upper halves.

That's now unsafe, so add a cast into pmd_none to test only the vital lower
half: we lose a little sensitivity to a corrupt middle directory, but not
enough to worry about.  It appears that i386 and UML were the only
architectures vulnerable in this way, and pgd and pud no problem.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:40 -07:00
Hugh Dickins
c74df32c72 [PATCH] mm: ptd_alloc take ptlock
Second step in pushing down the page_table_lock.  Remove the temporary
bridging hack from __pud_alloc, __pmd_alloc, __pte_alloc: expect callers not
to hold page_table_lock, whether it's on init_mm or a user mm; take
page_table_lock internally to check if a racing task already allocated.

Convert their callers from common code.  But avoid coming back to change them
again later: instead of moving the spin_lock(&mm->page_table_lock) down,
switch over to new macros pte_alloc_map_lock and pte_unmap_unlock, which
encapsulate the mapping+locking and unlocking+unmapping together, and in the
end may use alternatives to the mm page_table_lock itself.

These callers all hold mmap_sem (some exclusively, some not), so at no level
can a page table be whipped away from beneath them; and pte_alloc uses the
"atomic" pmd_present to test whether it needs to allocate.  It appears that on
all arches we can safely descend without page_table_lock.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:40 -07:00
Hugh Dickins
1bb3630e89 [PATCH] mm: ptd_alloc inline and out
It seems odd to me that, whereas pud_alloc and pmd_alloc test inline, only
calling out-of-line __pud_alloc __pmd_alloc if allocation needed,
pte_alloc_map and pte_alloc_kernel are entirely out-of-line.  Though it does
add a little to kernel size, change them to macros testing inline, calling
__pte_alloc or __pte_alloc_kernel to allocate out-of-line.  Mark none of them
as fastcalls, leave that to CONFIG_REGPARM or not.

It also seems more natural for the out-of-line functions to leave the offset
calculation and map to the inline, which has to do it anyway for the common
case.  At least mremap move wants __pte_alloc without _map.

Macros rather than inline functions, certainly to avoid the header file issues
which arise from CONFIG_HIGHPTE needing kmap_types.h, but also in case any
architectures I haven't built would have other such problems.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:40 -07:00
Hugh Dickins
872fec16d9 [PATCH] mm: init_mm without ptlock
First step in pushing down the page_table_lock.  init_mm.page_table_lock has
been used throughout the architectures (usually for ioremap): not to serialize
kernel address space allocation (that's usually vmlist_lock), but because
pud_alloc,pmd_alloc,pte_alloc_kernel expect caller holds it.

Reverse that: don't lock or unlock init_mm.page_table_lock in any of the
architectures; instead rely on pud_alloc,pmd_alloc,pte_alloc_kernel to take
and drop it when allocating a new one, to check lest a racing task already
did.  Similarly no page_table_lock in vmalloc's map_vm_area.

Some temporary ugliness in __pud_alloc and __pmd_alloc: since they also handle
user mms, which are converted only by a later patch, for now they have to lock
differently according to whether or not it's init_mm.

If sources get muddled, there's a danger that an arch source taking
init_mm.page_table_lock will be mixed with common source also taking it (or
neither take it).  So break the rules and make another change, which should
break the build for such a mismatch: remove the redundant mm arg from
pte_alloc_kernel (ppc64 scrapped its distinct ioremap_mm in 2.6.13).

Exceptions: arm26 used pte_alloc_kernel on user mm, now pte_alloc_map; ia64
used pte_alloc_map on init_mm, now pte_alloc_kernel; parisc had bad args to
pmd_alloc and pte_alloc_kernel in unused USE_HPPA_IOREMAP code; ppc64
map_io_page forgot to unlock on failure; ppc mmu_mapin_ram and ppc64 im_free
took page_table_lock for no good reason.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:40 -07:00
Hugh Dickins
46dea3d092 [PATCH] mm: ia64 use expand_upwards
ia64 has expand_backing_store function for growing its Register Backing Store
vma upwards.  But more complete code for this purpose is found in the
CONFIG_STACK_GROWSUP part of mm/mmap.c.  Uglify its #ifdefs further to provide
expand_upwards for ia64 as well as expand_stack for parisc.

The Register Backing Store vma should be marked VM_ACCOUNT.  Implement the
intention of growing it only a page at a time, instead of passing an address
outside of the vma to handle_mm_fault, with unknown consequences.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:39 -07:00
Hugh Dickins
f449952bc8 [PATCH] mm: mm_struct hiwaters moved
Slight and timid rearrangement of mm_struct: hiwater_rss and hiwater_vm were
tacked on the end, but it seems better to keep them near _file_rss, _anon_rss
and total_vm, in the same cacheline on those arches verified.

There are likely to be more profitable rearrangements, but less obvious (is it
good or bad that saved_auxv[AT_VECTOR_SIZE] isolates cpu_vm_mask and context
from many others?), needing serious instrumentation.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:39 -07:00
Hugh Dickins
365e9c87a9 [PATCH] mm: update_hiwaters just in time
update_mem_hiwater has attracted various criticisms, in particular from those
concerned with mm scalability.  Originally it was called whenever rss or
total_vm got raised.  Then many of those callsites were replaced by a timer
tick call from account_system_time.  Now Frank van Maarseveen reports that to
be found inadequate.  How about this?  Works for Frank.

Replace update_mem_hiwater, a poor combination of two unrelated ops, by macros
update_hiwater_rss and update_hiwater_vm.  Don't attempt to keep
mm->hiwater_rss up to date at timer tick, nor every time we raise rss (usually
by 1): those are hot paths.  Do the opposite, update only when about to lower
rss (usually by many), or just before final accounting in do_exit.  Handle
mm->hiwater_vm in the same way, though it's much less of an issue.  Demand
that whoever collects these hiwater statistics do the work of taking the
maximum with rss or total_vm.

And there has been no collector of these hiwater statistics in the tree.  The
new convention needs an example, so match Frank's usage by adding a VmPeak
line above VmSize to /proc/<pid>/status, and also a VmHWM line above VmRSS
(High-Water-Mark or High-Water-Memory).

There was a particular anomaly during mremap move, that hiwater_vm might be
captured too high.  A fleeting such anomaly remains, but it's quickly
corrected now, whereas before it would stick.

What locking?  None: if the app is racy then these statistics will be racy,
it's not worth any overhead to make them exact.  But whenever it suits,
hiwater_vm is updated under exclusive mmap_sem, and hiwater_rss under
page_table_lock (for now) or with preemption disabled (later on): without
going to any trouble, minimize the time between reading current values and
updating, to minimize those occasions when a racing thread bumps a count up
and back down in between.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:39 -07:00
Nick Piggin
b5810039a5 [PATCH] core remove PageReserved
Remove PageReserved() calls from core code by tightening VM_RESERVED
handling in mm/ to cover PageReserved functionality.

PageReserved special casing is removed from get_page and put_page.

All setting and clearing of PageReserved is retained, and it is now flagged
in the page_alloc checks to help ensure we don't introduce any refcount
based freeing of Reserved pages.

MAP_PRIVATE, PROT_WRITE of VM_RESERVED regions is tentatively being
deprecated.  We never completely handled it correctly anyway, and is be
reintroduced in future if required (Hugh has a proof of concept).

Once PageReserved() calls are removed from kernel/power/swsusp.c, and all
arch/ and driver code, the Set and Clear calls, and the PG_reserved bit can
be trivially removed.

Last real user of PageReserved is swsusp, which uses PageReserved to
determine whether a struct page points to valid memory or not.  This still
needs to be addressed (a generic page_is_ram() should work).

A last caveat: the ZERO_PAGE is now refcounted and managed with rmap (and
thus mapcounted and count towards shared rss).  These writes to the struct
page could cause excessive cacheline bouncing on big systems.  There are a
number of ways this could be addressed if it is an issue.

Signed-off-by: Nick Piggin <npiggin@suse.de>

Refcount bug fix for filemap_xip.c

Signed-off-by: Carsten Otte <cotte@de.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:39 -07:00
Hugh Dickins
4294621f41 [PATCH] mm: rss = file_rss + anon_rss
I was lazy when we added anon_rss, and chose to change as few places as
possible.  So currently each anonymous page has to be counted twice, in rss
and in anon_rss.  Which won't be so good if those are atomic counts in some
configurations.

Change that around: keep file_rss and anon_rss separately, and add them
together (with get_mm_rss macro) when the total is needed - reading two
atomics is much cheaper than updating two atomics.  And update anon_rss
upfront, typically in memory.c, not tucked away in page_add_anon_rmap.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:38 -07:00
Hugh Dickins
fc2acab31b [PATCH] mm: tlb_finish_mmu forget rss
zap_pte_range has been counting the pages it frees in tlb->freed, then
tlb_finish_mmu has used that to update the mm's rss.  That got stranger when I
added anon_rss, yet updated it by a different route; and stranger when rss and
anon_rss became mm_counters with special access macros.  And it would no
longer be viable if we're relying on page_table_lock to stabilize the
mm_counter, but calling tlb_finish_mmu outside that lock.

Remove the mmu_gather's freed field, let tlb_finish_mmu stick to its own
business, just decrement the rss mm_counter in zap_pte_range (yes, there was
some point to batching the update, and a subsequent patch restores that).  And
forget the anal paranoia of first reading the counter to avoid going negative
- if rss does go negative, just fix that bug.

Remove the mmu_gather's flushes and avoided_flushes from arm and arm26: no use
was being made of them.  But arm26 alone was actually using the freed, in the
way some others use need_flush: give it a need_flush.  arm26 seems to prefer
spaces to tabs here: respect that.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:37 -07:00
Hugh Dickins
4d6ddfa924 [PATCH] mm: tlb_is_full_mm was obscure
tlb_is_full_mm?  What does that mean?  The TLB is full?  No, it means that the
mm's last user has gone and the whole mm is being torn down.  And it's an
inline function because sparc64 uses a different (slightly better)
"tlb_frozen" name for the flag others call "fullmm".

And now the ptep_get_and_clear_full macro used in zap_pte_range refers
directly to tlb->fullmm, which would be wrong for sparc64.  Rather than
correct that, I'd prefer to scrap tlb_is_full_mm altogether, and change
sparc64 to just use the same poor name as everyone else - is that okay?

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:37 -07:00
Hugh Dickins
15a23ffa2f [PATCH] mm: tlb_gather_mmu get_cpu_var
tlb_gather_mmu dates from before kernel preemption was allowed, and uses
smp_processor_id or __get_cpu_var to find its per-cpu mmu_gather.  That works
because it's currently only called after getting page_table_lock, which is not
dropped until after the matching tlb_finish_mmu.  But don't rely on that, it
will soon change: now disable preemption internally by proper get_cpu_var in
tlb_gather_mmu, put_cpu_var in tlb_finish_mmu.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:37 -07:00
Hugh Dickins
65500d234e [PATCH] mm: page fault handlers tidyup
Impose a little more consistency on the page fault handlers do_wp_page,
do_swap_page, do_anonymous_page, do_no_page, do_file_page: why not pass their
arguments in the same order, called the same names?

break_cow is all very well, but what it did was inlined elsewhere: easier to
compare if it's brought back into do_wp_page.

do_file_page's fallback to do_no_page dates from a time when we were testing
pte_file by using it wherever possible: currently it's peculiar to nonlinear
vmas, so just check that.  BUG_ON if not?  Better not, it's probably page
table corruption, so just show the pte: hmm, there's a pte_ERROR macro, let's
use that for do_wp_page's invalid pfn too.

Hah!  Someone in the ppc64 world noticed pte_ERROR was unused so removed it:
restored (and say "pud" not "pmd" in its pud_ERROR).

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:37 -07:00
Hugh Dickins
a8fb5618da [PATCH] mm: unlink_file_vma, remove_vma
Divide remove_vm_struct into two parts: first anon_vma_unlink plus
unlink_file_vma, to unlink the vma from the list and tree by which rmap or
vmtruncate might find it; then remove_vma to close, fput and free.

The intention here is to do the anon_vma_unlink and unlink_file_vma earlier,
in free_pgtables before freeing any page tables: so we can be sure that any
page tables traversed by rmap and vmtruncate are stable (and other, ordinary
cases are stabilized by holding mmap_sem).

This will be crucial to traversing pgd,pud,pmd without page_table_lock.  But
testing the split-out patch showed that lifting the page_table_lock is
symbiotically necessary to make this change - the lock ordering is wrong to
move those unlinks into free_pgtables while it's under ptlock.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:37 -07:00
Hugh Dickins
ab50b8ed81 [PATCH] mm: vm_stat_account unshackled
The original vm_stat_account has fallen into disuse, with only one user, and
only one user of vm_stat_unaccount.  It's easier to keep track if we convert
them all to __vm_stat_account, then free it from its __shackles.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:37 -07:00
Andi Kleen
dfcd3c0dc4 [PATCH] Convert mempolicies to nodemask_t
The NUMA policy code predated nodemask_t so it used open coded bitmaps.
Convert everything to nodemask_t.  Big patch, but shouldn't have any actual
behaviour changes (except I removed one unnecessary check against
node_online_map and one unnecessary BUG_ON)

Signed-off-by: "Andi Kleen" <ak@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:35 -07:00
Rik Van Riel
eb92f4ef32 [PATCH] add sem_is_read/write_locked()
Add sem_is_read/write_locked functions to the read/write semaphores, along the
same lines of the *_is_locked spinlock functions.  The swap token tuning patch
uses sem_is_read_locked; sem_is_write_locked is added for completeness.

Signed-off-by: Rik van Riel <riel@redhat.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:35 -07:00
Ivan Kokshaysky
63f324cf07 [PATCH] fix alpha breakage
barrier.h uses barrier() in non-SMP case.  And doesn't include compiler.h.

Cc: Al Viro <viro@ftp.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:35 -07:00
Christoph Lameter
930fc45a49 [PATCH] vmalloc_node
This patch adds

vmalloc_node(size, node)	-> Allocate necessary memory on the specified node

and

get_vm_area_node(size, flags, node)

and the other functions that it depends on.

Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-10-29 21:40:35 -07:00
Jeff Garzik
0169e284f6 [libata] remove ata_chk_err(), ->check_err() hook.
We now depend on ->tf_read() to provide us with the contents
of the Error shadow register.
2005-10-29 21:25:10 -04:00
Herbert Xu
d32311fed7 [PATCH] Introduce sg_set_buf
sg_init_one is a nice tool for the block layer.  However, users
of struct scatterlist in other subsystems don't usually need the
DMA attributes.  For them it's a waste of time and space to
initialise the whole struct scatterlist structure.

Therefore this patch adds a new function sg_set_buf to initialise
a scatterlist without zeroing the DMA attributes.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2005-10-30 11:14:39 +11:00
Linus Torvalds
be15cd72d2 Merge master.kernel.org:/home/rmk/linux-2.6-arm 2005-10-29 14:02:16 -07:00
Nicolas Pitre
37d07b72ef [ARM] 3061/1: cleanup the XIP link address mess
Patch from Nicolas Pitre

Since vmlinux.lds.S is preprocessed, we can use the defines already
present in asm/memory.h (allowed by patch #3060) for the XIP kernel link
address instead of relying on a duplicated Makefile hardcoded value, and
also get rid of its dependency on awk to handle it at the same time.

While at it let's clean XIP stuff even further and make things clearer
in head.S with a nice code reduction.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2005-10-29 21:44:56 +01:00
Nicolas Pitre
f09b997999 [ARM] 3060/1: allow constants found in asm/memory.h to be used in asm code
Patch from Nicolas Pitre

This patch allows for assorted type of cleanups by letting assembly code
use the same set of defines for constant values and avoid duplicated
definitions that might not always be in sync, or that might simply be
confusing due to the different names for the same thing.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2005-10-29 21:44:55 +01:00
Linus Torvalds
e9d52234e3 Merge branch 'upstream' of git://ftp.linux-mips.org/pub/scm/upstream-linus 2005-10-29 12:19:15 -07:00
Andrew Isaacson
8a1417de9e BCM1480 HT support
PCI support code for PLX 7250 PCI-X tunnel on BCM91480B BigSur board.
    
Signed-Off-By: Andy Isaacson <adi@broadcom.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:49 +01:00
Andrew Isaacson
9a6dcea103 Support for BigSur board.
Signed-Off-By: Andy Isaacson <adi@broadcom.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:47 +01:00
Andrew Isaacson
93ce2f524e Add support for SB1A CPU.
Signed-Off-By: Andy Isaacson <adi@broadcom.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:46 +01:00
Andrew Isaacson
4f19f99047 Sibyte header cleanup
Update sibyte headers to match Broadcom internal copies:
 - comment cleanup and updates
 - fix LittleSur part number to match the board silkscreen
    
Signed-Off-By: Andy Isaacson <adi@broadcom.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:46 +01:00
Andrew Isaacson
4cbf2beac2 BCM1480 headers
Add header files for BCM1480/1280/1455/1255 family of chips, and
update sb1250 headers which are shared by BCM1480 family.
    
Signed-Off-By: Andy Isaacson <adi@broadcom.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

diff --git a/include/asm-mips/sibyte/bcm1480_int.h b/include/asm-mips/sibyte/bcm1480_int.h
new file mode 100644
2005-10-29 19:32:45 +01:00
Ralf Baechle
485a4a928a Make UL what should be UL.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:44 +01:00
Ralf Baechle
178086c86a Don't print file name and line in die and die_if_kernel.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:42 +01:00
Ralf Baechle
8f91ed6c2f Define EOWNERDEAD and ENOTRECOVERABLE.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:41 +01:00
Ralf Baechle
beb3ca82fc More configcheck fixes.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:40 +01:00
Ralf Baechle
4ee1303a78 2.6.14-rc1 updates for MIPS compat types.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:40 +01:00
Ralf Baechle
b4f8c42307 Complete the fcntl.h cleanup.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:40 +01:00
Ralf Baechle
3cd9b6802d Cleanup Sibyte Kconfig a bit further.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:39 +01:00
Ralf Baechle
8592d4c00e Fix weirdness in <asm/bug.h>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:38 +01:00
Ralf Baechle
ec917c2c1a Fixup a few lose ends in explicit support for MIPS R1/R2.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:37 +01:00
Ralf Baechle
5090dfb5bc Provide 64-bit address space definitions for the Sibyte SB1 CPU core.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:33 +01:00
Ralf Baechle
02cf211968 Cleanup the mess in cpu_cache_init.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:32 +01:00
Ralf Baechle
f5cfa980e5 Use R4000 TLB routines for SB1 also.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:31 +01:00
Ralf Baechle
0015365cc6 Fix ARCH_KMALLOC_MINALIGN values on MIPS
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:30 +01:00
Ralf Baechle
c78cbf49c4 Support for MIPSsim, the cycle accurate MIPS simulator.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:30 +01:00
Daniel Jacobowitz
ea3d710fe5 Revise MIPS 64-bit ptrace interface
Change the N32 debugging ABI to something more sane, and add support
for o32 and n32 debuggers to trace n64 programs.
    
Signed-off-by: Daniel Jacobowitz <dan@codesourcery.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:29 +01:00
Ralf Baechle
9d58f302ca Glue again after removal of BUILD_BUG().
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:28 +01:00
Ralf Baechle
80b47346b0 SMP on Malta needs to define ARCH_HAS_IRQ_PER_CPU since 2.6.14-rc1.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

diff --git a/include/asm-mips/mach-mips/irq.h b/include/asm-mips/mach-mips/irq.h
new file mode 100644
2005-10-29 19:32:27 +01:00
Ralf Baechle
bab056aafe Add SOCK_DCCP definition for MIPS also.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:26 +01:00
Ralf Baechle
e97288386a The type of sum in csum_tcpudp_nofold is "unsigned int", so when we assign
to it in an asm() block, and we're running on a system with 64-bit
registers, it is vitally important that we sign extend it correctly before
returning to C.  Otherwise the stray high bits will be preserved into
csum_fold, and on the SB-1 processor, 32-bit arithmetic on a non
sign-extended register will yield surprising results.
    
This caused incorrect checksums in some UDP packets for NFS root.  The
problem was mild when using a 10.0.1.x IP address, but severe when
using 192.168.1.x.
    
Signed-off-by: Daniel Jacobowitz <dan@codesourcery.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:25 +01:00
Pete Popov
2cce826322 Kernel gpio/2 routines that will be used by some drivers.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:25 +01:00
Pete Popov
d6460827af Updated pcmcia driver with pb1200 and db1200 support.
Updated db1200_defconfig so pcmcia is enabled by default.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:24 +01:00
Ralf Baechle
870d3d98eb Reorder & reformat a bit.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:23 +01:00
Ralf Baechle
61ed7f08b6 The values for SO_SNDBUFFORCE / SO_RCVBUFFORCE were already taken ...
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:22 +01:00
Ralf Baechle
ebfaebae36 Futexes for MIPS, for the time being only the R10000_LLSC_WAR version.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:21 +01:00
Ralf Baechle
9dbdfce85c Define pcibus_to_node() for IP27.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:21 +01:00
Pete Popov
26a940e217 Cleaned up AMD Au1200 IDE driver:
- converted to platform bus
- removed pci dependencies
- removed virt_to_phys/phys_to_virt calls
    
System now can root off of a disk.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

diff --git a/Documentation/mips/AU1xxx_IDE.README b/Documentation/mips/AU1xxx_IDE.README
new file mode 100644
2005-10-29 19:32:20 +01:00
Ralf Baechle
4f94afa258 Delete the SABLE_RTL case.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:19 +01:00
Ralf Baechle
097975fc66 Provide MODULE_ARCH_VERMAGIC for MIPS.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:18 +01:00
Maciej W. Rozycki
98e316d4b1 Move MIPS Technologies processor IDs to where they belong.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:16 +01:00
Ralf Baechle
5bcb9a58e6 Move genrtc.c's functions into <asm/rtc.h>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:15 +01:00
Thiemo Seufer
65dd7026a9 Define some more common ip22 CPU features.
Signed-off-by: Thiemo Seufer <ths@networkno.de>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:14 +01:00
Thiemo Seufer
aaa49075c6 Typo fix.
Signed-off-by: Thiemo Seufer <ths@networkno.de>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:14 +01:00
Thiemo Seufer
2fe25f67a5 More .set push/pop encapsulation, more eyefriendly code formatting.
Signed-off-by: Thiemo Seufer <ths@networkno.de>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:14 +01:00
Thiemo Seufer
f8670e66dc Fix MAP_BASE for 64bit ip22.
Signed-off-by: Thiemo Seufer <ths@networkno.de>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:13 +01:00
Ralf Baechle
f99d3023f3 Sprinkle a few more .set mipsX over xchg to make sure we dont' end up with
64-bit instructions on 32-bit processors, they tend to be unhappy about
that kind of food ;-)

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:11 +01:00
Ralf Baechle
fabffc13ed Remove workaround for binutils 2.15 assembler bug; this version is not
suitable to reliably build kernels anymore anyway and 2.16 has this
fixed.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:11 +01:00
Ralf Baechle
27c7c1657d Drop might_sleep() calls from get_user() & co. This should fix the issue
in http://www.linux-mips.org/cgi-bin/mesg.cgi?a=linux-mips&i=200508171321.20094.Joshua.Wise%40sicortex.com and it's the right thing to do anyway because
it was inflating those functions way too much.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:10 +01:00
Ralf Baechle
340ee4b98c Virtual SMP support for the 34K.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:10 +01:00
Ralf Baechle
533330bf7f On CONFIG_64BIT_PHYS_ADDR, pfn always fits in 'unsigned long', but
pfn<<PAGE_SHIFT sometimes extends beyond.  The pte is big enough to hold
'long long', but the shift in pfn_pte() needs to do its calculation with
enough bits to hold the result.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:09 +01:00
Ralf Baechle
0952e2905c Fix parenthesis in macros.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:08 +01:00
Ralf Baechle
8b200ce4a6 Define cpu_icache_snoops_remote_store. This is slight abuse of something
which originally was meant for SMP cache managment but it can be argued
to apply on the 34K as well.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:07 +01:00
Ralf Baechle
3fd5646cac Add missing space.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:05 +01:00
Ralf Baechle
479a0e3e02 Support for CoreFPGA-3.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:32:02 +01:00
Ralf Baechle
23fbee9dd5 Support for Toshiba's RBHMA4500 eval board for the TX4938.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:57 +01:00
Pete Popov
bdf21b18b4 Philips PNX8550 support: MIPS32-like core with 2 Trimedias on it.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:54 +01:00
Ralf Baechle
e01402b115 More AP / SP bits for the 34K, the Malta bits and things. Still wants
a little polishing.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:53 +01:00
Ralf Baechle
86071b637d Cleanups.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:53 +01:00
Ralf Baechle
7e35952baa Move Origin crapola into a machine-specific header file.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:52 +01:00
Ralf Baechle
8f40611d2b Detect the MIPS R2 vectored interrupt, external interrupt controller
options and the precense of the MT ASE.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:51 +01:00
Ralf Baechle
699dbc90e8 Macros to access the register of processors using the new MIPS
Multithreading ASE, also know as MT ASE.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

diff --git a/include/asm-mips/mipsmtregs.h b/include/asm-mips/mipsmtregs.h
new file mode 100644
2005-10-29 19:31:51 +01:00
Pete Popov
f10fae0240 Fix the fixup_bigphys_addr compile problem.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:50 +01:00
Pete Popov
10f6567e63 Removed __ilog2 since it's no longer needed and conflicts with the
generic one.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:49 +01:00
Ralf Baechle
7a0fc58cd9 A few more macros to access MIPS R2 architecture registers.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:49 +01:00
Ralf Baechle
97fb5de194 Add EF_MIPS_ARCH_32R2 and EF_MIPS_ARCH_64R2 for tagging of R2 binaries.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:47 +01:00
Ralf Baechle
f039b5d366 Add a few more SHN_MIPS_* symbols from glibc.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:47 +01:00
Ralf Baechle
7db36c858c Add inotify syscalls for MIPS.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:47 +01:00
Ralf Baechle
cc61c1fede MIPS R2 instruction hazard handling.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:46 +01:00
Ralf Baechle
bbc7f22f6d Detect the 34K.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:45 +01:00
Ralf Baechle
ff88f8a3d2 Use ei / di MIPS32 R2 instructions if available.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:44 +01:00
Ralf Baechle
6590326505 Use clz / dclz on MIPS32 / MIPS64 processors.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:43 +01:00
Ralf Baechle
60080265a1 Define kmap_atomic_pfn() for MIPS.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:42 +01:00
Ralf Baechle
c0ec406c80 Fix endianess bugs.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2005-10-29 19:31:41 +01:00
Thiemo Seufer
04988d6fda Protect noat assembly with .set push/pop and make it somewhat readable.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: Thiemo Seufer <ths@networkno.de>
2005-10-29 19:31:40 +01:00