Commit Graph

169 Commits

Author SHA1 Message Date
a65faeaed4 kmalloc cache: embed cache pointer into kmalloc_header
Conflicts:
	kernel/mem.c
2018-06-13 00:31:34 +09:00
4dea1842e0 kmalloc cache: embed cache pointer into kmalloc_header
Conflicts:
	kernel/mem.c
2018-06-13 00:31:34 +09:00
bd170e63ba kmalloc cache refactor and pre-alloc in HFI1 open() 2018-06-13 00:31:33 +09:00
28eb649056 Generic lock-free kmalloc cache implementation
Conflicts:
	kernel/mem.c
2018-06-13 00:31:33 +09:00
aeef55d1b0 kmalloc(): try to get from remote_free list when regular is empty 2018-06-13 00:31:32 +09:00
73731d2a0d ihk_mc_map/unmap_virtual(): do proper TLB invalidation 2018-03-24 07:58:08 +09:00
06b1b4f8ab Fix deadlock on thread->times_update in getrusage()
Set thread->in_kernel properly on exiting interrupt handler when entering
it from kernel mode.

Conflicts:
	arch/x86_64/kernel/cpu.c
	kernel/mem.c
2018-03-09 17:26:31 +09:00
559fc9746c signal: check_signal must be called after check_need_resched. 2018-01-28 13:38:51 +09:00
d649d6fc2d Include mbind support (this is a rebase commit to merge into development) 2017-11-27 11:16:53 +09:00
3573b8649e Guard call to gencore and freecore
The gencore() and freecore() code in gencore.c is guarded by
POSTK_DEBUG_ARCH_DEP_18, so the call to these functions should
also be guarded, otherwise linking fails.
2017-10-18 09:20:52 +09:00
0a22320a3c Don't allocate memory for 0-page-sized requests
Previously the allocator would return all availble memory for a
request of 0 pages. This is rather counter-intuitive and left no
memory for subsequent allocations.
2017-10-18 09:20:51 +09:00
3bd0137c25 Fix some race condition on arm64
* move barrier() to architecture depended region
* add barrier() in issue_ipi, kprintf, map_virtual
* enable the workaround for cavium thunderx
2017-10-18 09:20:51 +09:00
c3dfb1663d page_fault_handler: do not try to fault addresses < 4k
There is no good reason to map these low addresses (userspace could with
mmap fixed, but that is grounds for many exploits...);

the main advantage however is if we do a null deref or close to (0->foo)
within a pagefault we will get a panic stack instead of getting a hang
because we cannot get some locks.
2017-10-13 10:02:11 +09:00
4d4279121b process/vm; replace vm_range list by a rbtree
This replaces the chained list used to keep track of all memory ranges
of a process by a standard rbtree (no need of interval tree here
because there is no overlap)

Accesses that were done directly through vm_range_list before were
replaced by lookup_process_memory_range, even full list scan (e.g.
coredump).
The full scans will thus be less efficient because calls to rb_next()
will not be inlined, but these are rarer calls that can probably afford
this compared to code simplicity.

The only reference to the actual backing structure left outside of
process.c is a call to rb_erase in xpmem_free_process_memory_range.

v2: fix lookup_process_memory_range with small start address

v3: make vm_range_insert error out properly

Panic does not lead to easy debug, all error paths
are handled to just return someting on error

v4: fix lookup_process_memory_range (again)

That optimistically going left was a more serious bug than just
last iteration, we could just pass by a match and continue down
the tree if the match was not a leaf.

v5: some users actually needed leftmost match, so restore behavior
without the breakage (hopefully)
2017-10-13 10:00:27 +09:00
daa7526127 rusage and ihklib: Fix out-of-memory reporting and cleanup
1. Fix OOM: Count memory usage only when allocation succeeded
2. Fix OOM: Make user allocation fail when memory is running out
3. Fix OOM: Move rusage_init() before numa_init()
4. Cleanup: Rename ihkconfig/ihkosctl functions
5. Cleanup: Pass event type to eventfd()
6. Cleanup: arch/.../rusage.h --> arch/.../arch_rusage.h
2017-09-20 15:11:57 +09:00
aa7cb970c4 ihk_os_getrusage(): Compile LWK-specific results in mcctrl
1. User asks mcctrl for the result via ihk_os_getrusage() with passing void *
2. mcctrl compiles the results and passes them to the user
3. User interprets it by using the type defined in the LWK-specific header
2017-09-20 15:03:45 +09:00
a05b6e1ba8 Expand dump-functions for excluding user/unused memory (This is rebase commit for merging to development) 2017-09-11 15:49:04 +09:00
9989f41fd3 add arm64 support
- add arm64 dependent codes with GICv3 and SVE support
- fix bugs based on architecture separation requests
2017-09-05 15:06:27 +09:00
4cd1c120fa profile: add PROFILE_remote_page_fault 2017-07-23 19:00:00 +09:00
bf5ac7afc8 remote_flush_tlb_array_cpumask(): bundle remote TLB invalidations 2017-07-21 15:34:48 +09:00
6714161c25 profile remote TLB invalidations 2017-07-20 22:28:25 +09:00
dd9675d65e NUMA: only print a short summary at boot time 2017-07-19 09:11:44 +09:00
51ed8dce06 numa_init(): fix rusage memory counting 2017-07-19 08:23:05 +09:00
d7b8e7f4f4 fix to count user pages
refs #864
2017-07-14 09:51:39 +09:00
0f58e9e77d NUMA: expose correct /sys/devices/system/node/nodeX/meminfo 2017-07-07 00:59:32 +09:00
da7421e8ee memdebug: more detailed error report 2017-07-08 18:36:05 +09:00
f81722c63b __mckernel_free_pages_in_allocator(): fix deallocation of invalid physical range 2017-07-08 18:35:50 +09:00
201a7e2595 Red-black tree based physical memory management 2017-07-08 18:26:51 +09:00
67843151d3 fix how to count rss and num of threads
refs #864
refs #865
2017-07-03 16:27:46 +09:00
083cf3fcc9 rusage_max_memory is set sum of all memory chanks
refs #891
2017-07-03 14:49:35 +09:00
c5079898c2 mckernel_allocate_aligned_pages_node(): support explicit NUMA node designation 2017-05-23 19:58:52 +09:00
923dc4aa11 PROFILE_mpol_alloc_missed: profile allocations that fail to satisfy user requested memory policy 2017-05-23 02:42:06 +09:00
5a4148aaaf ___kfree(): disregard NULL pointer argument 2017-05-23 02:42:06 +09:00
e2f424846c profile: rewrite syscall tracker for generic profiling code 2017-05-23 02:42:05 +09:00
d66af42f7b Revert "IKC: separate IRQ between Master-channel and Regular-channel"
This reverts commit 3c98b9410966ceebe187ebae1038317b628fbb03.
2017-05-19 10:26:30 +09:00
65dc3440cb IKC: separate IRQ between Master-channel and Regular-channel 2017-05-19 10:26:30 +09:00
3fe2257929 create rusage branch. 2017-03-15 23:22:51 +09:00
da4a5ec44b page_allocator_init(): move memory_nodes to BSS 2017-02-24 19:33:25 +09:00
d35aa9b100 page_allocator_init(): clean-up code, eliminate initial flag 2017-02-24 14:25:22 +09:00
ba8dbf1b19 Put kernel image and page table into one chunk 2017-02-24 14:21:32 +09:00
d34884f9a4 numa_init(): error handling and propagation 2017-01-08 14:15:51 +09:00
c40e7105e6 NUMA: order nodes by distance for MPOL_BIND / MPOL_PREFERRED policies as well 2017-01-03 09:02:29 +09:00
e3f0662130 allocate_aligned_pages_node(): debug msg format 2016-12-31 16:25:14 +09:00
393cec513c allocate_aligned_pages_node(): follow user policiy only for user allocations 2016-12-31 10:10:42 +09:00
40d75baca2 ihk_mc_ap_flag: rewrite flag type, intro for denoting user level allocations 2016-12-30 19:19:34 +09:00
00f3fe0840 ihk_mc_alloc_aligned_pages_node(): support for explicit indication of target NUMA node 2016-12-30 19:03:59 +09:00
df9f1f8f78 allocate_aligned_pages(): take user set NUMA policy into account 2016-12-13 17:51:39 +09:00
551999ff6b NUMA: order nodes based on distances 2016-12-13 10:46:17 +09:00
8fced29978 page_fault_handler(): improved debug msg format 2016-12-07 11:25:02 +09:00
1130cafe41 ptrace: fixed for threads. 2016-11-28 11:19:30 +09:00