Commit Graph

382 Commits

Author SHA1 Message Date
6332903f0d Revert "xpmem: Support large page attachment"
This reverts commit a8696d811d.

Conflicts:
	kernel/include/process.h
	kernel/syscall.c
	kernel/xpmem.c

Change-Id: I726e74450f6228d3fc78fc62dda15b2067732a53
2020-06-16 13:25:57 +09:00
e774e1b984 Revert "xpmem: fix mapping of attachment and segment"
This reverts commit a5fcc91656.

Change-Id: If29415369d724391b291939ecce76482138e82f5
2020-06-16 11:28:02 +09:00
a5fcc91656 xpmem: fix mapping of attachment and segment
* Mapping attached part of segment is done at attach time instead of
  make time to work with runtimes (e.g. OpenMPI) xpmem_make-ing the
  entire user-space
* Mapping attached part of segment at attach time can be turned off by
  specifying xpmem_remote_on_demand in kernel argument
* Mapping attachment chooses appropriate page-sizes, i.e., largest
  allowed by memory range and segment page boundary

Fixes: a8696d8 "xpmem: Support large page attachment"
Change-Id: I44663865204036520e5f62fe22b9134ee4629f9b
2020-06-15 10:11:29 +09:00
d370e9241f Toggle preemption while faulting pages
Change-Id: I74201061bb3e7c7c4032e3884658ace87cb85948
2020-06-15 10:11:29 +09:00
f64731ab34 do_migrate: kick scheduler on target CPU
Change-Id: Ib5875ecf0c6a3118d32973329a6f1595a910562f
2020-06-15 09:58:55 +09:00
1afc3d9b70 Keep track of number of context switches per CPU
Change-Id: I7a2194c8777a7efcd34e1ed7f4734da03fb4d433
2020-06-13 17:18:10 +09:00
17a8f68d60 set_timer(): treat spin wait as PS_RUNNING
Change-Id: Iea1ad5b0a49a12d5e1aef38ad68fccb8d789af5e
2020-06-13 17:18:10 +09:00
6441aa1abb __sched_wakeup_thread(): check if timesharing needs to be enabled
Change-Id: I081d700f345abbbdb14dcac3b6246b79475d059b
2020-06-13 17:18:09 +09:00
9b55b68934 Allow other threads to run while waiting for I/O in page faults
Change-Id: I51e847a02a698b0ecf1e356d51599aa1c9400b15
2020-06-13 17:18:09 +09:00
d5aafca1ae VM: use RW spinlock for vm_range_lock
Change-Id: Id4654084207d55bf77cc9f8b42795e0f9873cfa0
2020-06-12 03:07:33 +00:00
3300e65efc madvise: Support MADV_WIPEONFORK, MADV_KEEPONFORK and MADV_NORMAL
Change-Id: I1d4cf5affa580d7304dfdc34fa4f1707c0df617c
refs: #1374
2020-03-23 09:13:01 +09:00
26bebb2749 sched_request_migrate(): fix race condition between migration req and IRQs
make sure the caller thread holds migration queue lock with IRQs disabled
until it notifies the target CPU so that an interrupt can not deschedule
it in the middle of the request.

Change-Id: I85995018ca1e8478ccc9723985b6e8efc9c3acfb
2020-03-09 07:05:15 +00:00
9e2196c9ce fix: memory leak due to forced termination during startup
Change-Id: Ide519f01702bfd17ae4576e04806b6d155ae846a
refs: #1397
2020-03-09 01:10:38 +00:00
67f5a1d4e0 migrate-cpu: Prevent migration target from calling schedule() twice
Symptom:
A thread could call schedule() twice.

Cause:
 (1) The migrator raises rescheduling flag
 (2) The thread calls check_need_resched() for other
     reason than the migrate IPI, e.g, response to system call
     offload. And it finds that the flag is set and it's trying to
     call schedule().
 (3) The thread is interrupted by the migrate IPI and it finds that
     the flag is set and calls schedule() in the interrupt context.
 (4) The thread resumes the execution and call schedule()

Solution:
 (1) Reset the rescheduling flag when checking it and it's set
 (2) Set it again if it's decided not to call schedule()

Change-Id: I5376662d0b02ca4ebb29b42732e347f3b82d766d
Refs: #1400
2020-03-05 15:51:28 +09:00
edf7b36669 runq_lock: Fix deadlock due to cpu migration.
Symptom and analysis:
runq_lock of the migration source is acquired on
the migration destination CPU.

This happens in the following steps:
 (1) The thread stores value of cpu_local_var(runq_lock)
     to its register when trying to perform
     ihk_mc_spinlock_lock() on the lock variable.
 (2) The thread takes IPI and migrates to another CPU.
 (3) The thread resumes execution and acquires the wrong lock.

Solution:
* Disable interrupts before getting the value of
  cpu_local_var(runq_lock)

Change-Id: Ia0ea450b97f872dd6116252537e4a79f85adfc88
Refs: #1400
2020-03-05 01:51:40 +00:00
41ea9d16c4 mremap: Fix to work correctly when old_page is large_page
Change-Id: I5a589383644a8098d910e49cd7ade6df325e0366
Refs: #1383
2020-02-13 06:15:25 +00:00
1db00ebc04 release_process_vm: free vm_range_numa_policy
Change-Id: I8084cd60a12b557b635b8e350f70d4e4f95d4c52
Refs: #1101
2019-12-20 07:12:16 +00:00
a8696d811d xpmem: Support large page attachment
Change-Id: I4d672eee1c905160ece204d278f0afd9b6d7dc01
Refs: #1259
2019-12-06 09:30:51 +09:00
37605740a4 support for backlog
Change-Id: Id8f503234e7afaa284e6b97dc264eb3a2af145c7
2019-12-05 07:08:13 +00:00
a563d780c1 munmap: fix deadlock with remote pagefault on vm range lock
Add similar protection to clear_host_pte than to set_host_vma (see #986)

Also make the page fault handler only skip taking lock if the munmap
happened on the same cpu id

Change-Id: I6d9e68e8f8905b20bb2ccfa72848e04fe6404ab6
2019-11-28 02:07:45 +00:00
edd3ea0103 Revert "memory_range_lock: Enable interrupt when trylock fails"
This reverts commit 0d3ef65092.

Reason for revert: This fix causes circular dependency with memory_range manipulation and TLB flush. See #1394.

Change-Id: I4774e81ff300c199629e283e538c0a30ad0eeaae
2019-11-11 15:28:08 +09:00
e932f2e70c fork: release_thread: fix release of cloned thread
Change-Id: I390093bdb47a348cfec287cceaff22712df36bd9
2019-09-26 01:56:16 +00:00
bb08742467 fork: clone_thread: free resources when an error is detected
Change-Id: I922f3fddc35942ef2c67db6673980770731dced9
2019-09-26 01:56:16 +00:00
3e9fdfc0f1 fork: copy_user_ranges: rollback on error
Change-Id: Icdb8399cbce31835abcaeb783dde3ff14d30af6a
2019-09-26 01:56:16 +00:00
de0e07f29e schedule: Skip save_fp_regs when the process ends
Change-Id: I32ff71a0dfcd7196d2c9e6cc1d68210933470bbb
Fujitsu: POSTK_DEBUG_ARCH_DEP_106
Refs: #1354
2019-09-25 06:43:08 +00:00
11ef2f8092 coredump: Support threads
Change-Id: Id75ade6c87b15abcff5d772d90f77950376a32c1
Refs: #1219
2019-08-09 04:00:15 +00:00
0d3ef65092 memory_range_lock: Enable interrupt when trylock fails
Also use read-write-lock

Change-Id: I03150b7208325ec1fe422dcd5f931e4e41c8e40e
Refs: #452
2019-08-08 00:38:55 +00:00
2dd8687974 flush instruction cache at context switch time if necessary
Change-Id: Ic09415ea772a9de6dca43a98168a8346ca86d3e7
2019-08-08 00:29:47 +00:00
4cee9b1a27 rusage: Add comment on counting COW-source pointed-to by only fileobj
Change-Id: I082f6738dd29257c05e8a0e4b0af23dd8ffab449
2019-07-05 01:15:47 +00:00
b55e164669 page_fault_process_memory_range: Disable COW for VM region with zeroobj
This fixes ostest-mem_limits.001 which tries to anonymous-mmap 95% of
total memory. It reports a failure because:
(1) McKernel tries to allocate physically contiguous area and
    fails
(2) It turns on demand-paging
(3) It tries to obtain a page from zeroobj and fails
(4) It allocates a new page
(5) It performs COW on the page, which is unnecessary

Change-Id: Iddf0548bb9216f9bf91fb03fa21f890e599bfdad
2019-07-04 13:58:22 +09:00
aa66fe2cb1 extend_process_region: Fall back to demand paging when not contiguous enough
This fixes ostest-mem_limits.005 which tries to move brk by 95% of
total memory. It reports a failure because McKernel tries to allocate
physically contiguous area and fails.

Change-Id: I50a61cb7103fdbdbe051f0ae276a79e8e2dcdda3
2019-07-03 07:49:45 +00:00
e9955a4bba Make heap and stack private mapping
Change-Id: I4306566b3bbbe27d206c5518a2d36d117ba4ca9f
2019-06-05 15:21:20 +09:00
de042b2cb2 IPI: use logical CPU ids in ihk_mc_interrupt_cpu()
Also make remote TLB invalidation arch independent,
removes POSTK_DEBUG_ARCH_DEP_8.

Change-Id: I2b0fbcfa2bfe5da07607863e3e772d8e892e8525
2019-03-29 07:45:06 +00:00
08f817a654 page fault: clear writable bit for non-dirtying access to shared ranges
Change-Id: I3f3212b2aac79587f04450dfbdee9cb8a56bee04
Fujitsu: POSTK_DEBUG_ARCH_DEP_21
2019-03-22 05:03:03 +00:00
21c9e57646 page fault: use cow for private device mappings
Private device mappings still need copy-on-write to work, even if
there is no page.

Change-Id: I96e3e1eea81104f6b09bb7fda1105d9eeb489155
Refs: #1254
2019-03-22 04:30:55 +00:00
03a85825ed copy_user_pte: base memobj copy on range & VR_PRIVATE
Some memobjs (e.g. devobj) will not be considered 'in memobj' by
page_is_in_memobj.
Instead of trying to play whack-a-mole with the non-fileobj memobjs,
base the copy check on range's memobj and VR_PRIVATE (do not copy
MAP_SHARED mappings, so the fault handler will do the right thing™
when required)

Change-Id: Ic32cdc7766754f6559753b34845eb8c5cff6ed13
Refs: #1255
2019-03-06 17:44:11 +09:00
3fbad79afb warnings: init pte in process.c functions
pte_make_fileoff() on arm does not always init the pte, so just
init it to PTE_NULL firsthand

Change-Id: If195c1aef5b1344f13f6c0c76bb431a5fa339265
2019-03-06 06:30:30 +00:00
9ec0aeeab5 debug.h: merge both instances into ihk/debug.h
We do not need two debug.h files.

Take Fujitsu's STATIC_ASSERT over BUILD_BUG_ON because it is more used

Change-Id: If04c17fbb7406ab15fe86267fed8d6da460cec62
Fujitsu: POSTK_DEBUG_ARCH_DEP_9
2019-03-01 05:10:35 +00:00
4606714c07 process stack: use PAGE_SIZE in aux vector
Don't ask me why this shares POSTK_DEBUG_ARCH_DEP_50 with the ksym lookups...

Change-Id: Ic3db2cd77ca88be361cefec85d8ed9deb21ffcd8
Fujitsu: POSTK_DEBUG_ARCH_DEP_50
2019-03-01 05:08:16 +00:00
81e665cb48 init_process: add missing initializations to proc struct
Change-Id: I4ea386ba3a8745202745bd8e35cab00c38262f65
Fujitsu: POSTK_DEBUG_ARCH_DEP_63
2019-03-01 04:39:59 +00:00
f07e20a381 copy_user_pte: vmap area not owned by McKernel
Refs: #1166
Fujitsu: POSTK_DEBUG_TEMP_FIX_14
Change-Id: Iae0f1145d58ec2c14cecc14409b08a1db3b067b7
2019-02-28 07:50:16 +00:00
f3c9fbf4ea rusage: Don't count PF_PATCH change
Fujitsu: POSTK_DEBUG_TEMP_FIX_86

Change-Id: Ia23f2d95c67062be3390acafad3e87f087466cdc
2019-02-18 14:50:56 +09:00
bdf5175d4c invalidate_one_page: Support shmobj and contiguous PTE
Change-Id: I15b74ee4afd8e2dc52c933925aae4a1e0d8bcc72
2019-02-14 16:26:18 +09:00
641d9f1b39 clear_range_l1, clear_range_middle: Fix handling contiguous PTE
Change-Id: I2609c94d7f9342fe25aa9a5cfc208375274d46fa
2019-02-14 16:26:14 +09:00
9cfc373538 Refactor "do write back only MAP_SHARED pages"
* free_process_memory_range() always passes memobj to
  ihk_mc_pt_free_range()
* clear_range_*() don't flush page in fileobj with MF_PRIVATE flag

Fujitsu: POSTK_DEBUG_TEMP_FIX_87
Change-Id: I8d46d029b3fc51ca6f0e59d748a2fe93e324a374
2019-02-14 16:25:58 +09:00
60dcd0e798 move rusage into kernel ELF image (avoid dynamic alloc before NUMA init)
Change-Id: I7fe86244c8707694b379e567b31de65ee2c56887
2019-02-07 10:43:47 +09:00
be708674d3 Reject "do_migrate: Send IPI"
Change-Id: If77a51c9bc6a3caef502dd35a276b0dba22b4d24
Fujitsu: POSTK_TEMP_FIX_57
2019-02-06 04:11:16 +00:00
6d38c34993 Merge branch 'postk_topic-contiguous_pte' into development
* Merge cd7ab307fae9bc8aa49d23b32becf37368a1603e
* Merge commit is changed to one commit for gerrit

Change-Id: I75f0f4cf6b8b3286284638ac2c7816c5257551e4
2019-02-01 15:15:12 +09:00
d4d78e9c61 Following arm64-support to development branch
This includes the following fixes:
* fix build of arch/arm64/kernel/vdso

Change-Id: I73b05034d29f7f8731ac17f9736edbba4fb2c639
2019-02-01 15:14:45 +09:00
f299fff266 stack: add hwcap auxval
Fix the AUXV_LEN to account for hwcap and remove the ifdefs

Change-Id: I303fc2c5fa4c8cea7ec9823f8580b8a66de2f58f
Fujitsu: POSTK_DEBUG_ARCH_DEP_65
2019-02-01 13:17:58 +09:00