Commit Graph

53 Commits

Author SHA1 Message Date
9c7d0cfaec getrusage: Fix memory_stat_mapped_file when SIGBUS occurs in file map
Change-Id: Ia4686f32a3c888d5c886ab6cc6c2b510885447f5
Refs: #1422
2021-03-01 05:55:37 +00:00
b0bd1feefb remap_file_pages: check file mapping
Change-Id: Ibf145a20181938a9825214253337a423fcd53064
Refs: #1521
2021-02-19 02:23:39 +00:00
83ef96a739 fileobj: disable IRQs while holding page hash locks, schedule() in I/O loop
Change-Id: Iaf72d55980f1a5df6c93c4a57fa57b0ae5b1d229
2020-06-13 17:18:09 +09:00
c0c80b71ca mmap and fileobj: handle MF_ZEROFILL properly
Change-Id: I6ee52b4cab212b1973339bc8d49065c1ec9263b0
2020-06-13 17:18:09 +09:00
c3c57940ba Memory ordering and usage of ASM cmpxchg() instead of compiler atomic intrinsics
Change-Id: I4dadebc32721744dad982f3fc5b3eea7ab7ca745
2020-06-13 17:18:08 +09:00
9ec0aeeab5 debug.h: merge both instances into ihk/debug.h
We do not need two debug.h files.

Take Fujitsu's STATIC_ASSERT over BUILD_BUG_ON because it is more used

Change-Id: If04c17fbb7406ab15fe86267fed8d6da460cec62
Fujitsu: POSTK_DEBUG_ARCH_DEP_9
2019-03-01 05:10:35 +00:00
c1270cdf6d fileobj, shmobj: free pages in object destructor (as opposed to page_unmap())
Change-Id: I3ea50fc13ae5c090ba32aad4461f9741a4c35665
2019-02-14 16:26:00 +09:00
9cfc373538 Refactor "do write back only MAP_SHARED pages"
* free_process_memory_range() always passes memobj to
  ihk_mc_pt_free_range()
* clear_range_*() don't flush page in fileobj with MF_PRIVATE flag

Fujitsu: POSTK_DEBUG_TEMP_FIX_87
Change-Id: I8d46d029b3fc51ca6f0e59d748a2fe93e324a374
2019-02-14 16:25:58 +09:00
39f9d7fdff Handle hugetlbfs file mapping
Hugetlbfs file mappings are handled differently than regular files:
 - pager_req_create will tell us the file is in a hugetlbfs
 - allocate memory upfront, we need to fail if not enough memory
 - the memory needs to be given again if another process maps the same
   file

This implementation still has some hacks, in particular, the memory
needs to be freed when all mappings are done and the file has been
deleted/closed by all processes.
We cannot know when the file is closed/unlinked easily, so clean up
memory when all processes have exited.

To test, install libhugetlbfs and link a program with the additional
LDFLAGS += -B /usr/share/libhugetlbfs -Wl,--hugetlbfs-align

Then run with HUGETLB_ELFMAP=RW set, you can check this works with
HUGETLB_DEBUG=1 HUGETLB_VERBOSE=2

Change-Id: I327920ff06efd82e91b319b27319f41912169af1
2018-10-11 08:54:13 +00:00
c25fb2aa39 memobj: transform memobj lock to refcounting
We had a deadlock between:
 - free_process_memory_range (take lock) -> ihk_mc_pt_free_range ->
... -> remote_flush_tlb_array_cpumask -> "/* Wait for all cores */"
and
 - obj_list_lookup() under fileobj_list_lock that disabled irqs
and thus never ack'd the remote flush

The rework is quite big but removes the need for the big lock,
although devobj and shmobj needed a new smaller lock to be
introduced - the new locks are used much more locally and
should not cause problems.

On the bright side, refcounting being moved to memobj level means
we could remove refcounting implemented separately in all object
types and simplifies code a bit.

Change-Id: I6bc8438a98b1d8edddc91c4ac33c11b88e097ebb
2018-09-12 18:03:25 +09:00
94d093f058 fileobj_create: Suppress message on getting -ESRCH
-ESRCH from mcctrl doesn't mean an error but the file is not a regular
file and mcctrl wants McKernel to treat it as a device file.

Change-Id: Ie121f0e6a8b1f0a29c2f2cf193a51f4f52337809
2018-08-23 04:01:20 +00:00
3185334c1c debug messages: implement dynamic debug
Heavily inspired off linux kernel's dynamic debug:
 * add a /sys/kernel/debug/dynamic_debug/control file
 (accessible from linux side in /sys/class/mcos/mcos0/sys/kernel/debug/dynamic_debug/control)
 * read from file to list debug statements (currently limited to 4k in size)
 * write to file with '[file foo ][func bar ][line [x][-[y]]] [+-]p' to change values

Side effects:
 * reindented all linker scripts, there is a new __verbose section
 * added string function strpbrk

Change-Id: I36d7707274dcc3ecaf200075a31a2f0f76021059
2018-07-26 14:16:31 +09:00
df2c993721 fileobj_create: only allocate new object if one wasn't found
Change-Id: I5e12439333bf0c9cc7dad6e3cf410bfee616f77e
2018-07-26 04:41:03 +00:00
9d587dcbe8 fileobj_release: do not notify linux of surplus refs
Surplus refs on the linux side will not change anything, so spare
ourselves a message.
The final message will free all refs at once when the object is
destroyed.

Change-Id: Ie086b9dda663729962037c67e8233370509234a5
2018-07-26 04:08:43 +00:00
9a79920ef9 Static analysis fixes
Change-Id: I7bc42545a1c497f704d7bfa6ea1b7e3893acc697
2018-07-26 03:36:50 +00:00
130751ff66 fileobj: avoid memory leak in path recording 2018-05-14 17:46:52 +09:00
f3d18eb9de fileobj/devobj: record path name (originally by Takagi-san) 2018-05-14 17:46:52 +09:00
249bda4aef fileobj: use MCS locks for per-file page hash 2018-05-14 17:46:52 +09:00
d649d6fc2d Include mbind support (this is a rebase commit to merge into development) 2017-11-27 11:16:53 +09:00
aa7cb970c4 ihk_os_getrusage(): Compile LWK-specific results in mcctrl
1. User asks mcctrl for the result via ihk_os_getrusage() with passing void *
2. mcctrl compiles the results and passes them to the user
3. User interprets it by using the type defined in the LWK-specific header
2017-09-20 15:03:45 +09:00
9989f41fd3 add arm64 support
- add arm64 dependent codes with GICv3 and SVE support
- fix bugs based on architecture separation requests
2017-09-05 15:06:27 +09:00
74f15783d2 ihk_os_getrusage(): Add per-page-size memory usage accounting 2017-08-17 12:49:34 +09:00
184c2d311c fileobj_flush_page(): Not flush when MF_HOST_RELEASED 2017-08-17 12:49:34 +09:00
d7b8e7f4f4 fix to count user pages
refs #864
2017-07-14 09:51:39 +09:00
5758dba7cf use spinlocks in MCS rwlock 2017-06-08 14:16:29 +09:00
12e7ed644f fileobj_flush_page(): do not offload for files with MF_HOST_RELEASED flag set 2017-06-05 22:20:25 +09:00
e306b1e838 fileobj_create(): fix --mpol-shm-premap for Quadrant mode 2017-05-31 08:33:29 +09:00
9b5ccb5a33 Pre-map file mappings from /dev/shm (--mpol-shm-premap mcexec argument) 2017-05-23 20:00:06 +09:00
746b459e7f profile: more detailed profiling of file PFs 2017-05-23 19:58:52 +09:00
105d373765 PROFILE_page_fault_XXX: more detailed page PF profiling 2017-05-23 19:57:08 +09:00
e936b2ebe1 memobj_release: don't call syscall_generic_forwarding after process termination
refs #816
2017-03-10 12:58:47 +09:00
1ff0afe6fb devobj/fileobj: do not try to free memory for device file mappings 2017-01-08 14:16:10 +09:00
4437ecc69a do_mmap(): indicate user level allocations for anonymous mappings 2016-12-31 10:09:49 +09:00
9635a628a9 fileobj/shmobj/devobj: add file size to memobj 2016-12-19 12:55:12 +09:00
ff37ff9ccf memobj: synch prefetch among processes 2016-12-18 21:12:38 +09:00
5b7bcb7170 fileobj: use read/write MCS locks in page hash 2016-12-18 21:12:37 +09:00
bce10e11e4 fileobj: rewrite for scalability using per-file page hash 2016-12-18 21:12:37 +09:00
11125b0d68 fileobj and shmemobj: delete unused variables 2016-10-14 21:34:30 +09:00
f4db8b96de fileobj/shmobj: release pages correctly according to dynamic page frame management 2016-10-14 21:34:30 +09:00
8eb3bf3559 physical page management: eliminate static page frame array and
maintain page structures dynamically covering only file mappings.
use hash table for address <-> page structure conversion.
2016-10-14 21:34:29 +09:00
9b82f1a52c use ihk_mc_alloc/free_pages() and eliminate direct calls to low level routines 2016-10-14 21:34:29 +09:00
648bacc90f device file mappings: communicate map flags and fault missing translations 2016-06-24 12:44:59 -07:00
bd7dddd415 fileobj: implement lookup_page method 2016-03-23 19:14:28 +09:00
04e193de13 refactoring process structures 2015-10-13 23:04:08 +09:00
bf12a5c45e Introduction of write-combined memory type mappings.
Introduction of VR_WRITE_COMBINED, PTATTR_WRITE_COMBINED and modification
to the memobj's get_page() interface so that Linux communicates back mapping
flags (such as write-combined).
2015-03-05 16:03:21 +09:00
16af976a71 support msync() system call. refs #382
Msync(2) of this version writes only the pages which the calling process
modified. Modifications of the other processes are not written.
2015-02-18 11:52:15 +09:00
0942bf0ce0 make dkprintf() evaluate its parameters always
Parameters of dkprintf() should be evaluated even if dkprintf() is
disabled.  Because this enables to find expression of parameter obsolete
and to avoid unnecessary compiler warnings such as "unused variable".
2014-12-22 16:58:03 +09:00
9efb5e4fc5 add memobj_has_pager() 2014-07-07 18:11:36 +09:00
ce7af5839d keep page.count valid through its life cycle 2014-05-07 21:24:45 +09:00
29c7d40005 use atomic operations for manipulating page.count 2014-05-07 21:24:45 +09:00