Transient Memory #2
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Updated git tracking file list for git repo server.
If the dependency chain resolved out to a blocked thread, we wanted to return from the blocked dependent thread with an error as having a dependency on a blocked thread does not make sense. This commit does this.
1) change the assembly to always deal with a cos_stk structure, rather than the top of the stack. 2) change the stack manager to include a thread id at the end of the cos_stk structure, and ensure that we are always doing calculations with sizeof(struct cos_stk). Next step: save thread ids into the stack.
1) ensure that stacks hold the thread id of a thread using the stack 2) in the stack manager, make sure that we use the sched_block with dependency support to provide priority inheritance of current thread causing a stack miss to the the thread holding the stack. 3) take into account thread suspensions wrt the dependencies created. TODO: 1) if we attempt to make a dependency on a thread that is waiting for a stack in a component with self-suspended threads, then we should increase the self-suspension count of the component that actually has the threads with the self-suspensions...not the one we are stack missing in. 2) add support in the stack manager/policy to recognize self-suspension stack usage, and allocate stacks to components with SRT threads where the components have self-suspensions with priority to those that do not have self-suspensions.
Changed the period representation of NRT threads in Synthetic Hierachical home component. Set period to 0 instead of -1 denotes none real time threads. This solved the problem. Tested PIP with three threads(high, mid, low).
pre_block more than one time. Ensure each component at least owns one stack in stack manager. 1) when we try to depend on other thread, we would execute pre_block which is the same as usual blocking. But we are not sure if we should be modifing wake_cnt here. Now the problem is if we depend on other threads more than once, the wake_cnt would be less than 0. We fixed it by only decreasing wake_cnt if it is greater than 0. 2) Record a number of "empty components" in stack manager. And ensure the remaining stacks number is at least equal to the number of empty components. This can ensure each component has at least one stack quota. Also modified stack policy to ensure enough stacks for time event component(for temporal stack test).
Manager and Stack Policy to ensure allocating enough stacks to self-suspension components. 1) Add a self-suspension counter to each component in Stack Manager. Increase the counter when we can't find any thread to depend on, which means all threads are self-suspended and this component is self-suspension related (Not necessary to be where exactly self-suspension happened). 2) Allow self-suspension over-quota allocation privilege. Will ignore the quota limit of self-suspension components when granting stacks to them. 3) Add a new interface to stack manager(detect_self_suspension). It returns the self-suspension counter of the component, where a positive value means there is self-suspension and 0 means not. Stack policy uses this interface to gather information about self-suspension. And policy ensures enough quota for those components. 4) Fixed a bug: pre_wakeup should be called if a thread is waken up due to its dependency thread self-suspended.
self-suspension component, we should remove the current thread from the block list(in stk manager) before we return to obtain a stack.
allocation. Add a over-quota limit(ss_max) to each component and a global limit in stack manager. Fixed several bugs.
assembly code. Before we have to get the thread ID from memory. Now we get it from register(eax) directly.
Conflicts: src/components/Makefile.comp src/components/implementation/pgfault/high_level/Makefile src/components/implementation/pgfault/stk_trace/Makefile src/components/interface/pgfault/stubs/s_stub.S src/kernel/include/shared/cos_config.h src/platform/linux/util/Makefile.end
scripts which work well!
pointer every time. Scheduler now supports nested lock, eg. take valloc component lock while holding stack manager lock. 1) Stack manager, timed event and lock components use mem_mgr_large interface as they use valloc and malloc. 2) Add a ncs_held to the thread structure. It keeps the number of critical sections that are held by the thread. 3) Valloc component uses the lock provided by scheduler rather than lock component as this can avoid depending on lock component.
create a tmem lib. 1) tmem.h contains all static inline functions and general data structure. 2) tmem_tk.h declares functions implemented in the tmem lib and functions need to be implemented in the manager. 3) tmem.c implements functions that need to call manager. 4) specific data structures need to be defined in tmem_conf.h, which should be placed in the interface dir.
…ess from grant function. Fixed bugs of cbuf. 1) Interfaces can have DEPENDENCIES in the Makefile. This is useful for interfaces that depend on other interfaces, eg. cbuf interface depends on valloc and mem_mgr_large interfaces. 2) Instead of returning the address of a stack, stkmgr just add the stk into local free-list, and the thread returns from stkmgr and gets the stack from local free-list. This can unify interfaces and avoid some race conditions. 3) cbuf uses valloc now. Fixed the bug of object size calculation.
First step cbuf_vect implementation:
1. When use page_alias, spd can only push out, not pull in
2. Client needs to push/map its page to cbufmgr
Instead using of STATIC Array, 2 levels vectors are always created through vect_add_id() TODO: 1. Data structure in MGR to maintain pages 2. Add thd_id
been tested... Sync code first.
Conflicts: src/components/implementation/cbuf_c/naive/cbuf.c src/components/implementation/other/cbufc_test/cbufc.c src/components/include/cbuf.h src/components/interface/cbuf_c/cbuf_c.c
1. All memory allocation/deallocation have been moved to manager side (no asymmetric
allocation anymore) -- tested
2. Data structure spd_cbvect_range (link list) and its operation are implemeted in manager
to track cbufs mapped bwtween clients and server -- tested
3. cbuf2buf (cache_miss and cbuf_retrieve) have been implemented and tested. Curretly, both
server and client have 2 level structured vectors. -- tested
4. cbuf_free (cbuf_delete) is implemented without reference counting. Currently, when onwer's
cbuf is removed, all mapped cbufs in other spds are also removed -- testing
5. PIP implementation is supported. Currently thread id is saved right next to the cbuf meta
in vector. There are potential race condition when update these values -- testing
6. Most part of cbuf manager has been integrated into transient memory scheme,
but there is no policy and memory pool yet. -- To be continued
share all 2nd level vector ...
when the local cache can provide cbufs, they do not have to go back to mgr even when relinquish is set
when create/delete, both cbufs and slabs info need be updated
2. each cbuf itmem is corresponding to a single page now 3. each cbuf can contain multiple shared memory
Conflicts: src/components/implementation/cbuf_c/naive/cbuf.c src/components/implementation/cbuf_c/tmem.c src/components/implementation/cbuf_c/tmem.h src/components/implementation/exe_cbuf_synth_hier/exe_cbuf_shrec/exe_cbuf_synth_hier.c src/components/include/cbuf.h src/components/include/cos_asm_stkmgr_stacks.h src/components/interface/cbuf_c/cbuf_c.c src/components/lib/cos_component.c
Components that use lock compoent: mem_pool cbuf manager stack manager Compoents that use simple stack: time_event lock mem_pool cbuf manager stack manager
Conflicts: src/components/interface/periodic_wake/stubs/s_stub.S src/components/interface/timed_blk/stubs/s_stub.S
when mmem_page_alias/revoke/release
estimate concurrency. Included new waking up method from Gabe. But not used as we found some issues.
Conflicts: src/components/implementation/exe_cbuf_synth_hier/exe_cbuf_shrec/exe_cbuf_synth_hier.c src/components/interface/cbuf_c/cbuf_c.c
just for test. Remember to modify the number if used on different machine.
…ound a free item here. Return to the tmem_grant function and try to use this one.
Conflicts: src/components/implementation/other/tmem_policy/tmem_policy.c
Best effort threads are going to be added
Conflicts: src/components/interface/mem_pool/mem_pool.h
Tmem scripts after merge. Everything works well! Conflicts: src/Makefile.src src/components/Makefile.comp src/components/implementation/cbuf_c/naive/Makefile src/components/implementation/cbuf_c/naive/cbuf.c src/components/implementation/mem_mgr/naive/mem_man.c src/components/implementation/no_interface/boot/booter.c src/components/implementation/no_interface/cbufc_test/Makefile src/components/implementation/no_interface/cbufc_test/cbufc.c src/components/implementation/other/ping/ping.c src/components/implementation/pong/pingpong/pong.c src/components/implementation/sched/cos_scheduler.h src/components/implementation/stkmgr/naive/stkmgr.c src/components/implementation/stkmgr_new/naive/Makefile src/components/implementation/timed_blk/timed_evt/Makefile src/components/implementation/valloc/simple/Makefile src/components/implementation/valloc/simple/valloc.c src/components/include/cbuf.h src/components/interface/Makefile.subdir src/components/interface/cbuf_c/cbuf_c.c src/components/interface/lock/stubs/s_stub.S src/components/interface/valloc/stubs/s_stub.S src/kernel/include/shared/consts.h src/kernel/include/shared/cos_config.h src/platform/linux/module/modules.order src/platform/linux/net/modules.order src/platform/linux/util/Makefile.end
Merge branch 'master' of 128.164.157.181:~/master/composite Conflicts: src/components/implementation/sched/cos_scheduler.h
Merge branch 'master' of https://github.com/gparmer/Composite Conflicts: src/platform/linux/util/Makefile.end src/platform/linux/util/lpp.sh
gparmer
added a commit
that referenced
this pull request
Jan 19, 2012
Transient Memory -- Merge all work done for the RTAS '12 submission including: - stack manager - cbuf manager - tmem shared code - memory pool - assorted bug fixes
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Code has been merged and cleaned up.