Wednesday, 5 October 2016

MuQSS - The Multiple Queue Skiplist Scheduler v0.106

Another day and time for yet another release.

There are 0.106 versions and incrementals available for linux-4.7:
 http://ck.kolivas.org/patches/muqss/4.0/4.7/
and linux-4.8:
http://ck.kolivas.org/patches/muqss/4.0/4.8


Two large remaining races that could lead to warnings, stalls, or in the worst case, crashes, have been fixed in this version.


Additionally the multiple-runqueue locking has been significantly optimised to take only the runqueues needed for as long as they're needed only and dropped as soon as possible which should bring the lock contention levels down even further. This is a performance enhancement, more so in non-interactive mode, though it will only start being demonstrable if you're lucky enough to have many CPUs.


This version addresses all the known bugs and warnings I've received to date so hopefully I can have a little rest and let people out there actually give it a go. What will you expect if you use this instead of BFS? If I've done this correctly, you will notice absolutely no difference since the idea was to preserve the interactivity and responsiveness of BFS and make it scalable to more CPUs than most people can afford.


Keep the feedback coming, thanks.

Enjoy!
 お楽しみ下さい
-ck
Posted by at
Labels: , , ,

23 comments:

  1. Anonymous6 October 2016 at 03:30

    Also this 106 muxes very well on my machine with kernel 4.7.6 (+ BFQv8r3, WBTv7 and my TOI-mod). :-)
    Thanks and BR,
    Manuel Krause

    ReplyDelete
  2. Anonymous6 October 2016 at 05:49

    Thanks Con.
    I've tested linux 4.8 + muqss106.
    https://docs.google.com/spreadsheets/d/1ZfXUfcP2fBpQA6LLb-DP6xyDgPdFYZMwJdE0SQ6y3Xg/edit?usp=sharing

    No more errors related to xfs in kernel logs after the tests (I had errors with linux 4.8 + muqss105 and below).
    On the throughput side, interactive=0 with frequency scaling enabled (intel_pstate) give strange results on make j2 and bz2.
    Enjoy some rest.

    Pedro

    ReplyDelete
    Replies
    1. ck6 October 2016 at 06:52

      Thanks very much for those as always. The changes in 106 should have addressed your xfs bug (along with bugs affecting other people.) There are hopefully still improvements that can be made to performance but my main priority at this early stage has been stability.

      Delete
    2. ck6 October 2016 at 10:22

      Actually I'm pretty sure I know why the scaling with intel_pstate is underperforming now. The 002 pending patch for 106 should improve things. http://ck.kolivas.org/patches/muqss/4.0/4.8/Pending/

      Delete
    3. Anonymous7 October 2016 at 00:59

      Thanks Con. You got it !
      I tested muqss106 + the 3 first patches in pending. No more regressions when using pstate and interactive=0.

      By the way I noticed you already put 3 more patches in pending. Well you didn't rest to long :)
      And don't feel rushed by the benchmarks I do, it's just that I have free time and a test machine.

      Pedro

      Delete
    4. Anonymous7 October 2016 at 07:34

      @Pedro:
      Can you please take older results, not compliant to your current testing scenario, off this chart/ record?
      If you've time, please re-do the 'old' testings with your current setup, but don't leave the old results in the chart, as it may lead to confusion. They get already cited on phoronix. But they may don't know how to deal with.

      BR, Manuel Krause

      Delete
    5. Anonymous7 October 2016 at 08:08

      @Manuel
      Thanks for the suggestion. I know I'm piling up the results for now as Con is so productive. Probably they don't make so much sense for someone who don't follow this blog.
      I'll try to make them clearer.

      Pedro

      Delete
    6. Anonymous8 October 2016 at 02:04

      @Pedro:
      Sorry for having suggested this. I wasn't aware how much work it would mean. (Sometimes I still forget that this effort is all voluntary.)
      But now the results are much clearer, with info, and properly divided into old and new. :-)

      Thank you and BR, Manuel Krause

      Delete
  3. Anonymous7 October 2016 at 02:11

    Finally got my x64 dev PC (old Nforce MCP61, Athlon64 X2) going again recently... I built linux-ck (Arch AUR) with the MQS 106 Scheduler, including both pending patches; all looking good so far, but didn't use it much yet. Thanks Con!! Side note: I personally prefer MQS or MQSS, as some have seen 'MuQSS' as 'mucus/mucous' rather than 'mux/mucks', plus its simpler. :)
    -jeremywh

    ReplyDelete
    Replies
    1. Anonymous7 October 2016 at 07:07

      The "mux" pronounciation is much nearer than any other one -- even for nonnative speakers. Noone would ever associate slimy stuff to either BFS or MuQSS, even if brain-fucked or not. And is not a topic of discussion. Con decided. Period.
      The main effort is: Multiple Queue Skiplist Scheduler works well.
      BR, Manuel Krause

      Delete
    2. Anonymous7 October 2016 at 09:32

      >even if brain-fucked or not
      Haha; nicely done Manuel :)

      However, 'Mu' on its own is usually pronounced as 'myoo/mew', the greek letter, which is how others got to thinking this way, like for the first forum post on a prominent Linux news site:
      https://www.phoronix.com/forums/forum/software/general-linux-open-source/902223-bfs-updated-for-linux-4-8-to-be-succeeded-by-new-muqss-scheduler?p=902237#post902237
      My other reasoning was tradition; the schedulers are usually 3-letter acronyms. I'm just looking at it in marketing/semantics terms...but hey, fuck tradition... :-P

      Btw, what do you think of M{,u}QSS vs Alfred's -vrq so far?
      -jwh

      Delete
    3. ck7 October 2016 at 09:38

      MQS already has over 6 million hits on google. MuQSS isn't even a week old and is about the only significant search hit.

      Delete
    4. Anonymous7 October 2016 at 09:45

      Touche sir Con; yup I see something about a 'MQS LINUX' in a Google search. MQSS also has random old hits as well. Well done :)
      -jwh

      Delete
    5. Anonymous8 October 2016 at 02:34

      @-jwh:
      Yes, yes, I was amused about this Phoronix forum thread, as the postings mainly only deal with Con's naming decision (and not with the innovation). And that at a time when Con already had explained on his blog how he came to "MuQSS". Looking back, it's quite same amusing, that Con delivered the pronounciation help already with his first announcement of MuQSS, "mux", so that noone should overlook how to speak "multiple"... and notice this new key feature vs. BFS.
      Btw., doesn't MuQSS look much more scientifically founded than BFS (even without knowing the latter's exact wording)? ;-) And, don't come with Sex sells"... :-P

      BR, Manuel Krause

      Delete
    6. jwh78 October 2016 at 23:55

      Ya... Those news article forums always devolve into (or start as) BS like that though. Anything related to systemd is the worst. :-)

      Delete
  4. Anonymous7 October 2016 at 07:17

    @CK:
    Also with the current 6 patches from 4.8/pending applied to my 4.7.6 kernel, everythings is behaving fine.
    BR, Manuel Krause

    ReplyDelete
  5. Anonymous7 October 2016 at 09:40

    Holy shit Con; I thought you were gonna take a break!? :-P
    Update; I've suspended/resumed a couple times w/o issue, and been using the -ck dev PC for an hour; getting ready to build this for my old netbook (Asus Eee 701, Intel Atom), after I update it for the new patches. :)

    ReplyDelete
    Replies
    1. ck7 October 2016 at 09:41

      Heh and there's already more to come in the next few minutes. When I said rest I mean I'll at least try and get some normal sleep. It was keeping me up into the wee hours of the morning when it was still unstable.

      Delete
    2. Anonymous7 October 2016 at 09:52

      I know what you mean man; I had lots of issues migrating this dev PC from Win7/Arch/syslinux/IDE to Win10/Arch/GRUB/RAID(SATA-AHCI)...both in Windows and Win/Lin bootloaders.

      Up late, plus I was hella excited to install the MuQSS kernel (built overnight) this morning...delayed me getting to work. :)

      But I digress...more when I get this netbook i686 MuQSS kernel built! Rest well!
      -jwh

      Delete
  6. ck7 October 2016 at 10:34

    v107 coming up so if you're about to, or are in the middle of, building a new one, hold off. If you have all 8 pending patches for 106 you're good to go already.

    ReplyDelete
    Replies
    1. ck7 October 2016 at 10:48

      Cancel that, I have one more pending patch too...

      Delete
    2. kernelOfTruth7 October 2016 at 10:56

      FWIW the ttwu (106-007) patch fails on 4.7 with:

      * ERROR: Failed to compile the "bzImage" target...
      *
      * -- Grepping log... --
      *
      * CC security/apparmor/resource.o
      * CC security/integrity/integrity_audit.o
      * CC kernel/rcu/srcu.o
      * CC arch/x86/kernel/fpu/bugs.o
      *kernel/sched/MuQSS.c: In function ‘try_to_wake_up’:
      *kernel/sched/MuQSS.c:1826:2: error: implicit declaration of function ‘smp_cond_load_acquire’ [-Werror=implicit-function-declaration]
      * smp_cond_load_acquire(&p->on_cpu, !VAL);
      * ^
      *kernel/sched/MuQSS.c:1826:37: error: ‘VAL’ undeclared (first use in this function)
      * smp_cond_load_acquire(&p->on_cpu, !VAL);
      * ^
      *kernel/sched/MuQSS.c:1826:37: note: each undeclared identifier is reported only once for each function it appears in
      *kernel/sched/MuQSS.c: In function ‘sched_fork’:
      *kernel/sched/MuQSS.c:1933:13: error: ‘TASK_NEW’ undeclared (first use in this function)
      *--
      * CC security/integrity/digsig.o
      * LD security/apparmor/apparmor.o
      * CC fs/ioctl.o
      * LD security/apparmor/built-in.o
      * CC fs/readdir.o
      *cc1: some warnings being treated as errors
      *make[2]: *** [scripts/Makefile.build:289: kernel/sched/MuQSS.o] Error 1
      *make[1]: *** [scripts/Makefile.build:440: kernel/sched] Error 2
      *--
      * LD arch/x86/kernel/fpu/built-in.o
      * CC arch/x86/kernel/ptrace.o
      * CC fs/file.o
      * CC arch/x86/kernel/tls.o
      * LD kernel/rcu/built-in.o
      *make: *** [Makefile:988: kernel] Error 2

      Delete
    3. ck7 October 2016 at 10:57

      Darn. I guess 4.7 will start falling behind then sorry. I don't have time to keep both in sync.

      Delete

Subscribe to: Post Comments (Atom)