Here is an updated set of BFS patches with the accumulated bugfixes as debugged on this blog for kernels 3.13 to 3.16 inclusive. The main obvious bug which affected people was the ath9k module which would hang on suspend/resume. However there were likely a number of subtle bugs across the board that most people would not be aware of and even I only noticed that kvm behaved much better after this applied bugfix which stretches back to every BFS after 3.12.
In order to make up for the fact that there are numerous kernels out there based on BFS across the different versions, I have updated BFS and numbered the versions according to which base kernel they are on. Note that there are no feature backports on the older kernels, only the bugfixes, so SMT nice is only on the 3.16 BFS.
3.13-sched-bfs-453.patch
3.14-sched-bfs-454.patch
3.15-sched-bfs-455.patch
3.16-sched-bfs-456.patch
And along with that an updated ck branded release for 3.16, 3.16-ck2:
3.16-ck2
Enjoy!
お楽しみ下さい
Just want to post here that I'm working on a multiple run queue lock solution for BFS recently. I have finished the first stage work and working on the next stage. I wrote a detail post at http://cchalpha.blogspot.com/2014/08/variable-run-queue-lockingvrql-for-bfs.html
ReplyDeleteAnd plz consider the code is very experimental and use for test only. Many thanks.
@ Alfred Chen:
DeleteCannot post on your BLOG. Closed one for "anonymous" :-(
Please, at first, sync your work to the current 3.16 kernel BFS-456/CK-2, as your code reads 20140818.
I won't test this, as this is way too old for the actual BFS/CK-2.
Give us a newer one -- then I'll test it.
Best regards, Manuel Krause
and a patch set would be nice, like con's broken out
Delete@Manuel
DeleteI have updated the BLOG setting. Please have a try.
I have updated linux-3.16.y-gc branch sync with 3.16.1 and bfs 456, the -vrq branch is not updated yet, as I'm still trying things on it.
Anyway, stay turned. :)
hi, thank you for the time you spend doing this, really, thank you soooooo much for the effort.
ReplyDeleteI'm writting a blog page in spanish for installing zfs on root with ubuntu, I will also post my experience with the ck series of patches, will post later.
thanks again.
this bfs 456 is perfect, functioning well before this version all versions of the kernel bfs after 3:12 had sporadic problems locking, preventing normal use of the computer. eutou using the ck2-3.16, and it is working perfectly, much faster than the official scaled kernel, the performance is much better, taking 30% less time on the task of converting video, best gaming performance and processes in parallel and indexing faster. Thank you for this launch my congratulations.
ReplyDeleteThanks for all of your efforts and outstanding code.
ReplyDeleteGalen
@CK: Thank you very much for your continued work!!! Fine on here! Regards, Manuel
ReplyDeleteHi Con
ReplyDeleteI have to prepare a fixed kernel 3.15.10 variant with smt nice 6
I have understood that to fix the ath9k trouble are two patches
I suppose the last one is
1>http://ck.kolivas.org/patches/bfs/3.0/3.15/test/3.15-ck1-smtnice6.patch
plus should add the fixes
2> http://ck.kolivas.org/patches/bfs/3.0/3.16/test/bfs450-sched-ipi.patch
3> http://ck.kolivas.org/patches/bfs/3.0/3.16/test/bfs450-add-preempt-resched.patch
so, it is ok?
thanks
bye, NicCo
Maybe you could also try first http://ck.kolivas.org/patches/bfs/3.0/3.15/3.15-sched-bfs-455.patch (what is dedicatedly for 3.15)
Deleteand then the http://ck.kolivas.org/patches/bfs/3.0/3.15/test/3.15-ck1-smtnice6.patch
Just an idea, that you'd get all BFS bugfixes in at first and then try to apply the SMTnice patch later. (I haven't tried this on my own, so far.)
Regards, Manuel
Good idea, it works!
Deletethanks
Regards, NicCo
Con, you've forgotten to revert KVM fix as it is not needed anymore, and I post it here just in case you'd like to include it into -ck3 or so.
ReplyDeletehttps://gist.github.com/a5e494db2b5c20bafa85
I'd like to use BFS on my laptop, but I wonder if doing so I would gain responsivity at the cost of loosing the energy-aware features in the official scheduler.
ReplyDeleteQuoting from phoronix: "Of the highlights for the scheduler tree with the Linux 3.16 merge window are NUMA scheduling updates for better performance, CPU idle changes to improve the high level idle scheduling logic, standardized idle polling across architectures, and continued work on preparing better power/energy-aware scheduling. Another change to point out is for using the deepest C-state always when in the "freeze" sleep state."
please read through http://ck-hack.blogspot.co.at/2014/08/smt-nice-3.html?showComment=1407719588192#c6086962841605930671
Deleteand to answer your question: personally I've been using kernel with BFS for a long time on my laptop and the battery runtime was always better compared to Windows Vista, 7, 8, etc.
and not necessarily worse than with CFS
in a nutshell: deeper sleep, lower C-states, etc. are still available with BFS - so you're not really losing anything and gaining smoother operation
to the naysayers:
where BFS+BFQ really shine is under heavy load and true multi-tasking, which of course can't be tested with most benchmarks
Thanks kOT.
Delete@ck
ReplyDeleteI think I found a resched_best_idle issue when investigating the regression of VRQ solution. In short, resched_best_idle() should not be called when prev is idle or prev is deactivated. For detail, please check my post at http://cchalpha.blogspot.com/2014/08/after-reversed-commit-which-i-bitsect.html
Code change is https://bitbucket.org/alfredchen/linux-gc/commits/4af4686c764f326b8a409176776281fcdc3effd0?at=linux-3.16.y-gc
The tests show that there is abut 3% improvement for 50% jobs/cores ratio throughput test.
Are you suggesting to add your fix "Don't reschedule an idle task or deactivated task" also on the latest official 3.15-sched-bfs-455.patch ?
DeleteRegards, NicCo
Feed back request here
Deletehttp://ck-hack.blogspot.com.au/2014/08/smthyperthreading-nice-and-scheduling.html?showComment=1409445323216#c6490839717259700774
Im new to linux and just built a desktop running an AMD 8 core cpu.
There are 2 related patches.
DeleteThe most important one is "Don't reschedule an idle task or deactivated task" in the above post, Here is a dedicated patch can clean apply upon bfs 0456.
https://bitbucket.org/alfredchen/linux-gc/downloads/0456-resched_best_idle.patch
An other related patch that I have posted in 3.15 thread is
bfs: Remove unnecessary resched_suitable_idle() in schedule().
https://bitbucket.org/alfredchen/linux-gc/commits/daf5b14d1186a9d9626fd400cf2bc3acd3f68a36?at=v3.16.1-gc
It's on not a very hot code path, you can safely ignore it.
BR Alfred
resched_suitable_idle is still required. Please do not remove it as you seem to misunderstand how it would be helpful.
Delete@ck plz check my post.
DeleteI am not remove all resched_suitable_idle() calls. Just add the condition doesn't call resched_suitable_idle for idle task and deactivated task, it's no point to reschedule these tasks to what best idle cpu.
I believe to have understood that:
Deletehttps://bitbucket.org/alfredchen/linux-gc/downloads/0456-resched_best_idle.patch
1> the first one, is that you have tested a gain of about 3% to the plain bfs
2> while for the second modify, what is the purpose of that?
what improvement or what fixing from the second change?
https://bitbucket.org/alfredchen/linux-gc/commits/daf5b14d1186a9d9626fd400cf2bc3acd3f68a36?at=v3.16.1-gc
where is the dedicated patch can clean apply upon bfs456?
thanks a lot,
Regards, NicCo
@NicCo
DeleteI decided to combine these resched_best_idle() related patch together and make it clean apply upon 0456.
Here is the patch, plz have a try
https://bitbucket.org/alfredchen/linux-gc/commits/f6ee9a2411dab6bcf75642464a91a57866877c15
BR Alfred
@Alfred Chen
DeleteI am using it on 3.15.10, so far all is working fine
thanks
NicCo
Do we still need to apply https://gist.githubusercontent.com/pfactum/9332896/raw/0001-ck-3.12-fix-BFS-compiling-with-CONFIG_SMP-n.patch on top of this updated 3.14 BFS patch ? I tested it and it still applies cleanly, but don't know for sure if we still need it.
ReplyDeletenot sure if the build got fixed
Deletebut if you're compiling your kernel with
CONFIG_SMP=y
you do *not* need this patch
only if
# CONFIG_SMP is not set
is set in kernel config (usually /usr/src/linux/.config)
Hi,
ReplyDeleteI maintain the linux-lts-ck package in the AUR and I created a bfs447-454.patch for the 3.14 to apply on top of 3.14-ck1.
You can look at it here if you wish to adpot the patch in your repository: http://pkgbuild.com/git/aur-mirror.git/diff/linux-lts-ck/bfs447-454.patch?id=957fe91142c6524ee2fbb0d342a8a255db2a627d
Has this the HT/SMT Nice 6 feature applied? Not...
DeleteIt would be great having that HT/SMT into LTS 3.14
for all those people using the current LTS Kernel
Regards, NicCo
Not at the moment.
DeleteI could probably backport it, but I'm currently busy dealing with another (kernel) package I maintain.
Claire
OK, so here's an untested mockup patch of smtnice-6 onto 3.14 + bfs454:
ReplyDeletehttps://github.com/clfarron4/linux-lts-ck-archlinux/blob/master/bfs454-smtnice-6.patch
If you study it carefully, you should realise it's the same as applying the following file straight onto 3.14 + bfs454:
http://ck.kolivas.org/patches/bfs/3.0/3.15/test/bfs449-smtnice-6.patch
I will get around to testing it when I've some free time from this other package.
I tried smtnice-6 onto 3.14 + bfs454
Deletepatches applied then had build issue
What was the error?
DeleteClaire
+ /usr/bin/make -j2 -s all
Deletekernel/sched/bfs.c:814:25: warning: ‘thread_cpumask’ used but never defined [enabled by default]
....................................................
....................................................
kernel/built-in.o: In function `smt_should_schedule':
bfs.c:(.text+0x2ffbc): undefined reference to `thread_cpumask'
kernel/built-in.o: In function `check_smt_siblings':
bfs.c:(.text+0x30084): undefined reference to `thread_cpumask'
kernel/built-in.o: In function `schedule':
(.sched.text+0x1234): undefined reference to `thread_cpumask'
make: *** [vmlinux] Error 1
Regards, NicCo
Hi,
DeleteThere are more changes between 3.14-ck1 and 3.16-ck2 than I am prepared to play with, given my lack of programming ability, to integrate smtnice6 into the older versions of BFS.
So, I'll leave it alone for now (though curiosity may kill the cat in time).
Claire
With BFS and "Debug preemptible kernel" enabled I get lots of warnings about smp_processor_id() usage while using qemu/kvm. Is that BFS issue?
ReplyDeleteDisabling "Debug preemptible kernel" just mutes warnings, and everything still works OK.
It has been reported that kernel 3.15.10 with ht/smt nice6 enabled
ReplyDelete- vmware 10 > the linux guests with ht/smt
these linux guests run very slow, perhaps the cause may be:
> "ata_sff" stays forever at 99%
do you have some experience about that?
thanks
bye, NicCo
Very nice :)
ReplyDelete@Alfred Chen:
ReplyDeleteHave you already begun to re-propagate your pending patches of your pure BFS branch to Con? Before he'd release his 3.17 version... BTW, they're running well since published.
Best regards to all of you,
Manuel Krause
Only patch I am proposing for 3.17 BFS is still the 3% low workload performance boost code change. Other patches are minor code changes which very hard to measure.
Deletehttps://bitbucket.org/alfredchen/linux-gc/commits/ad9dd03db1002717f155c859ee613641620d3ba0?at=linux-3.16.y-gc
Thank you for publishing your BFS-port to 3.17 below! I really hope, CK would pick up --at least-- some of your improvements.
DeleteCurrently I'm very glad with the 3.16 kernel until now: 3.16.5 + bfs/ck2 + gc-patches + bfq + my reworked revert patch for "drm-i915-Move-all-ring-resets-before-setting-the-HWS-page".
In this fashion, this kernel gets the least annoying and best performing one since months.
BTW, would someone of you NOT recommend to enable CONFIG_OPTIMIZE_INLINING ? I'm not completely sure whether to blame recent mainline changes or this config setting. Overall performance and responsiveness seems to be better with it enabled?! My gcc is a 4.8.3 20140627 [gcc-4_8-branch revision 212064].
Many thanks to all participants in all involved projects!!!
Manuel Krause
P.S.: Not only the upgraded bfq is missing, I'd also wish to see a new TuxOnIce for 3.17. ^^
Thanks for the head-up. I have started porting 0456 to 3.17 last weekend and there still some sync-up with mainline core.c left, but it already running on one of my machines. Hopefully my porting can be released in a day or two.
ReplyDeleteBR Alfred.
Hi, all,
ReplyDeleteMy porting of bfs 0456 to 3.17 is ready, if you want to try bfs on 3.17 before new bfs release by ck, you can apply the blow 2 patches upon 3.17.
#1 Original BFS 0456 apply on 3.17
https://bitbucket.org/alfredchen/linux-gc/commits/f9bf49826dda7530106b50e337b030514f9aadd6
#2 bfs: 0456 porting to 3.17 and sync with mainline.
https://bitbucket.org/alfredchen/linux-gc/commits/35d0568d06e79d42e6643aee5169579e7f1aabdb
The others patches are not official accepted by ck, but as bug fix and improvement, apply them as you like.
#3 revert KVM workaround due to proper cond_resched() fix, by pf
https://bitbucket.org/alfredchen/linux-gc/commits/6531d1dca19b58b544fab697ac3b10ccd2ad5b40
#4 bfs: Refine resched_best_idle() call logic in schedule(). ---- the 3% performance boot patch under low workload.
https://bitbucket.org/alfredchen/linux-gc/commits/993df75b0292abd1c8e16bf5a43d32eebf7f0231
...
other bfs related patches on my linux-3.17.y-gc branch are refactory/improvement that normally hard to be noticeable.
https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-3.17.y-gc
PS: my -gc branch is not yet completed, still waiting for new release of BFQ
BR Alfred
Thanks a lot Alfred
DeleteTesting right now =)
Wasn't aware that you're a Gentoo-user, too
BFQ v7r6 is out, btw:
https://groups.google.com/forum/#!topic/bfq-iosched/EufEsuDaEuU
I have prepared kernel 3.17.1 with your four suggested patches, installed, now I'm running this kernel since few hours, and is working fine...
ReplyDeletenow I'm searching for USKM and TOI for kernel 3.17: there are unofficial patches of these two?
Thank you!
NicCo
Seems like all Australians are on holiday? ;-) O.k. they should enjoy, of course!
ReplyDeleteI can encourage you to test Alfred Chen's latest -gc patches for the 3.17 kernel. Unfortunately, these are up to 18 (!) separate patches to be applied, including the last BFS with enhancements. I hope, he would provide a all-in-one complete patch sometimes. But so he let's us choose our dose of poison. They apply well on 3.17.2 and do also work well. Two days of everyday's desktop work based testing.
http://cchalpha.blogspot.de/2014/11/whats-new-for-317-gc-patch-set.html
I really miss a new revision of TuxOnIce. :-(
Best regards,
Manuel Krause
As you may notice, I've released 3.17-pf1 with TuxOnIce as well.
Delete--post-factum
@post-factum
DeleteThanks for the reminder! And of course for your work, too.
With the help of your shortlog for 3.17-pf1 and your and the kernel git repository, I was able to patch my 3.17.2 with the "old" TuxOnIce on here yesterday. (This enables me to not use your monolithic -pf1 patch, but to patch several addons separately, e.g. more of Alfred's improvements than you've included and to also omit UKSM what has shown to be prone for errors in my past experience.)
It's working fine and seems to be stable.
Best regards, Manuel Krause
@Manuel
ReplyDeleteThanks for testing. The reason I go for separated patches is it's easier to bisect and find out which patch cause the problem if something wrong happened.
This way is o.k. like it is.
DeleteBTW, your 3.17-gc patches do still work well on here. Now with post-factum's style TOI makeup. ;-)
Manuel
no more ck-patches
ReplyDeleteNah just too busy, lack of time and enthusiasm. I will get around to it eventually.
DeleteThanks so much for your work on this.
ReplyDeleteReally appreciated.