Monday, 25 August 2014

BFS 453/454/455/456 and 3.16-ck2

Here is an updated set of BFS patches with the accumulated bugfixes as debugged on this blog for kernels 3.13 to 3.16 inclusive. The main obvious bug which affected people was the ath9k module which would hang on suspend/resume. However there were likely a number of subtle bugs across the board that most people would not be aware of and even I only noticed that kvm behaved much better after this applied bugfix which stretches back to every BFS after 3.12.

In order to make up for the fact that there are numerous kernels out there based on BFS across the different versions, I have updated BFS and numbered the versions according to which base kernel they are on. Note that there are no feature backports on the older kernels, only the bugfixes, so SMT nice is only on the 3.16 BFS.

3.13-sched-bfs-453.patch
3.14-sched-bfs-454.patch
3.15-sched-bfs-455.patch
3.16-sched-bfs-456.patch

And along with that an updated ck branded release for 3.16, 3.16-ck2:

3.16-ck2


Enjoy!
お楽しみ下さい
Posted by at
Labels: , , , , ,

52 comments:

  1. Alfred Chen25 August 2014 at 21:33

    Just want to post here that I'm working on a multiple run queue lock solution for BFS recently. I have finished the first stage work and working on the next stage. I wrote a detail post at http://cchalpha.blogspot.com/2014/08/variable-run-queue-lockingvrql-for-bfs.html

    And plz consider the code is very experimental and use for test only. Many thanks.

    ReplyDelete
    Replies
    1. Anonymous26 August 2014 at 12:03

      @ Alfred Chen:
      Cannot post on your BLOG. Closed one for "anonymous" :-(

      Please, at first, sync your work to the current 3.16 kernel BFS-456/CK-2, as your code reads 20140818.
      I won't test this, as this is way too old for the actual BFS/CK-2.
      Give us a newer one -- then I'll test it.

      Best regards, Manuel Krause

      Delete
    2. Anonymous27 August 2014 at 01:56

      and a patch set would be nice, like con's broken out

      Delete
    3. Alfred Chen30 August 2014 at 19:26

      @Manuel
      I have updated the BLOG setting. Please have a try.

      I have updated linux-3.16.y-gc branch sync with 3.16.1 and bfs 456, the -vrq branch is not updated yet, as I'm still trying things on it.

      Anyway, stay turned. :)

      Delete
  2. nestorespinoza26 August 2014 at 07:59

    hi, thank you for the time you spend doing this, really, thank you soooooo much for the effort.
    I'm writting a blog page in spanish for installing zfs on root with ubuntu, I will also post my experience with the ck series of patches, will post later.

    thanks again.

    ReplyDelete
  3. beto26 August 2014 at 12:05

    this bfs 456 is perfect, functioning well before this version all versions of the kernel bfs after 3:12 had sporadic problems locking, preventing normal use of the computer. eutou using the ck2-3.16, and it is working perfectly, much faster than the official scaled kernel, the performance is much better, taking 30% less time on the task of converting video, best gaming performance and processes in parallel and indexing faster. Thank you for this launch my congratulations.

    ReplyDelete
  4. Anonymous26 August 2014 at 12:05

    Thanks for all of your efforts and outstanding code.

    Galen

    ReplyDelete
  5. Anonymous26 August 2014 at 12:13

    @CK: Thank you very much for your continued work!!! Fine on here! Regards, Manuel

    ReplyDelete
  6. Anonymous26 August 2014 at 16:14

    Hi Con

    I have to prepare a fixed kernel 3.15.10 variant with smt nice 6
    I have understood that to fix the ath9k trouble are two patches

    I suppose the last one is
    1>http://ck.kolivas.org/patches/bfs/3.0/3.15/test/3.15-ck1-smtnice6.patch
    plus should add the fixes
    2> http://ck.kolivas.org/patches/bfs/3.0/3.16/test/bfs450-sched-ipi.patch
    3> http://ck.kolivas.org/patches/bfs/3.0/3.16/test/bfs450-add-preempt-resched.patch

    so, it is ok?


    thanks
    bye, NicCo

    ReplyDelete
    Replies
    1. Anonymous27 August 2014 at 14:14

      Maybe you could also try first http://ck.kolivas.org/patches/bfs/3.0/3.15/3.15-sched-bfs-455.patch (what is dedicatedly for 3.15)
      and then the http://ck.kolivas.org/patches/bfs/3.0/3.15/test/3.15-ck1-smtnice6.patch

      Just an idea, that you'd get all BFS bugfixes in at first and then try to apply the SMTnice patch later. (I haven't tried this on my own, so far.)

      Regards, Manuel

      Delete
    2. Anonymous31 August 2014 at 01:01

      Good idea, it works!

      thanks
      Regards, NicCo

      Delete
  7. Oleksandr Natalenko27 August 2014 at 00:03

    Con, you've forgotten to revert KVM fix as it is not needed anymore, and I post it here just in case you'd like to include it into -ck3 or so.

    https://gist.github.com/a5e494db2b5c20bafa85

    ReplyDelete
  8. Anonymous28 August 2014 at 00:37

    I'd like to use BFS on my laptop, but I wonder if doing so I would gain responsivity at the cost of loosing the energy-aware features in the official scheduler.

    Quoting from phoronix: "Of the highlights for the scheduler tree with the Linux 3.16 merge window are NUMA scheduling updates for better performance, CPU idle changes to improve the high level idle scheduling logic, standardized idle polling across architectures, and continued work on preparing better power/energy-aware scheduling. Another change to point out is for using the deepest C-state always when in the "freeze" sleep state."

    ReplyDelete
    Replies
    1. kernelOfTruth1 September 2014 at 12:15

      please read through http://ck-hack.blogspot.co.at/2014/08/smt-nice-3.html?showComment=1407719588192#c6086962841605930671

      and to answer your question: personally I've been using kernel with BFS for a long time on my laptop and the battery runtime was always better compared to Windows Vista, 7, 8, etc.

      and not necessarily worse than with CFS

      in a nutshell: deeper sleep, lower C-states, etc. are still available with BFS - so you're not really losing anything and gaining smoother operation


      to the naysayers:
      where BFS+BFQ really shine is under heavy load and true multi-tasking, which of course can't be tested with most benchmarks

      Delete
    2. Anonymous1 September 2014 at 16:20

      Thanks kOT.

      Delete
  9. Alfred Chen30 August 2014 at 19:18

    @ck
    I think I found a resched_best_idle issue when investigating the regression of VRQ solution. In short, resched_best_idle() should not be called when prev is idle or prev is deactivated. For detail, please check my post at http://cchalpha.blogspot.com/2014/08/after-reversed-commit-which-i-bitsect.html

    Code change is https://bitbucket.org/alfredchen/linux-gc/commits/4af4686c764f326b8a409176776281fcdc3effd0?at=linux-3.16.y-gc

    The tests show that there is abut 3% improvement for 50% jobs/cores ratio throughput test.

    ReplyDelete
    Replies
    1. Anonymous31 August 2014 at 00:55

      Are you suggesting to add your fix "Don't reschedule an idle task or deactivated task" also on the latest official 3.15-sched-bfs-455.patch ?

      Regards, NicCo

      Delete
    2. Anonymous31 August 2014 at 10:50

      Feed back request here
      http://ck-hack.blogspot.com.au/2014/08/smthyperthreading-nice-and-scheduling.html?showComment=1409445323216#c6490839717259700774
      Im new to linux and just built a desktop running an AMD 8 core cpu.

      Delete
    3. Alfred Chen31 August 2014 at 11:34

      There are 2 related patches.
      The most important one is "Don't reschedule an idle task or deactivated task" in the above post, Here is a dedicated patch can clean apply upon bfs 0456.
      https://bitbucket.org/alfredchen/linux-gc/downloads/0456-resched_best_idle.patch

      An other related patch that I have posted in 3.15 thread is
      bfs: Remove unnecessary resched_suitable_idle() in schedule().

      https://bitbucket.org/alfredchen/linux-gc/commits/daf5b14d1186a9d9626fd400cf2bc3acd3f68a36?at=v3.16.1-gc

      It's on not a very hot code path, you can safely ignore it.

      BR Alfred

      Delete
    4. ck31 August 2014 at 11:38

      resched_suitable_idle is still required. Please do not remove it as you seem to misunderstand how it would be helpful.

      Delete
    5. Alfred Chen31 August 2014 at 11:50

      @ck plz check my post.
      I am not remove all resched_suitable_idle() calls. Just add the condition doesn't call resched_suitable_idle for idle task and deactivated task, it's no point to reschedule these tasks to what best idle cpu.

      Delete
    6. Anonymous1 September 2014 at 18:30

      I believe to have understood that:
      https://bitbucket.org/alfredchen/linux-gc/downloads/0456-resched_best_idle.patch
      1> the first one, is that you have tested a gain of about 3% to the plain bfs

      2> while for the second modify, what is the purpose of that?
      what improvement or what fixing from the second change?
      https://bitbucket.org/alfredchen/linux-gc/commits/daf5b14d1186a9d9626fd400cf2bc3acd3f68a36?at=v3.16.1-gc

      where is the dedicated patch can clean apply upon bfs456?

      thanks a lot,
      Regards, NicCo

      Delete
    7. Alfred Chen1 September 2014 at 23:02

      @NicCo
      I decided to combine these resched_best_idle() related patch together and make it clean apply upon 0456.

      Here is the patch, plz have a try
      https://bitbucket.org/alfredchen/linux-gc/commits/f6ee9a2411dab6bcf75642464a91a57866877c15

      BR Alfred

      Delete
    8. Anonymous11 September 2014 at 21:07

      @Alfred Chen
      I am using it on 3.15.10, so far all is working fine

      thanks
      NicCo

      Delete
  10. Unknown1 September 2014 at 01:07

    Do we still need to apply https://gist.githubusercontent.com/pfactum/9332896/raw/0001-ck-3.12-fix-BFS-compiling-with-CONFIG_SMP-n.patch on top of this updated 3.14 BFS patch ? I tested it and it still applies cleanly, but don't know for sure if we still need it.

    ReplyDelete
    Replies
    1. kernelOfTruth1 September 2014 at 12:09

      not sure if the build got fixed

      but if you're compiling your kernel with
      CONFIG_SMP=y

      you do *not* need this patch

      only if
      # CONFIG_SMP is not set

      is set in kernel config (usually /usr/src/linux/.config)

      Delete
  11. Unknown1 September 2014 at 21:24

    Hi,

    I maintain the linux-lts-ck package in the AUR and I created a bfs447-454.patch for the 3.14 to apply on top of 3.14-ck1.

    You can look at it here if you wish to adpot the patch in your repository: http://pkgbuild.com/git/aur-mirror.git/diff/linux-lts-ck/bfs447-454.patch?id=957fe91142c6524ee2fbb0d342a8a255db2a627d

    ReplyDelete
    Replies
    1. Anonymous1 September 2014 at 22:28

      Has this the HT/SMT Nice 6 feature applied? Not...
      It would be great having that HT/SMT into LTS 3.14
      for all those people using the current LTS Kernel

      Regards, NicCo

      Delete
    2. Unknown1 September 2014 at 23:55

      Not at the moment.

      I could probably backport it, but I'm currently busy dealing with another (kernel) package I maintain.

      Claire

      Delete
  12. Unknown2 September 2014 at 03:42

    OK, so here's an untested mockup patch of smtnice-6 onto 3.14 + bfs454:

    https://github.com/clfarron4/linux-lts-ck-archlinux/blob/master/bfs454-smtnice-6.patch

    If you study it carefully, you should realise it's the same as applying the following file straight onto 3.14 + bfs454:

    http://ck.kolivas.org/patches/bfs/3.0/3.15/test/bfs449-smtnice-6.patch

    I will get around to testing it when I've some free time from this other package.

    ReplyDelete
    Replies
    1. Anonymous3 September 2014 at 00:46

      I tried smtnice-6 onto 3.14 + bfs454
      patches applied then had build issue

      Delete
    2. Anonymous3 September 2014 at 18:03

      What was the error?

      Claire

      Delete
    3. Anonymous28 September 2014 at 08:01

      + /usr/bin/make -j2 -s all
      kernel/sched/bfs.c:814:25: warning: ‘thread_cpumask’ used but never defined [enabled by default]
      ....................................................
      ....................................................
      kernel/built-in.o: In function `smt_should_schedule':
      bfs.c:(.text+0x2ffbc): undefined reference to `thread_cpumask'
      kernel/built-in.o: In function `check_smt_siblings':
      bfs.c:(.text+0x30084): undefined reference to `thread_cpumask'
      kernel/built-in.o: In function `schedule':
      (.sched.text+0x1234): undefined reference to `thread_cpumask'
      make: *** [vmlinux] Error 1

      Regards, NicCo

      Delete
    4. Unknown3 November 2014 at 22:46

      Hi,

      There are more changes between 3.14-ck1 and 3.16-ck2 than I am prepared to play with, given my lack of programming ability, to integrate smtnice6 into the older versions of BFS.

      So, I'll leave it alone for now (though curiosity may kill the cat in time).

      Claire

      Delete
  13. Oleksandr Natalenko12 September 2014 at 04:58

    With BFS and "Debug preemptible kernel" enabled I get lots of warnings about smp_processor_id() usage while using qemu/kvm. Is that BFS issue?

    Disabling "Debug preemptible kernel" just mutes warnings, and everything still works OK.

    ReplyDelete
  14. Anonymous19 September 2014 at 23:56

    It has been reported that kernel 3.15.10 with ht/smt nice6 enabled
    - vmware 10 > the linux guests with ht/smt

    these linux guests run very slow, perhaps the cause may be:
    > "ata_sff" stays forever at 99%

    do you have some experience about that?


    thanks
    bye, NicCo

    ReplyDelete
  15. ck2 October 2014 at 08:29

    Very nice :)

    ReplyDelete
  16. Anonymous7 October 2014 at 11:06

    @Alfred Chen:
    Have you already begun to re-propagate your pending patches of your pure BFS branch to Con? Before he'd release his 3.17 version... BTW, they're running well since published.

    Best regards to all of you,
    Manuel Krause

    ReplyDelete
    Replies
    1. Alfred Chen8 October 2014 at 19:21

      Only patch I am proposing for 3.17 BFS is still the 3% low workload performance boost code change. Other patches are minor code changes which very hard to measure.

      https://bitbucket.org/alfredchen/linux-gc/commits/ad9dd03db1002717f155c859ee613641620d3ba0?at=linux-3.16.y-gc

      Delete
    2. Anonymous15 October 2014 at 11:33

      Thank you for publishing your BFS-port to 3.17 below! I really hope, CK would pick up --at least-- some of your improvements.

      Currently I'm very glad with the 3.16 kernel until now: 3.16.5 + bfs/ck2 + gc-patches + bfq + my reworked revert patch for "drm-i915-Move-all-ring-resets-before-setting-the-HWS-page".
      In this fashion, this kernel gets the least annoying and best performing one since months.
      BTW, would someone of you NOT recommend to enable CONFIG_OPTIMIZE_INLINING ? I'm not completely sure whether to blame recent mainline changes or this config setting. Overall performance and responsiveness seems to be better with it enabled?! My gcc is a 4.8.3 20140627 [gcc-4_8-branch revision 212064].

      Many thanks to all participants in all involved projects!!!
      Manuel Krause

      P.S.: Not only the upgraded bfq is missing, I'd also wish to see a new TuxOnIce for 3.17. ^^

      Delete
  17. Alfred Chen13 October 2014 at 11:35

    Thanks for the head-up. I have started porting 0456 to 3.17 last weekend and there still some sync-up with mainline core.c left, but it already running on one of my machines. Hopefully my porting can be released in a day or two.

    BR Alfred.

    ReplyDelete
  18. Alfred Chen14 October 2014 at 17:02

    Hi, all,

    My porting of bfs 0456 to 3.17 is ready, if you want to try bfs on 3.17 before new bfs release by ck, you can apply the blow 2 patches upon 3.17.

    #1 Original BFS 0456 apply on 3.17
    https://bitbucket.org/alfredchen/linux-gc/commits/f9bf49826dda7530106b50e337b030514f9aadd6

    #2 bfs: 0456 porting to 3.17 and sync with mainline.
    https://bitbucket.org/alfredchen/linux-gc/commits/35d0568d06e79d42e6643aee5169579e7f1aabdb

    The others patches are not official accepted by ck, but as bug fix and improvement, apply them as you like.

    #3 revert KVM workaround due to proper cond_resched() fix, by pf
    https://bitbucket.org/alfredchen/linux-gc/commits/6531d1dca19b58b544fab697ac3b10ccd2ad5b40

    #4 bfs: Refine resched_best_idle() call logic in schedule(). ---- the 3% performance boot patch under low workload.
    https://bitbucket.org/alfredchen/linux-gc/commits/993df75b0292abd1c8e16bf5a43d32eebf7f0231

    ...
    other bfs related patches on my linux-3.17.y-gc branch are refactory/improvement that normally hard to be noticeable.

    https://bitbucket.org/alfredchen/linux-gc/commits/branch/linux-3.17.y-gc

    PS: my -gc branch is not yet completed, still waiting for new release of BFQ

    BR Alfred

    ReplyDelete
    Replies
    1. kernelOfTruth21 October 2014 at 05:21

      Thanks a lot Alfred

      Testing right now =)

      Wasn't aware that you're a Gentoo-user, too


      BFQ v7r6 is out, btw:

      https://groups.google.com/forum/#!topic/bfq-iosched/EufEsuDaEuU

      Delete
  19. Anonymous27 October 2014 at 21:39

    I have prepared kernel 3.17.1 with your four suggested patches, installed, now I'm running this kernel since few hours, and is working fine...

    now I'm searching for USKM and TOI for kernel 3.17: there are unofficial patches of these two?


    Thank you!
    NicCo

    ReplyDelete
  20. Anonymous8 November 2014 at 06:35

    Seems like all Australians are on holiday? ;-) O.k. they should enjoy, of course!

    I can encourage you to test Alfred Chen's latest -gc patches for the 3.17 kernel. Unfortunately, these are up to 18 (!) separate patches to be applied, including the last BFS with enhancements. I hope, he would provide a all-in-one complete patch sometimes. But so he let's us choose our dose of poison. They apply well on 3.17.2 and do also work well. Two days of everyday's desktop work based testing.
    http://cchalpha.blogspot.de/2014/11/whats-new-for-317-gc-patch-set.html

    I really miss a new revision of TuxOnIce. :-(

    Best regards,

    Manuel Krause

    ReplyDelete
    Replies
    1. Anonymous8 November 2014 at 19:52

      As you may notice, I've released 3.17-pf1 with TuxOnIce as well.

      --post-factum

      Delete
    2. Anonymous10 November 2014 at 08:37

      @post-factum
      Thanks for the reminder! And of course for your work, too.
      With the help of your shortlog for 3.17-pf1 and your and the kernel git repository, I was able to patch my 3.17.2 with the "old" TuxOnIce on here yesterday. (This enables me to not use your monolithic -pf1 patch, but to patch several addons separately, e.g. more of Alfred's improvements than you've included and to also omit UKSM what has shown to be prone for errors in my past experience.)
      It's working fine and seems to be stable.

      Best regards, Manuel Krause

      Delete
  21. Alfred Chen8 November 2014 at 22:27

    @Manuel
    Thanks for testing. The reason I go for separated patches is it's easier to bisect and find out which patch cause the problem if something wrong happened.

    ReplyDelete
    Replies
    1. Anonymous10 November 2014 at 08:50

      This way is o.k. like it is.
      BTW, your 3.17-gc patches do still work well on here. Now with post-factum's style TOI makeup. ;-)

      Manuel

      Delete
  22. Anonymous9 November 2014 at 02:56

    no more ck-patches

    ReplyDelete
    Replies
    1. ck9 November 2014 at 08:09

      Nah just too busy, lack of time and enthusiasm. I will get around to it eventually.

      Delete
  23. Anonymous27 February 2015 at 17:10

    Thanks so much for your work on this.
    Really appreciated.

    ReplyDelete

Subscribe to: Post Comments (Atom)