Monday 16 January 2012

3.2-ck1

These are patches designed to improve system responsiveness and interactivity with specific emphasis on the desktop, but suitable to any commodity hardware workload.

This is an upgrade to BFS 416, trivial bugfixes and a resync from 3.1.0-ck1 . I've changed the versioning to not specify a 3 point release since it usually applies to successive 3 point releases for some time.

Apply to 3.2.x:
patch-3.2-ck1.bz2


Broken out tarball:
3.2-ck1-broken-out.tar.bz2

Discrete patches:
patches


BFS by itself:
bfs


Web:
kernel.kolivas.org


Code blog when I feel like it:
ck-hack.blogspot.com


Each discrete patch usually contains a brief description of what it does at
the top of the patch itself.

Full patchlist:

3.2-sched-bfs-416.patch
mm-minimal_swappiness.patch
mm-enable_swaptoken_only_when_swap_full.patch
mm-drop_swap_cache_aggressively.patch
mm-kswapd_inherit_prio-1.patch
mm-background_scan.patch
mm-idleprio_prio-1.patch
mm-lru_cache_add_lru_tail-2.patch
mm-decrease_default_dirty_ratio-1.patch
kconfig-expose_vmsplit_option.patch
hz-default_1000.patch
hz-no_default_250.patch
hz-raise_max.patch
preempt-desktop-tune.patch
ck1-version.patch

Enjoy!
お楽しみください

--ck

P.S. What this really means is I finished playing zelda.

56 comments:

  1. Thanks, and please stop playing videogames, I need timely bfs updates :P j/k

    ReplyDelete
  2. working fine here on three machines, with added BFQ v3r2. :)

    ReplyDelete
  3. One little problem BFQ not register in kernel:

    dmesg:
    [0.000000] Linux version 3.2.1 (micron@*****) (gcc version 4.6.2 (GCC) ) #1
    SMP Mon Jan 16 20:54:16 EET 2012
    [6.338228] io scheduler noop registered
    [6.338230] io scheduler deadline registered
    [6.338291] io scheduler cfq registered

    Missing line with BFQ registered (default)

    In kernel config:


    #
    # IO Schedulers
    #
    CONFIG_IOSCHED_NOOP=y
    CONFIG_IOSCHED_DEADLINE=y
    CONFIG_IOSCHED_CFQ=y
    CONFIG_IOSCHED_BFQ=y
    # CONFIG_DEFAULT_DEADLINE is not set
    # CONFIG_DEFAULT_CFQ is not set
    CONFIG_DEFAULT_BFQ=y
    # CONFIG_DEFAULT_NOOP is not set
    CONFIG_DEFAULT_IOSCHED="bfq"

    And thanks CK for your work.

    Best regars
    Martin

    gl hf

    ReplyDelete
  4. BFQ has nothing to do with CK.

    ReplyDelete
  5. @ Micron: you should report bfq related problems here: http://groups.google.com/group/bfq-iosched

    ReplyDelete
  6. I do find the following in my dmesg:

    io scheduler noop registered
    io scheduler bfq registered (default)

    ReplyDelete
  7. i use a last patch 3.2.0-v3r2 and to list in dmesg

    oki i write to bfq team in google and wait for info..

    m.

    ReplyDelete
  8. How`s the latency measurements?

    ReplyDelete
  9. BFS runs very well, this is an output of my own script on a Gentoo running linx-3.2.1-ck1-92queuedpatches

    #+ .# #- +- .# 58.5 0:33 SLl chrome @15s@ 15.6 1:17 Sl firefox
    .. -. -# -= -. 58.9 0:37 SLl chrome @20s@ 15.5 1:17 Sl firefox
    #+ #. ++ -= .# 59.0 0:40 SLl chrome @25s@ 15.3 1:17 Sl firefox
    .. ## ## .. .. 57.3 0:41 SLl chrome @30s@ 15.2 1:17 Sl firefox
    .. == ## .. .. 55.4 0:43 SLl chrome @35s@ 15.1 1:17 Sl firefox
    .. .. .. .. .. 53.9 0:44 RLl chrome @40s@ 14.9 1:17 Sl firefox
    .. .. .. .. +# 52.5 0:46 SLl chrome @45s@ 14.8 1:17 Sl firefox
    .. =- .. .. .. 51.0 0:47 SLl chrome @50s@ 14.7 1:17 Sl firefox
    .. .. .. .. .. 49.8 0:48 SLl chrome @55s@ 14.5 1:17 Sl firefox
    .. .. .. ## .. 48.7 0:50 SLl chrome @60s@ 14.4 1:18 Sl firefox

    Chromium showing an Aljazeera live newsTV. Left the points and double crosses shows the intel core2-duo frequences. After five frequences mesurement there is an insertion of the two topmost top output:

    They contiously go down. From nearly 60 percent to 38. Also firefox.
    I don't know if these two self adjusting applications are programed to do it. Or if your BFS scheduler adjusts throughput.

    ReplyDelete
  10. CK please write my to mail to private contact to you...

    m.

    ReplyDelete
  11. Thank you for the 3.2 release. It's working very well for me. How was the game? How many hours did it take for you to complete it?

    By the way, I made a new blog about tips and tweaks for Linux. I'm going to write an article soon about how to patch and recompile the kernel with ck and bfq on dpkg/apt based systems. You can find my blog here: http://tux9656.blogspot.com/

    ReplyDelete
  12. It would be nice if you could provide a git tree. This might already have been asked before, but that would facilitate code exchange. Setting up a forked tree on GitHub is really trivial.

    Thanks!

    ReplyDelete
  13. It has been asked before, CK said he doesn't do git for bfs because of the way he ports his code. Can't blame him, the patch is not so complex as to require git.

    ReplyDelete
  14. Um, and here's BLD, a new... scheduler? https://lkml.org/lkml/2012/2/12/64

    ReplyDelete
    Replies
    1. That's a load distribution algorithm for the existing CFQ scheduler. Didn't you even read the first line in the post you linked to? :-P

      Delete
    2. But doesnt this algorithm sound like BFS. And the goal of having minimum calculations with quasi no heuristics?

      I would like some response from Con Kolivas about this!

      [Ralph Ulrich]

      Delete
    3. This indeed does reproduce one aspect of BFS and behaves more like a global queue, but from a completely different approach. This inherently should lead to lower latencies on SMP machines, but may also decrease throughput substantially if poorly done. It does not, however, address the issue of how to order tasks such that the interactive ones go first based on pure fairness, and leaves that aspect still the CFS run/sleep algorithm. So it makes for a halfway point between CFS and BFS. It's hard to know if it's the best of both worlds... or the worst without testing :)

      Delete
  15. I'm having an interesting time building for Ubuntu here. :)

    Just wanted to say that these instructions seem to work: http://blog.avirtualhome.com/2012/01/13/compile-linux-kernel-3-2-for-ubuntu-11-10/

    I just finished building 3.2.0 w/ latest Ubuntu patches, with the CK patchset and BFQ scheduling included. Everything patches cleanly and the process was smooth.

    Installed. Rebooting ...

    ReplyDelete
  16. Yay! So far so good. Thank you!

    ReplyDelete
  17. Hey all,

    Any thoughts on the effectiveness and/or impact on performance of the mm-decrease_default_dirty_ratio-1, mm-background_scan and mm-lru_cache_add_lru_tail-2 patches in -ck?

    In particular I've heard mixed opinions on the vm_dirty_ratio/dirty_background_ratio values for desktops, e.g. here http://groups.google.com/group/zen_kernel/browse_thread/thread/655c055d7175df2d/

    Thanks

    ReplyDelete
  18. Using 3.2.6:

    $ patch -p1 < ../patches/mm-idleprio_prio-1.patch
    patching file include/linux/sched.h
    Hunk #1 FAILED at 39.
    1 out of 1 hunk FAILED -- saving rejects to file include/linux/sched.h.rej
    patching file mm/vmscan.c
    Hunk #1 FAILED at 2103.
    1 out of 1 hunk FAILED -- saving rejects to file mm/vmscan.c.rej

    ReplyDelete
  19. Used BFS 2.6 kernel for a while on Ubuntu 10.10 -> switched to Fedora since 11.04, but used stock kernel, until Cinnamon came out to replace Gnome Shell, which led me to go back to Ubuntu 11.10 -> came back and checked out BFS, used the whole afternoon to build the kernel, now everything's back to normal. The responsiveness improvements are just there. Totally worth it.

    ReplyDelete
  20. Now this is my desktop box with a Athlon X2 5000+, 2G DDR. Slow as hell with the stock Ubuntu kernel. ck1 on 3.2 really changed everything. EVERYTHING.

    Thank you Con. I used to doubt the effectiveness of BFS. Not any more.

    ReplyDelete
  21. @CK
    [Manuel Krause]
    Do you remember my testcase from some months ago? This user mode decrypt <-> shm <-> swap thing?!

    If I omit==revert the following patches from current ck-patch, everything works fine (almost no lags in parallel video playing in vlc etc.):

    09.mm-decrease_default_dirty_ratio-1.patch
    08.mm-lru_cache_add_lru_tail-2.patch
    07.mm-idleprio_prio-1.patch
    05.mm-kswapd_inherit_prio-1.patch.orig
    02.mm-minimal_swappiness.patch

    Still adding good old bfs-313.autoiso-xorg.patch to my Kernel. BTW-question...: Is it needed anymore nowadays?!

    Thanks for your work!
    [Manuel Krause]

    ReplyDelete
  22. Currently I'm running 3.2.8 (openSUSE flavour).

    ReplyDelete
  23. - I get a gentoo-sources kernel and a ck-sources (gentoo-sources + ck patchset) kernel with all common options set identically.
    - I moved both from 2.6.38 version to 3.2.1 keeping all common options identical and setting all new options carefully.
    - My system is based on an Intel CoreII duo + 4Gb RAM.

    On my system running its typical load, I noticed a drop in performances from ck-sources 2.6.38 to ck-sources 3.2.1, drop that I cannot objectively quantify.

    I am currently trying to investigate the problem and started throughput tests with an Iozone benchmark :
    /usr/sbin/iozone -r 8 -s 50M -l 1 -u 10 -i 0 -i 1 -b filename

    I generally use the deadline scheduler and specific mount options but I tried here on an ext4 with default mount options and the noop scheduler.
    BTW, I always keep the rr_interval to its default value of 6.

    The results for write and rewrite tests show that :

    - Under a 2.6.38 + BFS, the throughput is approximately identical to a 2.6.38 + CFS for a number of processes less or equal to 4. With more than 4 processes, the throughput is divided by 4.
    Normal! well, at least, this is what I am used to.

    - Under a 3.2.1 + BFS, the throughput is a fourth of a 3.2.1 + CFS irrespective of the number of processes.

    I know that threaded IO has always been a weakness of BFS but my typical load never creates situations in which more than 4 processes are competing for writing on the same IO ressource, so I generally do not care.

    - Did you notice the same kind of performance drop ?
    - Did I do something wrong with my configuration ?
    - Is this benchmark meaningless ?
    - I read about the modifications made on ext4 in 3.2.1 but did not realize that they would have any impact here. Am I wrong ?

    ReplyDelete
  24. Yes that's intentional. I/O throughput happens at the expense of CPU interactivity so the -ck patchset is continually trying to reign in the I/O throughput to improve interactivity. If you care more about keeping your I/O throughput then use just the BFS patch and not the full -ck patchset.

    ReplyDelete
  25. Thank you Con for answering and, of course, for all the work you achieve.

    I understand the tradeoff and, at the extreme limit, I know that I can increase the rr_interval if necessary.

    What was puzzling me more is why this significant drop (-75%) of average throughput for a number of processes <= 4 from 2.6.38 to 3.2.1 ?

    ReplyDelete
  26. =When you said throughput, I'm pretty sure you mean I/O throughput since the CPU throughput is unchanged. I said it was intentional. It's in the -ck patches and due to changes to the VM and I/O scheduler between that older kernel and the current one that make -ck more effective at reducing throughput. The patches in -ck CONSCIOUSLY REDUCE I/O THROUGHPUT. So the patches are both designed to do that, and are successfully achieving that. So once again, if I/O throughput is important to you, you should not be using the -ck patchset. rr_interval has NOTHING to do with I/O throughput. Now if you're not talking about I/O throughput, then please explain further.

    ReplyDelete
  27. Yes ! I was talking about I/O throughput.
    Thanks for clarifying the drop from 2.6.38 to 3.2.1
    As for the impact of the rr_interval, I had noticed a slight difference in I/O throughput when increasing from 6 to 300, to the expense of the latency. But it definitely appears that I had misunderstood your FAQ. I apologize for this.

    I am not that much concerned by I/O throughput and much more in the possibility to globally achieve my work in about 85% of the time they need under a non ck-patched kernel.
    I see Sysbench thread test achieved in 60% of the time needed on a 3.2.1 non patched kernel.
    The ratio was of 77% with 2.6.38

    Thank you again for knowing that everyone does not get google's architectures and needs...

    ReplyDelete
  28. So... reading through your replies, i should only use my kernel with one lonely BFS-patch?
    For my ancient system, I/O througput is most probably the main bottleneck (except for AGP4x limits that are not on your bill)?

    Best regards
    [Manuel Krause]

    ReplyDelete
  29. I'm not the'Anonymous' you may have replied to, earlier, though.

    Best regards
    [Manuel Krause]

    ReplyDelete
  30. @Anonymous: From 2.6.38 to 3.2.1 there had been many major changes, call them improvements. ;-) But that's several releases away now!

    [ManuelKrause]

    ReplyDelete
  31. It's not that simple. Your disk I/O may well be the slowest component in your system, but if you do things to increase the I/O it almost always leads to other slowdowns, usually of the perceptible interactivity type. So you may speed up writing a big file but then your desktop may come to a standstill while you wait for it to write that file. That's why I'm forever doing things in -ck that make I/O slower rather than faster.

    ReplyDelete
    Replies
    1. Thank you very much for your explanations!
      But if I look at my overlapping /dev/shm that sometimes goes from RAM to swap (and maybe kicks other desktop-related things to disk as well) there should'nt be "double brakes" or triple or even...: for disk I/O, swap related I/O (mm-idleprio_prio-1.patch and mm-kswapd_inherit_prio-1.patch.orig as I read them), and some possible BUGs that make shm <-> RAM <-> swap transitions and vice versa so CPU/Bridge intense.
      I still cannot reflect, why the removal of my previously mentionned patches from complete CK makes such a difference in desktop's responsiveness.
      This is no critcism of your patchset related work, please read it as another spot to point your next optimization work to.
      I seem to be the last one running an uptodate kernel with CK on an over 12 years old machine, that seems to reveal kernel bottlenecks more easily. :-(

      Feel free to contact [Manuel Krause]

      Delete
  32. Well I did say I was having trouble validating whether those patches even did what they're supposed to do any more and I expect I will be abandoning them for 3.3-ck1 and just make it BFS only.

    ReplyDelete
    Replies
    1. O.k., yes, several times now. ;-) I've realized that message, finally.
      Are there any shareable insights on whether the autoiso-xorg.patch from bfs-313 days and or the schedtool approach are of benefit any more? (I'd always compiled it in. And I was used to 'enhance' a blocking application to finish earlier via schedtool -I `pidofproc app` from time to time.)

      [Manuel Krause]

      Delete
  33. I never recommended anyone use the autoiso patch. That's why it was never included in any -ck release. The potential for it blocking audio which is even more important than xorg is why I decided against it.

    ReplyDelete
    Replies
    1. We'll see, I'm just recompiling, meet you next week.

      Delete
    2. Good, that I didn't delete the old kernel compilations .oO

      One or both of the following patches add an overall plus to the personal(!) feeling of a quickly responding desktop on my vintage machine:
      04.mm-drop_swap_cache_aggressively.patch
      06.mm-background_scan.patch
      with BFS-416 and current bfq and kernel 3.2.9 (openSUSE).

      So, that's mainly the same setup as posted on 2012-03-03, with reverting from the ck-patch:
      09.mm-decrease_default_dirty_ratio-1.patch
      08.mm-lru_cache_add_lru_tail-2.patch
      07.mm-idleprio_prio-1.patch
      05.mm-kswapd_inherit_prio-1.patch.orig
      02.mm-minimal_swappiness.patch

      Too poor: This comment only reflects subjective recognitions with my previously posted testing and daily workflows.
      But maybe there's some usefulness for some people so that you don't abandon all mm related patches.

      [Manuel Krause]

      Delete
    3. Sorry, English is not my main language. When you say "add an overall plus" you mean they make your system a bit faster, right? Or slower?

      Delete
    4. Sorry, English isn't mine either.
      My system feels like responding better to desktop input and hardware throughput, e.g. playing videos additionally to other actions. So, that's nothing about "My system is faster." it's about "My system interacts faster with me."
      From my point of view that has more value than any benchmarked numbers. But it's only subjective, and not reproducible from one to another.
      [Manuel Krause]

      Delete
    5. @CK:
      There's no real advantage if I set Xorg to SCHED_NORMAL (unlike in the autoiso.patch) as it affects the input system as well. Or setting pulseaudio to SCHED_ISO or both. Pulseaudio is running @ NICE -11 (default on here). One of both always suffers somehow when CPU/Gfx/IO are under heavy load. I assume I'd need to experiment with the NICE levels to find out what helps more. But that's quite a boring test work ;-)
      [Manuel Krause]

      Delete
    6. Of course there is no "advantage" to setting Xorg to SCHED_NORMAL since that's... doing nothing. The issue is in YOUR desktop YOU are running pulseaudio and at nice -11. But that is not everyone, and I need a kernel that works best for everyone. If you find the patch helps you, go right ahead, but I will not include that patch by default.

      Delete
    7. Short Question: What priority is your pulseaudio running at?! To have better comparisons!
      [Manuel Krause]

      Delete
    8. If I set Xorg to SCHED_ISO (like in autoiso.patch) input is more responsive.
      [Manuel Krause]

      Delete
    9. Maybe I don't run pulseaudio and just use alsa? Maybe I don't change any priorities?

      Delete
    10. anonymous coward14 March 2012 at 19:30

      pulseaudio sucks :p

      Delete
    11. @ck:
      Maybe... I am really boring you with my comments/questions? Your replies make me feel like that. If I would not want to get the best experience from my existing system, what I supposed to be your goal, too, I've never had asked.

      @anonymous coward:
      Most recent releases of pulseaudio can manage multiple sources of audio like video + skype + system notifications that alsa was never capable of. Maybe... you like to give it a new try!
      [Manuel Krause]

      Delete
    12. anonymous coward16 March 2012 at 17:22

      pulseaudio is just bells and whistles, it's bloated, i don't feel the need for it thanks.

      Delete
  34. For the git lover. I set up a git repository to trace the linux-stable git tree and add some of my favorite patch sets. Of course, bfs/ck patch set is the major party of them. Current, bfs/ck, bfs and phc-intel are patch sets I am tracing in this reposiroty.

    Branches, there will be a branch to trace each kind of patch, for example, linux-3.2.y-bfs to trace the BFS patch.
    Tags, once a stable kernel released and the patch apply successfully and build out, a signed tag will be added, such like v3.2.9-ck1 and so on.
    Test, a kernel build test will be run to test the patch with kernel release before test. I just use the -gc one by myself, so I have no promises all of them runs ok.
    Merge strategy, when a new stable kernel version bumped, this repository will be sync-up with the upstream repo( linux-stable.git ). -bfs,-bfq and phc-intel branch will be rebased to the new stable tag, after build test, a tag with patch name sub-fix will be signed. -ck branch will be rebased with the new created -bfs tag, after build test then signed the -ck tag. Finally, -gc branch will be rebased with phc-intel, -ck and -bfq branch, build then signed the tag.
    To use, the git way, clone the repository, checkout the tag you want and build you kernel. Tarball way, visit the download tab, click on the download link of given tarball format with the tag you want.

    Repository address: https://bitbucket.org/alfredchen/linux-gc

    ReplyDelete
  35. I am using Ubuntu 12.04 and so downloaded the default kernel source from repo:

    apt-get source linux-image$(uname -r)

    The above, downloads the official kernel of Ubuntu 12.04 which is the 3.2.0-29.46 (this is build on 3.2.24 mainline of kernel.org).

    So I downloaded the "patch-3.2-ck1" and successfully applied it. Unfortunately after starting the compilation, a little later I get an error and it stops the compilation:

    CC arch/x86/kernel/pcspeaker.o
    CC arch/x86/kernel/check.o
    CC arch/x86/kernel/devicetree.o
    LD arch/x86/kernel/built-in.o
    AS arch/x86/kernel/head_32.o
    CC arch/x86/kernel/head32.o
    CC arch/x86/kernel/head.o
    CC arch/x86/kernel/init_task.o
    /home/user/Public/CompilingFolder/linux-stable/arch/x86/kernel/init_task.c:31:8: error: unknown field ‘deadline’ specified in initializer
    /home/user/Public/CompilingFolder/linux-stable/arch/x86/kernel/init_task.c:31:8: error: unknown field ‘run_list’ specified in initializer
    /home/user/Public/CompilingFolder/linux-stable/arch/x86/kernel/init_task.c:31:32: error: ‘struct task_struct’ has no member named ‘run_list’
    /home/user/Public/CompilingFolder/linux-stable/arch/x86/kernel/init_task.c:31:32: error: ‘struct task_struct’ has no member named ‘run_list’
    /home/user/Public/CompilingFolder/linux-stable/arch/x86/kernel/init_task.c:31:8: error: unknown field ‘time_slice’ specified in initializer
    make[4]: *** [arch/x86/kernel/init_task.o] Error 1
    make[3]: *** [arch/x86/kernel] Error 2
    make[2]: *** [arch/x86] Error 2
    make[1]: *** [sub-make] Error 2
    make[1]: Leaving directory `/home/user/Public/CompilingFolder/linux-stable'
    make: *** [/home/user/Public/CompilingFolder/linux-stable/debian/stamps/stamp-build-optimus] Error 2

    Any ideas how I can manage to fix it or why it is happening ?

    ReplyDelete
    Replies
    1. Mursel, I got the same balalayka. Did you find how to fix this issue?

      Delete
    2. Same problem here, any solutions?

      CC arch/x86/kernel/init_task.o
      /build/buildd/linux-3.2.0/arch/x86/kernel/init_task.c:31:8: error: unknown field 'deadline' specified in initializer
      /build/buildd/linux-3.2.0/arch/x86/kernel/init_task.c:31:8: error: unknown field 'run_list' specified in initializer
      /build/buildd/linux-3.2.0/arch/x86/kernel/init_task.c:31:32: error: 'struct task_struct' has no member named 'run_list'
      /build/buildd/linux-3.2.0/arch/x86/kernel/init_task.c:31:32: error: 'struct task_struct' has no member named 'run_list'
      /build/buildd/linux-3.2.0/arch/x86/kernel/init_task.c:31:8: error: unknown field 'time_slice' specified in initializer
      make[4]: *** [arch/x86/kernel/init_task.o] Error 1
      make[3]: *** [arch/x86/kernel] Error 2
      make[2]: *** [arch/x86] Error 2
      make[1]: *** [sub-make] Error 2

      Delete
  36. Does not work with 3.2.57 and .58 kernel. Yes with 3.2.2.

    ReplyDelete
  37. "Does not work with 3.2.57 and .58 kernel. Yes with 3.2.2."

    Unfortunately it's true...

    ReplyDelete