A one bug fix release.
Detach threads after creating them on the compression side. Not joining them meant that compressing massive files requiring hundreds of threads would eventually hit the resource limit of number of threads created even though the threads themselves would exit.
lrzip will no longer fail with a "resource temporarily unavailable" error when compressing files over 100GB that require hundreds of threads to complete.
Get it here: LRZIP
I created an rpm for PCLinuxOS. I don't know if or when it will be included, hopefully soon!ReplyDelete
Thanks! Quite a few distros are now including lrzip, which is nice. The 'file' command has been upgraded at last to recognise lrzip files, and RPM I believe even has support for lrzip compression now. It seems the comprehensive STDIO support was by far the most valuable feature to add to lrzip.ReplyDelete
I noticed one bug in rzip.c. lastpct and last_chunkpct are set to 0 in every iteration of the loop, so the status prints every time it is checked, instead of only when the percentage changing.
Noticed this while using it over ssh, and wondering why I was using so much bandwidth.
Aha well spotted. I wondered why it wasn't doing the incremental progress only as planned. Thanks!ReplyDelete