Why is bzip2 slow

Files that are compressed with pbzip2 are broken up into pieces and each individual piece is compressed. Files that are compressed with pbzip2 will also gain considerable speedup when decompressed using pbzip2. Files that were compressed using bzip2 will not see speedup " — osgx. Very interesting. I will definitely take a look. Thanks — Greg Sadetsky. Sign up or log in Sign up using Google. Sign up using Facebook. Sign up using Email and Password.

Post as a guest Name. Email Required, but never shown. The Overflow Blog. Does ES6 make JavaScript frameworks obsolete? Podcast Do polyglots have an edge when it comes to mastering programming Featured on Meta. Now live: A fully responsive profile. Related Hot Network Questions. Notify me of follow-up comments by email. Notify me of new posts by email. Email Address. RootUsers Guides, tutorials, reviews and news for System Administrators.

The Benchmarking Process The linux Below are the commands that were run for compression level 1: time bzip2 -1v linux The versions pf these tools were gzip 1. Results The raw data that the below graphs have been created from has been provided in tables below and can also be accessed in this spreadsheet. Compressed Size The table below indicates the size in bytes of the linux Linux , Technology Benchmark , Disk , Linux.

Leave a comment? Al Wegener November 26, at pm. Jarrod November 26, at pm. You did a great job clearly laying out what you found about the various compression programs. Jarrod December 29, at pm. Herb Hirsch April 28, at am. Jarrod April 28, at am. Rasel Khan June 6, at pm. JoBo June 28, at am. It is also worth noting that the memory usage for xz can be extreme. Jarrod June 28, at am. SurveyMan October 7, at pm. This was a rather interesting read and for sure it will be referenced by many for years to come.

Ali Mihandoost December 28, at pm. Andrew Nyago August 29, at am. Jakob May 18, at pm. Hiya, In your conclusion it may be a good idea to under which conditions is it is wise to change compressors.

Emil June 21, at am. As someone already said, this would be quite interesting for archivists. Wellington Torrejais da Silva February 9, at am. Nice comparison! The share buttons not working for me :- Thanks! Kalimuthu March 26, at pm. Markus Krainz August 27, at am. The Compression Ratio was very confusing. Jarrod August 28, at pm. In this test bzip2 is a tough adversary to lzmash in fast modes.

XMMS 1. The file was first gunzipped, resulting uncompressed size of bytes 5. For some reason, "bzip2 -6" took more time than even "bzip -9". The result didn't change when the test was repeated. The extreme mode of lzmash creates a few bytes bigger files; seems that using "lzmash -e" makes compression both slower and less efficient with smaller files. Speed tables are omitted because the smaller test file makes measuring the elapsed time with 'time' command too inaccurate.

For some reason, in compression "bzip2 -6" was a little faster than "bzip -5" but "bzip -6" still created smaller file. The memory requirements depend only on the used compression mode This small memory mode hasn't been tested.

When there's need for a very fast compression, gzip is the clear winner. It has also very small memory footprint, making it ideal for systems with limited memory.

Nowadays most source code is available as both gzip and bzip2 compressed tar archives. However the memory requirements increase with every option meaning that "lzmash -3", "lzmash -5" and "lzmash -6" are usually useful only if you or the recipient do not have enough memory for "lzmash -4" or "lzmash -7".

The extreme mode "lzmash -e" roughly doubles the compression time, but especially with small files can lead to even worse compression ratio than normal the mode.