Here are the results of tests performed in August 2002 to compare lossless compression of True Color (24-bit) images by all known programs developed for such purpose, including BMF, JPEG-LS and PNG software, 7-Zip, Pkzip, and archivers with special algorithms for 24-bit images. See Archive Comparison Test by Jeff Gilchrist for more info: http://compression.ca If anybody wants to start or continue such tests, or can suggest some other sets of images, or other compression programs, (not sources or algorithm descriptions, executable programs only) or knows we have missed something important, (some new fantastic technology, an algorithm or even a program capable of lossless compression of up to 1000:1 and lossy compression of up to 10000:1, we hear about it from time to time since PC-XTs) please let us know immediately: artest@inbox.ru Thank you! [[1]] COMPRESSION QUALITY ========================= (see also [[2]] Speed [[3]] Details [[4]] Comments) The last 13th line shows results for the sum of all 12 sets (627 images, 2 Gb) Original BMF RKIM UHIC NK ERI ArHanGeL UHARC ACE LSP 312.42% 113.54 115.37 112.90 124.36 100% 125.45 120.91 129.46 131.09 289.12% 100% 124.50 103.67 132.34 114.40 114.61 112.13 121.23 136.11 245.22% 100% 101.39 107.08 109.56 119.38 112.92 118.80 114.34 111.40 238.64% 100% 112.66 103.30 118.11 115.14 110.53 116.50 114.13 122.74 472.81% 102.89 100% 111.46 114.09 136.49 113.07 124.03 120.74 235.68 234.76% 100% 101.90 104.00 106.86 111.69 109.64 115.99 112.66 112.06 300.89% 100% 104.16 103.36 108.19 111.54 111.11 113.97 117.33 123.56 342.65% 100% 102.61 104.87 110.30 116.76 111.61 120.43 118.36 116.50 227.58% 100% 104.53 103.07 108.85 110.58 108.92 114.43 111.00 110.36 547.51% 100% 104.26 103.35 122.41 119.24 107.77 126.17 156.04 132.31 298.76% 108.25 109.28 108.51 118.03 100% 121.55 115.69 124.64 126.27 336.86% 109.13 108.43 108.95 118.18 100% 118.07 109.60 127.27 127.96 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 294.02% 100% 102.58 103.42 109.04 110.21 110.54 114.83 116.54 117.36 SBC DC RK LOCO LSP BTPC PNGCrush RAR Display Pkzip 135.18 136.17 137.24 169.98 173.17 156.67 170.75 138.18 174.71 190.14 132.44 127.16 127.77 156.42 160.20 157.26 151.37 130.67 164.86 208.56 120.62 123.53 125.13 119.20 122.55 128.46 129.46 122.23 139.54 195.64 125.49 124.69 124.77 127.46 131.33 133.62 134.30 124.18 144.00 184.72 123.48 137.85 134.12 176.37 161.59 153.29 145.17 195.16 157.13 187.70 117.66 118.60 120.00 115.91 119.55 121.67 119.56 127.05 128.84 173.10 118.23 122.63 122.20 120.16 123.42 126.97 129.43 137.46 136.48 169.65 119.24 126.75 127.19 127.12 129.49 132.28 134.58 130.89 143.27 201.55 116.47 120.30 120.62 118.46 120.67 122.54 122.16 117.20 133.11 181.12 118.41 131.16 132.39 151.18 150.78 152.21 141.97 130.70 148.75 193.33 128.38 130.83 130.82 151.91 155.97 143.55 156.67 133.92 161.61 179.17 120.00 127.74 127.52 143.54 148.09 139.05 141.79 152.45 151.10 188.14 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 117.79 122.11 122.59 124.70 127.79 128.21 129.61 131.02 137.89 180.73 [[2]] Speed =========== A set of 24 Kodak images from ftp://ftp.cipr.rpi.edu/stills/kodak/color/ was used for this test, and a 970MHz PC with 256Mb RAM and Windows98. Programs, Compression/ Overall Average Users' Compressed options Extraction, Score Score Size seconds seconds, % seconds, % bytes , % no compression 0 0 7865 304 7865 308 28312848 312 NK 139 28 3398 131 3272 128 11270106 124 locoe -c1 13 13 4305 167 4292 168 15404818 170 locoe -c2 15 14 4320 167 4305 169 15446962 170 lsp.bat e 13 13 3326 129 3313 130 11880422 131 lsp.bat e -f 11 11 3411 132 3401 133 12203421 135 ace32 a -d4096 43 10 3305 128 3263 128 11709297 129 ace32 a -d4096 -m1 42 10 3306 128 3264 128 11715709 129 ace32 a -d4096 -m5 50 10 3318 128 3269 128 11730213 129 arh a -mm -1 -2 69 38 3265 126 3197 125 11368943 125 bmf -f 22 8 3087 119 3066 120 11007688 121 bmf -f -q9 104 7 3161 122 3058 120 10979484 121 bmf -f -s 90 78 3026 117 2937 115 10289304 114 bmf -f -s -q9 240 78 3176 123 2938 115 10289304 114 compcl c -b15 -s 63 69 3868 150 3806 149 13451942 148 dc e -b16300 -mb5 58 18 3503 135 3446 135 12339333 136 display -bpng -i --same 42 5 4445 172 4404 172 15833142 175 eri a -3h13 32 37 2585 100 2554 100 9062479 100 eri a -3f13 32 37 2601 101 2570 101 9117643 101 eri a -3e13 32 37 2625 102 2594 102 9203321 102 eri a -3e13 -h90 23 29 2642 102 2619 103 9323447 103 gcac a 47 21 3553 137 3507 137 12547410 138 gcac s 48 21 3606 139 3558 139 12729808 140 imp98 a -mm -2 22 9 3923 152 3901 153 14008321 155 imp98 a -2 -s4 22 9 3926 152 3904 153 14019704 155 pkzip -es 4 1 5750 222 5746 225 20681803 228 pkzip -a 7 1 4837 187 4830 189 17381963 192 pkzip -exx 9 1 4839 187 4830 189 17383851 192 pkzipc -add 7 2 4853 188 4846 190 17436386 192 pkzipc -add -max 8 2 4845 187 4837 189 17405433 192 pkzipc -add -defl 15 2 4816 186 4801 188 17274828 191 pkzipc -add -max -defl 20 2 4809 186 4789 187 17231333 190 pngc-mmx -q 128 - 4427 171 4300 168 15474254 171 pngc-mmx -q -loco 127 - 3733 144 3607 141 12982336 143 pngc-mmx -q -brute 2351 - 6609 256 4282 168 15328841 169 qlfc a 28 21 3764 146 3736 146 13373069 148 rk -mf1 37 26 3924 152 3888 152 13901688 153 rk -mx1 476 478 4429 171 3958 155 12510696 138 rk -mx2 669 674 4769 184 4107 161 12336268 136 rk -mx3 866 868 5116 198 4259 167 12176044 134 rkim c 58 64 3145 122 3087 121 10883083 120 rkim cx 144 76 3123 121 2981 117 10454964 115 sbc c -m2 -b63 63 33 3500 135 3438 135 12254334 135 sbc c -m3 -b63 75 36 3514 136 3440 135 12251188 135 sbc c -m3 -b63 -ad 75 37 3507 136 3433 134 12224642 135 sbc c -m3 -b63 -os 79 36 3510 136 3432 134 12223134 135 zzip a 35 27 3816 148 3781 148 13515967 149 zzip a -mm -mx -26m 41 28 3817 148 3777 148 13495871 149 zzip a -mm -26m 41 28 3817 148 3777 148 13495871 149 BTPC 36 20 4001 154 3968 155 14197743 156 EPM c009 570 560 4736 183 4222 165 12978427 143 EPM c012 572 562 4740 183 4225 165 12980575 143 EPM c015 573 562 4741 183 4225 165 12980015 143 EPM c018 573 563 4742 183 4226 165 12980002 143 slim a 2113 2193 8963 346 7060 276 16762214 184 slim a -w22 2420 2512 9588 370 7410 290 16761898 184 slim a -i3 2126 2197 8979 347 7066 276 16762214 184 slim a -i3 -w22 2420 2507 9583 370 7405 289 16761898 184 UHIC e 51 53 2986 115 2939 115 10371444 114 UHIC m 106 110 3059 118 2963 116 10231963 112 7za a -tzip 28 2 4584 177 4559 178 16394253 180 7za a -mx -tzip 50 2 4601 177 4556 178 16376722 180 7za a -t7z 74 5 4091 158 4023 157 14439153 159 7za a -mx -t7z 116 5 4085 158 3980 155 14270422 157 abc13 -c 24 21 3828 148 3806 149 13615771 150 abc24 -c 46 56 3795 146 3753 146 13289311 146 bee a 281 267 4165 161 3911 153 13016700 143 bee a -m3 260 267 4144 160 3909 153 13016700 143 bee a -d6 281 267 4165 161 3911 153 13016700 143 bee a -m3 -d6 282 267 4165 161 3911 153 13016700 143 rar a -m1 39 3 4718 182 4682 183 16831106 185 rar a -m3 30 6 3738 144 3711 145 13327192 147 rar a -m5 31 5 3735 144 3706 145 13312529 146 rar a -m1 -mcc+ 25 5 3506 135 3484 136 12516038 138 rar a -m3 -mcc+ 25 5 3507 135 3484 136 12516038 138 rar a -m5 -mcc+ 26 6 3510 135 3486 136 12521337 138 rar a -m1 -mcc+ -s 27 6 3507 135 3482 136 12504418 137 rar a -m3 -mcc+ -s 27 6 3507 135 3482 136 12504424 137 rar a -m5 -mcc+ -s 29 7 3510 135 3484 136 12505518 137 rar32 a -m5 -mcc+ -s 36 7 3517 136 3484 136 12505518 137 uharc a -m1 -md32768 172 25 3572 138 3416 133 12146207 134 uharc a -mz -md32768 34 37 3958 153 3927 153 13991800 154 uharc a -mx -md32768 307 285 3662 141 3385 132 11049009 121 Losslessly compressed with ERI32 to 8,891,702 bytes, this set can be found at http://artest.virtualave.net/ki/ and http://geocities.com/rao-inc/kodak/ Overall score is calculated by adding compression time, extraction time, and time it would take to transfer the compressed file over a 28,800bps network: (compressed_size)/3600 Average Users' score is calculated by adding (compress_time/10)+ extract_time + time it would take to transfer the compressed file over a 28,800bps network. Compression time is divided by 10 here, because more than 90% of people would never compress anything during their life (with compression programs), but they use compressed data almost _every_ time they use computers and/or Internet. That's why compression time is not so actual for them. [[3]] Details ============= are no longer put to this main text (984 lines reporting 25600 results on 627 files in 12 sets), but can be found in FULL version with GRAPHICS.DAT and all *.BAT at http://compression.ru/artest/artest25.zip or https://artest1.tripod.com/artest25.zip 1,2,3,4,9th image sets were taken from ftp://ftp.cipr.rpi.edu/stills/ and /sequences/ , converted from Sun Raster format to Windows Bitmap, (this conversion changes image headers only, 32 bytes are replaced by 54 bytes, but image data is not modified) This site www.cipr.rpi.edu is described in comp.compression FAQ, in 29 lines of "Subject: [55] Where can I find Lenna and other images?" Eight images in the 5th set are from the famous Waterloo Color Set, ( losslessly compressed to 2,801,690 bytes, it can be found at http://geocities.com/eri32 ) 6,7,8,12th image sets are from ftp://download.intel.com/pressroom/images/ They were converted from TIFF format to Windows Bitmap. 10th and 11th sets were downloaded from ftp://ftp.kodak.com/www/images/ Some images were slightly renamed. See GRAPHICS.DAT inside artest25.zip for more details. Some new images (from download.intel.com) were not added yet. [[4]] Comments ============== Links to download programs: ~~~~~~~~~~~~~~~~~~~~~~~~~~~ 7-Zip 2.30b32 :W http://www.7-zip.org/dl/7z230b32.exe WinRAR 3.20 :W http://www.rarlab.com/rar/wrar320.exe RAR32 3.20 :e http://www.rarlab.com/rar/rarx320.exe ACE 2.04 :W http://winace.host.sk/ace204.exe BMF 1.1 :e http://compression.ru/ds/bmf_1_10.rar ERI 5.1fre :e http://compression.ru/artest/eri51fre.zip UHArc 0.4b :eW ftp://ftp.elf.stuba.sk/pub/pc/pack/uharc04.zip ZZip 0.36c :W http://debin.org/zzip/files/zzip-win32.zip ArHanGeL 1.40 :a http://geocities.com/SiliconValley/Lab/6606/arh140.zip DC 0.98b :W ftp://ftp.elf.stuba.sk/pub/pc/pack/dc124.zip Imp 1.1 :e http://www.technelysium.com.au/imp110d.zip Imp-win 1.12 :W http://www.technelysium.com.au/imp112.exe Pkzip 2.50 :a ftp://ftp.simtel.net/pub/simtelnet/msdos/arcers/pk250dos.exe PkzipC 4.00 :W ftp://ftp.pkware.com/pkzc400s.exe RK-dos 1.04.1 :e http://rksoft.virtualave.net/downloads/rk104a1d.exe RK 1.04.1 :W http://rksoft.virtualave.net/downloads/rk104a1w.exe RKim 1.06 :W http://rksoft.virtualave.net/downloads/rkim106.zip UHIC 2.0 :e ftp://ftp.elf.stuba.sk/pub/pc/pack/uhic20.zip NK 2.0beta :W ftp://tu.tusur.ru/pub/ImageTools/NK2beta.ZIP PngCrush 1.5.10:e http://download.sourceforge.net/pmt/pngcrush-1.5.10-DOS.zip JPEG-LS 1.00 :W http://www.hpl.hp.com/loco/jlsrefV100.zip SPIHT Nov'96 :W http://www.cipr.rpi.edu/research/SPIHT/EW_Code/lsp.exe BTPC 5 :W http://www.intuac.com/userport/john/btpc5/btpc5.zip Display 1.90b5:e http://www.freenet.edmonton.ab.ca/~crnelson/disp.exe SEA 1.3 :e ftp://ftp.elf.stuba.sk/pub/pc/graph/sea13.zip Alchemy 1.10 :a ftp://ftp.elf.stuba.sk/pub/pc/graph/alch110.zip PPMN_km b4 :W http://compression.ru/ms/ppmn_km.rar EPM 7 :W http://compression.ru/so/epm_7.zip SBC_d 0.969br1 :e http://personal.inet.fi/musiikki/sjm/sbc0969b_dos.zip SBC 0.969br1 :W http://personal.inet.fi/musiikki/sjm/sbc0969b_win32.zip Slim b13 :W http://www.slim-fb.by.ru/files/slim0013.zip Compressia 1.0b :W http://www.compressia.com/compressia.exe :a - any DOS - DOS programs, will run under pure DOS or in a DOS box :e - extender - DOS programs using DOS extenders like DOS/4GW or CWSDPMI :W - windows - Windows95/98/NT/etc programs If direct link doesn't work-most probably newer version of the program appeared at the same site: visit web page, or read the whole directory from ftp server (i.e. try the same URL, but without filename). Homepages: ~~~~~~~~~~ 7-Zip : http://www.7-zip.org ACE,WinACE : http://www.winace.com Arhangel : http://geocities.com/SiliconValley/Lab/6606 Eri32 : http://compression.ru/artest mirror : http://artest1.tripod.com RK, RKim : http://rksoft.virtualave.net Imp,WinImp : http://www.technelysium.com.au/winimp.html PkZip : http://www.pkware.com RAR,WinRAR : http://www.rarsoft.com SBC : http://sbcarchiver.netfirms.com ZZip : http://debin.org/zzip/ JPEG-LS : http://www.hpl.hp.com/loco PngCrush : http://pmt.sourceforge.net/pngcrush SPIHT : http://www.cipr.rpi.edu/research/SPIHT BTPC : http://www.intuac.com/userport/john/btpc5/ Display : http://fn2.freenet.edmonton.ab.ca/~crnelson/display.html Alchemy : http://www.handmadesw.com/Products/Image_Alchemy.htm SEA : http://www.photodex.com/products/dos BMF : http://compression.ru/ds/ What's new: ~~~~~~~~~~~ 9 new programs were tested: 7-zip 2.30b32 RAR 3.20 ABC 1.3 ABC 2.4 UHArc 0.5np2 UHIC 2.0 BTPC 5 EPM 7 Slim b13 Latest beta version of DC and UHArc were available from authors by e-mail: DC: EdgarBinder@t-online.de UHArc: Uwe.Herklotz@gmx.de Results of IMP, 7-zip, ZZip, ABC, EPM, Slim and PPMN are in the full version only, see GRAPHICS.DAT. WARNINGS: ~~~~~~~~~ NK can't losslessly decompress some 8-bit images. BMF doesn't save all fields of bmp-headers. ESP, Rkive and many other programs are not tested any more, their results and links can be found in previous volumes of ARTest. The LATEST RELEASE, and all previous volumes can be found at http://compression.ru/artest/ Send your suggestions, comments to artest@inbox.ru With best kind regards, A.Ratushnyak Back to main ARTest page
The Art of Lossless Data Compression vol. 25g