Here are the results of tests performed in August 2002 to compare lossless compression of True Color (24-bit) images by all known programs developed for such purpose, including BMF, JPEG-LS and PNG software, 7-Zip, Pkzip, and archivers with special algorithms for 24-bit images. See Archive Comparison Test by Jeff Gilchrist for more info: http://compression.ca If anybody wants to start or continue such tests, or can suggest some other sets of images, or other compression programs, (not sources or algorithm descriptions, executable programs only) or knows we have missed something important, (some new fantastic technology, an algorithm or even a program capable of lossless compression of up to 1000:1 and lossy compression of up to 10000:1, we hear about it from time to time since PC-XTs) please let us know immediately: artest@inbox.ru Thank you! [[1]] COMPRESSION QUALITY ========================= (see also [[2]] Speed [[3]] Details [[4]] Comments) The last 13th line shows results for the sum of all 12 sets (627 images, 2 Gb) Original BMF RKIM UHIC NK ERI ArHanGeL UHARC ACE LSP 312.42% 113.54 115.37 114.15 124.36 100% 125.45 122.15 129.46 131.09 289.12% 100% 124.50 104.88 132.34 114.40 114.61 112.22 121.23 136.11 245.22% 100% 101.39 107.94 109.56 119.38 112.92 118.90 114.34 111.40 238.64% 100% 112.66 104.21 118.11 115.14 110.53 116.65 114.13 122.74 472.81% 102.89 100% 112.25 114.09 136.49 113.07 125.36 120.74 235.68 234.76% 100% 101.90 104.87 106.86 111.69 109.64 116.21 112.66 112.06 300.89% 100% 104.16 104.39 108.19 111.54 111.11 114.53 117.33 123.56 342.65% 100% 102.61 106.05 110.30 116.76 111.61 120.70 118.36 116.50 227.58% 100% 104.53 104.01 108.85 110.58 108.92 114.55 111.00 110.36 547.51% 100% 104.26 104.85 122.41 119.24 107.77 127.08 156.04 132.31 298.76% 108.25 109.28 109.79 118.03 100% 121.55 116.62 124.64 126.27 336.86% 109.13 108.43 110.28 118.18 100% 118.07 110.08 127.27 127.96 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 294.02% 100% 102.58 104.48 109.04 110.21 110.54 115.21 116.54 117.36 SBC DC RK LOCO ZZip PNGCrush RAR Compressia Display Pkzip 135.18 136.17 137.24 169.98 148.92 170.75 138.18 148.45 174.71 190.14 132.44 127.16 127.77 156.42 132.25 151.37 130.67 165.23 164.86 208.56 120.62 123.53 125.13 119.20 130.84 129.46 122.23 160.19 139.54 195.64 125.49 124.69 124.77 127.46 130.69 134.30 124.18 158.01 144.00 184.72 123.48 137.85 134.12 176.37 139.34 145.17 195.16 158.07 157.13 187.70 117.66 118.60 120.00 115.91 125.98 119.56 127.05 131.56 128.84 173.10 118.23 122.63 122.20 120.16 128.24 129.43 137.46 131.27 136.48 169.65 119.24 126.75 127.19 127.12 133.98 134.58 130.89 148.70 143.27 201.55 116.47 120.30 120.62 118.46 126.11 122.16 117.20 141.28 133.11 181.12 118.41 131.16 132.39 151.18 137.54 141.97 130.70 156.88 148.75 193.33 128.38 130.83 130.82 151.91 137.53 156.67 133.92 140.20 161.61 179.17 120.00 127.74 127.52 143.54 129.09 141.79 152.45 123.20 151.10 188.14 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 117.79 122.11 122.59 124.70 128.42 129.61 131.02 135.81 137.89 180.73 [[2]] Speed =========== A set of 24 Kodak images from ftp://ftp.cipr.rpi.edu/stills/kodak/color/ was used for this test, and a 970MHz PC with 256Mb RAM and Windows98. Programs, Compression/ Overall Average Users' Compressed options Extraction, Score Score Size seconds seconds, % seconds, % bytes , % no compression 0 0 7865 304 7865 308 28312848 312 NK 139 28 3398 131 3272 128 11270106 124 locoe -c1 13 13 4305 167 4292 168 15404818 170 locoe -c2 15 14 4320 167 4305 169 15446962 170 cbtpc 18 9 4724 183 4706 184 16908835 187 lsp.bat e 13 13 3326 129 3313 130 11880422 131 lsp.bat e -f 11 11 3411 132 3401 133 12203421 135 7za a -tzip 28 2 4584 177 4557 178 16394253 181 7za a -mx -tzip 51 2 4602 178 4552 178 16376722 181 7za a -t7z 75 5 4094 158 4020 157 14449249 159 7za a -mx -t7z 101 5 4094 158 3994 156 14353985 158 ace32 a -d4096 43 10 3305 128 3263 128 11709297 129 ace32 a -d4096 -m1 42 10 3306 128 3264 128 11715709 129 ace32 a -d4096 -m5 50 10 3318 128 3269 128 11730213 129 arh a -mm -1 -2 69 38 3265 126 3197 125 11368943 125 bmf -f 22 8 3087 119 3066 120 11007688 121 bmf -f -q9 104 7 3161 122 3058 120 10979484 121 bmf -f -s 90 78 3026 117 2937 115 10289304 114 bmf -f -s -q9 240 78 3176 123 2938 115 10289304 114 compcl c -b15 -s 63 69 3868 150 3806 149 13451942 148 dc e -b16300 -mb5 58 18 3503 135 3446 135 12339333 136 display -bpng -i --same 42 5 4445 172 4404 172 15833142 175 eri a -3h13 32 37 2585 100 2554 100 9062479 100 eri a -3f13 32 37 2601 101 2570 101 9117643 101 eri a -3e13 32 37 2625 102 2594 102 9203321 102 eri a -3e13 -h90 23 29 2642 102 2619 103 9323447 103 gcac a 47 21 3553 137 3507 137 12547410 138 gcac s 48 21 3606 139 3558 139 12729808 140 imp98 a -mm -2 22 9 3923 152 3901 153 14008321 155 imp98 a -2 -s4 22 9 3926 152 3904 153 14019704 155 pkzip -es 4 1 5750 222 5746 225 20681803 228 pkzip -a 7 1 4837 187 4830 189 17381963 192 pkzip -exx 9 1 4839 187 4830 189 17383851 192 pkzipc -add 7 2 4853 188 4846 190 17436386 192 pkzipc -add -max 8 2 4845 187 4837 189 17405433 192 pkzipc -add -defl 15 2 4816 186 4801 188 17274828 191 pkzipc -add -max -defl 20 2 4809 186 4789 187 17231333 190 pngc-mmx -q 128 - 4427 171 4300 168 15474254 171 pngc-mmx -q -loco 127 - 3733 144 3607 141 12982336 143 pngc-mmx -q -brute 2351 - 6609 256 4282 168 15328841 169 qlfc a 28 21 3764 146 3736 146 13373069 148 rar a -m1 -mcc+ -s 27 6 3507 136 3479 136 12504418 138 rar a -m3 -mcc+ -s 28 6 3507 136 3480 136 12504424 138 rar a -m4 -mcc+ -s 29 7 3509 136 3480 136 12503013 138 rar a -m5 -mcc+ -s 29 7 3510 136 3481 136 12505518 138 rk -mf1 37 26 3924 152 3888 152 13901688 153 rk -mx1 476 478 4429 171 3958 155 12510696 138 rk -mx2 669 674 4769 184 4107 161 12336268 136 rk -mx3 866 868 5116 198 4259 167 12176044 134 rkim c 58 64 3145 122 3087 121 10883083 120 rkim cx 144 76 3123 121 2981 117 10454964 115 sbc c -m2 -b63 63 33 3500 135 3438 135 12254334 135 sbc c -m3 -b63 75 36 3514 136 3440 135 12251188 135 sbc c -m3 -b63 -ad 75 37 3507 136 3433 134 12224642 135 sbc c -m3 -b63 -os 79 36 3510 136 3432 134 12223134 135 uharc a -m1 -md32768 173 26 3573 138 3401 133 12146238 134 uharc a -mz -md32768 35 38 3960 153 3926 154 13994087 154 uharc a -mx -md32768 268 246 3616 140 3350 131 11167998 123 uhic e 55 58 3024 117 2969 116 10479144 116 uhic m 107 110 3091 120 2985 117 10344923 114 zzip a 35 27 3816 148 3781 148 13515967 149 zzip a -mm -mx -26m 41 28 3817 148 3777 148 13495871 149 zzip a -mm -26m 41 28 3817 148 3777 148 13495871 149 Losslessly compressed with ERI32 to 8,891,702 bytes, this set can be found at http://artest.virtualave.net/ki/ and http://geocities.com/rao-inc/kodak/ Overall score is calculated by adding compression time, extraction time, and time it would take to transfer the compressed file over a 28,800bps network: (compressed_size)/3600 Average Users' score is calculated by adding (compress_time/10)+ extract_time + time it would take to transfer the compressed file over a 28,800bps network. Compression time is divided by 10 here, because more than 90% of people would never compress anything during their life (with compression programs), but they use compressed data almost _every_ time they use computers and/or Internet. That's why compression time is not so actual for them. [[3]] Details ============= are no longer put to this main text (984 lines reporting 25600 results on 627 files in 12 sets), but can be found in FULL version with GRAPHICS.DAT and all *.BAT at http://geocities.com/SiliconValley/Bay/1995/artest24.zip or https://artest1.tripod.com/artest24.zip 1,2,3,4,9th image sets were taken from ftp://ftp.cipr.rpi.edu/stills/ and /sequences/ , converted from Sun Raster format to Windows Bitmap, (this conversion changes image headers only, 32 bytes are replaced by 54 bytes, but image data is not modified) This site www.cipr.rpi.edu is described in comp.compression FAQ, in 29 lines of "Subject: [55] Where can I find Lenna and other images?" Eight images in the 5th set are from the famous Waterloo Color Set, ( losslessly compressed to 2,801,690 bytes, it can be found at http://geocities.com/eri32 ) 6,7,8,12th image sets are from ftp://download.intel.com/pressroom/images/ They were converted from TIFF format to Windows Bitmap. 10th and 11th sets were downloaded from ftp://ftp.kodak.com/www/images/ Some images were slightly renamed. See GRAPHICS.DAT inside artest24.zip for more details. Some new images (from download.intel.com) were not added yet. [[4]] Comments ============== Links to download programs: ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ACE 2.04 :W http://winace.host.sk/ace204.exe 593K BMF 1.1f :e http://geocities.com/eri32/bmf11f.zip 76K ERI 5.1fre :e http://geocities.com/eri32/eri51fre.zip 89K UHArc 0.4b :eW ftp://ftp.elf.stuba.sk/pub/pc/pack/uharc04.zip 220K ZZip 0.36c :W http://www.via.ecp.fr/~damien/downloads/zzip-win32.zip 35K ArHanGeL 1.40 :a http://geocities.com/SiliconValley/Lab/6606/arh140.zip 50K DC 0.98b :W ftp://ftp.elf.stuba.sk/pub/pc/pack/dc124.zip 55K Imp 1.1 :e http://www.technelysium.com.au/imp110d.zip 266K Imp-win 1.12 :W http://www.technelysium.com.au/imp112.exe 122K Pkzip 2.50 :a ftp://ftp.simtel.net/pub/simtelnet/msdos/arcers/pk250dos.exe 202K PkzipC 4.00 :W ftp://ftp.pkware.com/pkzc400s.exe 3470K RK-dos 1.04.1 :e http://rksoft.virtualave.net/downloads/rk104a1d.exe 461K RK 1.04.1 :W http://rksoft.virtualave.net/downloads/rk104a1w.exe 380K RKim 1.06 :W http://rksoft.virtualave.net/downloads/rkim106.zip 57K UHIC 1.0 :e ftp://ftp.elf.stuba.sk/pub/pc/pack/uhic10.zip 52K NK 2.0beta :W ftp://tu.tusur.ru/pub/ImageTools/NK2beta.ZIP 52K PngCrush 1.5.10:e http://download.sourceforge.net/pmt/pngcrush-1.5.10-DOS.zip JPEG-LS 1.00 :W http://www.hpl.hp.com/loco/jlsrefV100.zip 325K SPIHT Nov'96 :W http://www.cipr.rpi.edu/research/SPIHT/EW_Code/lsp.exe 72K BTPC 4.1 :W http://www.engr.mun.ca/~john/btpc4.1.zip 715K Display 1.90b5:e http://www.freenet.edmonton.ab.ca/~crnelson/disp.exe 811K SEA 1.3 :e ftp://ftp.elf.stuba.sk/pub/pc/graph/sea13.zip 705K Alchemy 1.10 :a ftp://ftp.elf.stuba.sk/pub/pc/graph/alch110.zip 1462K PPMN_km b4 :W http://compression.graphicon.ru/ms/ppmn_km.rar 7-Zip 2.30b23 :W http://belnet.dl.sourceforge.net/sevenzip/7z230b23.exe WinRAR 3.00 :W http://www.rarlab.com/rar/wrar300.exe RAR32 3.00 :e http://www.rarlab.com/rar/rarx300.exe SBC_d 0.969br1 :e http://personal.inet.fi/musiikki/sjm/sbc0969b_dos.zip SBC 0.969br1 :W http://personal.inet.fi/musiikki/sjm/sbc0969b_win32.zip Compressia 0.98b :W http://www.compressia.com/compcl.exe :a - any DOS - DOS programs, will run under pure DOS or in a DOS box :e - extender - DOS programs using DOS extenders like DOS/4GW or CWSDPMI :W - windows - Windows95/98/NT/etc programs If direct link doesn't work-most probably newer version of the program appeared at the same site: visit web page, or read the whole directory from ftp server (i.e. try the same URL, but without filename). Homepages: ~~~~~~~~~~ Arhangel : http://geocities.com/SiliconValley/Lab/6606 Eri32 : http://geocities.com/eri32 mirror : http://artest1.tripod.com RK, RKim : http://rksoft.virtualave.net Imp,WinImp : http://www.technelysium.com.au/winimp.html ACE,WinACE : http://www.winace.com PkZip : http://www.pkware.com RAR,WinRAR : http://www.rarsoft.com JPEG-LS : http://www.hpl.hp.com/loco PngCrush : http://pmt.sourceforge.net/pngcrush SPIHT : http://www.cipr.rpi.edu/research/SPIHT BTPC : http://www.engr.mun.ca/~john/btpc.html Display : http://fn2.freenet.edmonton.ab.ca/~crnelson/display.html Alchemy : http://www.handmadesw.com/Products/Image_Alchemy.htm SEA : http://www.photodex.com/products/dos SBC : http://sbcarchiver.netfirms.com ZZip : ? http://www.zzip.f2s.com Compressia : http://www.compressia.com/ Ufa,777, 7-Zip : http://www.7-zip.org What's new: ~~~~~~~~~~~ 6 new programs were tested: PngCrush 1.5.10 7-zip 2.30 b22 RAR 3.00 SBC 0.969b r1 PPMN_km b4 Compressia 0.98b 7-zip, PngCrush and BZip2 perform exactly as their previous versions tested in March or earlier. Latest beta version of DC was available from author by e-mail: DC: EdgarBinder@t-online.de Results of IMP, 7-zip, BTPC and PPMN are in the full version only, see GRAPHICS.DAT. WARNINGS: ~~~~~~~~~ NK can't losslessly decompress some 8-bit images. BMF doesn't save all fields of bmp-headers. ESP, Rkive and some other programs are not tested any more, their results and links can be found in previous versions: ESP - artest3...artest12 Rkive - 3...12 UFA - 3...12 777 - 13...14 BOA - 7...14 IMP - 13...22 SEA - 12...19 CompuShow 2000! - 3...11 The LATEST RELEASE, and all previous versions of these tests can be found at http://geocities.com/SiliconValley/Bay/1995/ and https://artest1.tripod.com/ Send your suggestions, comments to artest@inbox.ru With best kind regards, A.Ratushnyak Back to main ARTest page
The Art of Lossless Data Compression vol. 24g 
