Bug 1275672 - Change symbols compression format to MSZIP; r?ted draft
authorGregory Szorc <gps@mozilla.com>
Fri, 27 May 2016 16:03:58 -0700
changeset 372311 cf75e12716e0b1042bb8017e14c99253072f18a6
parent 371902 4d63dde701b47b8661ab7990f197b6b60e543839
child 522156 2213003e9680d9c10f2888f05d2f8f732dc345a5
push id19500
push userbmo:gps@mozilla.com
push dateFri, 27 May 2016 23:31:51 +0000
reviewersted
bugs1275672
milestone49.0a1
Bug 1275672 - Change symbols compression format to MSZIP; r?ted makecab.exe has 3 options for compression: disable, MSZIP, and LZX. Here is a breakdown of the 3 levels of compression for an opt 32-bit build on my i7-6700K: directory size full.zip xul.pd_ `buildsymbols` None 1,360 MB 227 MB 146 MB 49s MSZIP 520 MB 221 MB 142 MB 113s LZX 436 MB 169 MB 102 MB 248s (The original size of xul.pdb is ~500 MB.) This commit switches us to MSZIP as the compression format. This makes `builsymbols` >2x faster while only increasing the full zip archive size by ~31%. This feels like an appropriate trade-off. The memory related flag has been removed because it only applies to LZX compression. It's worth noting that using `zip` to compress xul.pdb and xul.sym: Level Zip Size xul.pdb Compressed Time 9 160.6 MB 139.8 MB 76s 7 161.4 MB 140.5 MB 30s 5 164.7 MB 143.2 MB 16s 4 170.0 MB 147.3 MB 12s 3 176.4 MB 151.6 MB 11s So "MSZIP" compression appears to be using level 9. If we could swap in our own cab generator that uses a zlib compression level less than 9, we'll make symbol generation significantly faster without sacrificing too much size. I'm inclined to punt that to a follow-up bug. MozReview-Commit-ID: GbbClkn9PLN
toolkit/crashreporter/tools/symbolstore.py
--- a/toolkit/crashreporter/tools/symbolstore.py
+++ b/toolkit/crashreporter/tools/symbolstore.py
@@ -780,18 +780,17 @@ class Dumper_Win32(Dumper):
         self.fixedFilenameCaseCache[file] = result
         return result
 
     def CopyDebug(self, file, debug_file, guid, code_file, code_id):
         def compress(path):
             compressed_file = path[:-1] + '_'
             # ignore makecab's output
             success = subprocess.call(["makecab.exe", "/D",
-                                       "CompressionType=LZX", "/D",
-                                       "CompressionMemory=21",
+                                       "CompressionType=MSZIP", "/D",
                                        path, compressed_file],
                                       stdout=open(os.devnull, 'w'),
                                       stderr=subprocess.STDOUT)
             if success == 0 and os.path.exists(compressed_file):
                 os.unlink(path)
                 return True
             return False