Bug 1408111 - Update webrender documentation. r?mstange draft
authorKartikaya Gupta <kgupta@mozilla.com>
Thu, 12 Oct 2017 15:52:37 -0400
changeset 679510 3663f493e74e343244f680780d48979447a0fde7
parent 679509 98247fbf95c260786361e12ad924c4370885f686
child 735619 1fae4325393040d224935aa5c3ba1e4f88e43d18
push id84243
push userkgupta@mozilla.com
push dateThu, 12 Oct 2017 19:53:25 +0000
reviewersmstange
bugs1408111
milestone58.0a1
Bug 1408111 - Update webrender documentation. r?mstange MozReview-Commit-ID: 1AMHVWmNTA5
gfx/doc/README.webrender
--- a/gfx/doc/README.webrender
+++ b/gfx/doc/README.webrender
@@ -1,82 +1,178 @@
-Step 1: Install Rust if you don't have it already
-    If you are doing gecko builds already, you should already have Rust as it is a build requirement.
-    If not, you can install it using |mach bootstrap| (recommended) or from https://www.rust-lang.org/
-        Note: If installing manually, use the stable 64-bit release - on Windows make sure to use the MSVC ABI installer.
-    Ensure that rustc and cargo are in your $PATH (adding $HOME/.cargo/bin/ should be sufficient)
-
-Step 2: Set up mozconfig
-    Add the following line to your mozconfig:
-        ac_add_options --enable-webrender
-    The first time you do a build with this changes, you should also run |mach clobber|
-
-Step 3:
-    Build using |mach build|
-
+To build and run WebRender in Gecko:
+1. Install Rust if you don't have it already
+   If you are doing gecko builds already, you should already have Rust as it is a build requirement.
+   If not, you can install it using |mach bootstrap| (recommended) or from https://www.rust-lang.org/
+       Note: If installing manually, use the stable 64-bit release - on Windows make sure to use the MSVC ABI installer.
+   Ensure that rustc and cargo are in your $PATH (adding $HOME/.cargo/bin/ should be sufficient)
+2. Build using |mach build|.
+   You don't need anything special in your mozconfig for local builds; webrender will be built by default.
+   If you are building a non-nightly version (e.g. beta) you may need to add |ac_add_options --enable-webrender=build| to your mozconfig.
+3. Run with |MOZ_WEBRENDER=1| in your environment. e.g. |MOZ_WEBRENDER=1 ./mach run|.
+   Alternatively, you can set the gfx.webrender.enabled pref to true (browser restart required).
+   Note that on Linux, acceleration is disabled by default and it needs to be enabled for WebRender to work.
+   On Linux you can enable acceleration by putting |MOZ_ACCELERATED=1| in your environment, or setting layers.acceleration.force-enabled to true in about:config.
+4. Verify WebRender is enabled. You can do this by going to about:support and checking the "Compositing" line in the Graphics section. It should say "WebRender".
+   There should also be a WebRender section under "Decision Log" in about:support, which will provide some more detail on what caused it to be enabled/disabled.
 
 When making changes:
     - Make the changes you want.
     - Run |mach build| or |mach build binaries| as desired.
 
-
 For a debug webrender build:
     Use a debug mozconfig (ac_add_options --enable-debug)
     You can also use an opt build but make webrender less optimized by putting opt-level=0 in the [profile.release] section of your toolkit/library/rust/Cargo.toml file
     See also https://groups.google.com/forum/#!topic/mozilla.dev.servo/MbeMcqqO1fs
 
+--------------------------------------------------------------------------------
 
-What if you have to pull in an update to webrender itself?
+What if you have to pull in an update to webrender itself? You have two options,
+listed below. Both options will give you a set of patches and the ability to do
+try pushes to verify the update. After that, continue with the steps below to
+actually land the update into the tree.
+
+Option A:
+   Use a script to do the update for you. This will usually work, if you satisfy
+   all the assumptions the script is making. The script can be found at
+   https://github.com/staktrace/moz-scripts/blob/master/try-latest-webrender.sh
+   and contains documentation on how to use it. Read the documentation carefully
+   before trying to use it. The only extra change you need to make with this
+   option is to manually update the revision at the bottom of gfx/doc/README.webrender
+   so that it points to the new WR version you are landing. The script doesn't
+   do that yet.
 
-1) Update your graphics branch checkout to the latest code on the
-   graphics branch
-2) Check out and update the webrender repo to the version you want
-3) Copy over the webrender and webrender_api folders into gfx/. The best way
-   to do this is to simply delete the gfx/webrender and gfx/webrender_api
-   folders and use |cp -R| to copy them in again from the webrender repo. Update
-   the "latest commit" information at the bottom of this file with the version.
-4) If you need to modify webrender_bindings/Cargo.toml to include or remove
-   features, do so now.
-4) Commit your changes to the graphics branch locally
-5) Run |mach vendor rust| to update the rust dependencies in third_party/rust
-6) Commit the vendored changes locally
-7) Build and test. You may need to make changes in bindings.rs or on
-   the C++ side depending on what changed in webrender. This can
-   potentially be quite tricky if you don't fully understand the API
-   changes on the webrender side. In this step, try to not use your new
-   features yet, just get the build working with the minimal changes.
-8) Commit the changes locally from step 7, and push everything to the
-   graphics branch.
-9) Now you have an update webrender with the new features you wanted,
-   so you can write gecko code against them.
-
-Yes, this is somewhat painful. It used to be worse. :)
+Option B:
+   Do the update manually. This is a little more cumbersome but may be required
+   if the script doesn't work or the repos are in a state that violates hidden
+   assumptions in the script (e.g. if the webrender_bindings/Cargo.toml file is
+   no longer in the format expected by the script). The steps to do this are,
+   roughly:
+   - Update your mozilla-central checkout to the latest code on mozilla-central.
+   - Check out and update the webrender repo to the version you want
+   - Copy over the webrender and webrender_api folders into gfx/. The best way
+     to do this is to simply delete the gfx/webrender and gfx/webrender_api
+     folders and use |cp -R| to copy them in again from the webrender repo. Update
+     the "latest commit" information at the bottom of this file with the version.
+   - If you need to modify webrender_bindings/Cargo.toml file, do so now. Changes
+     at this step usually consist of:
+     (a) Updating version numbers. Go through the version numbers of ALL the
+         dependencies in the Cargo.toml file (webrender, euclid, etc.) and make
+         sure the version numbers listed match what's in the new
+         gfx/webrender/Cargo.toml and gfx/webrender_api/Cargo.toml files.
+     (b) Turning on or off any new features that were added in upstream WR. This
+         used to happen a lot but is pretty rare now.
+   - Go to toolkit/library/rust and run |cargo update -p webrender -p webrender_api|.
+     If it complains about version numbers of other crates not lining up, add those
+     as well, e.g. |cargo update -p webrender -p webrender_api -p gleam -p euclid|.
+     You may need to do this a few times until you get all the crates to make it
+     happy.
+   - Run the same cargo update command from the previous step in the
+     toolkit/library/gtest/rust folder.
+   - Commit your changes locally. You'll need to do this before the next step or
+     it will complain.
+   - At the top of the tree, run |mach vendor rust| to update the rust
+     dependencies in third_party/rust.
+   - Commit your changes locally.
+   - Build and test. You may need to make changes in bindings.rs or on the C++
+     side depending on what changed in webrender. This can potentially be quite
+     tricky if you don't fully understand the API changes on the webrender side.
+     Get help if you need it. For simplicity in bisecting, try to not use your
+     new features yet, just get the build working with the minimal changes.
+   - Commit any changes from the previous step, and do a try push to make sure
+     everything is good. Generally we do two try pushes, one for builds and
+     linux tests. This should be totally green. The other forces WR enabled on
+     Windows and runs reftests, which currently fails. However if it fails with
+     more than just regular reftest failures (e.g. it crashes or has an assertion
+     failure) then that's potentially going to be a problem for Windows users
+     running WebRender and will need investigation.
+   - You now have an updated webrender, so you can land it or write gecko
+     code against the new features.
 
-Note that when webrender is built as part of gecko, it may end up using slightly
-different versions of its dependencies than when it is built standalone from the
-webrender repo. The reason is that the Cargo.lock files in m-c and in the WR
-repo may reference different versions of the dependencies. Both builds will be
-compatible in terms of semantic versioning, but may produce different results -
-for example the standalone webrender might use euclid 0.10.4 while the
-one in gecko uses euclid 0.10.3. Although both choices are "valid" per
-the semantic versioning rules in webrender's Cargo.toml, the 0.2.3 may provide
-a bugfix that is needed for correct behaviour in webrender. If this is the case,
-the technically "correct" fix is to change the upstream webrender Cargo.toml
-file to require the correct version. Alternnatively, you can update the
-Cargo.lock files in m-c to pull in the new version. The way to do this is as
-follows:
-- Go to toolkit/library/rust and run |cargo update -p <package> --precise <version>|.
-  Repeat this for as many libraries as you need to update. Run the same commands
-  in toolkit/library/gtest/rust and js/src (ignore any errors about unmatched
-  packages). Commit all the changes locally.
-- Run |mach vendor rust|, which will update the corresponding libraries in
-  third_party/rust to the versions you specified.
-The reason we don't do this by default is to work around bug 1336528. Specifically,
-there is another crate in m-c called mozjs_sys which is built separately but uses
-the same folder to store its rust dependencies. If one of the libraries that is
-required by both mozjs_sys and webrender is updated without updating the other
-project's Cargo.lock file, that results in build bustage.
-This means that any time you do this sort of manual update of packages, you need
-to make sure that mozjs_sys also has its Cargo.lock file updated if needed, hence
-the need to run the cargo update command in js/src as well. Hopefully this will
-be resolved soon.
+Once you have followed either Option A or Option B and have a good update, you
+might want to land it in the tree. To do this:
+- Find the current wr-future-update bug, by going to https://bugzil.la/wr-future-update
+- Clone this bug (there is a little dropdown in the bottom right corner of the
+  page which gives you an option to "Create a new bug ... as a clone of this bug").
+- This will take you to a bug entry page with some stuff prepopulated. Do NOT
+  submit it yet, but make the following changes:
+  (a) Modify the "Description" to remove the SECOND instance of the text "+++ This
+      bug was initially created as a clone of ... +++". Keep the first instance
+      as it points to the bug you just cloned, and keep the rest of the text unless
+      you feel it needs changing.
+  (b) Add wr-future-update into the "Alias" field
+  (c) Clear the bugs in the "Depends on" field
+  (d) For each bug in the "Blocks" field, except for 1311790 and 1386670, go
+      to the bug and check the "See Also" link for the corresponding WR issue/PR,
+      if any. If there is a WR issue that is not yet resolved in the update you
+      are landing, leave the bug in the "Blocks" field of your clone. In a later
+      step you will remove the dependency from the update you are landing. At
+      end of this step the "Blocks" field should contain 1311790, 1386670, and
+      any bugs tracking upstream WR issues that are not fixed in the update.
+  (e) You still cannot submit the clone as a new bug, because you can't have two
+      bugs in the system with the same alias. So hold on a sec.
+- Go back to the tab with the current wr-future-update bug, and click on the edit
+  button. Make the following changes:
+  (a) Assign the bug to yourself.
+  (b) Clear the "Alias" field.
+  (c) Remove bugs from the "Blocks" field that you kept in step (d), other than
+      1311790 and 1386670. In other words, update the "Blocks" field so that it
+      contains 1311790, 1386670, and any bugs that are actually fixed by the
+      update.
+  (d) Submit your changes to this bug.
+- Now you can submit your changes to the clone bug which will create a new
+  wr-future-update bug.
+- Update your patch queue so that the patches are properly formatted with
+  bug number, reviewer, etc. and push to MozReview. This is kind of important,
+  because you want these patches to land on autoland rather than inbound. If it
+  lands on inbound there's a high chance of it conflicting with the servo-vcs-sync
+  bot that is regularly pushing to autoland, and then you'll only find out about
+  it when the sheriff tries to do a merge and backs you out. If you push to
+  autoland you're likely to find out about the problem at push time, when the
+  patches won't rebase.
+
+
+Troubleshooting tips:
 
-Latest Commit: 6440dff485271cdfd24a22c920cea31e01e2b164
+1. Note that when webrender is built as part of gecko, it may end up using slightly
+   different versions of its dependencies than when it is built standalone from the
+   webrender repo. The reason is that the Cargo.lock files in m-c and in the WR
+   repo may reference different versions of the dependencies. Both builds will be
+   compatible in terms of semantic versioning, but may produce different results -
+   for example the standalone webrender might use euclid 0.10.4 while the
+   one in gecko uses euclid 0.10.3. Although both choices are "valid" per
+   the semantic versioning rules in webrender's Cargo.toml, the 0.2.3 may provide
+   a bugfix that is needed for correct behaviour in webrender. If this is the case,
+   the technically "correct" fix is to change the upstream webrender Cargo.toml
+   file to require the correct version. Alternnatively, you can update the
+   Cargo.lock files in m-c to pull in the new version. The way to do this is as
+   follows:
+   - Go to toolkit/library/rust and run |cargo update -p <package> --precise <version>|.
+     Repeat this for as many libraries as you need to update. Run the same commands
+     in toolkit/library/gtest/rust and js/src (ignore any errors about unmatched
+     packages). Commit all the changes locally.
+   - Run |mach vendor rust|, which will update the corresponding libraries in
+     third_party/rust to the versions you specified.
+   The reason we don't do this by default is to work around bug 1336528. Specifically,
+   there is another crate in m-c called mozjs_sys which is built separately but uses
+   the same folder to store its rust dependencies. If one of the libraries that is
+   required by both mozjs_sys and webrender is updated without updating the other
+   project's Cargo.lock file, that results in build bustage.
+   This means that any time you do this sort of manual update of packages, you need
+   to make sure that mozjs_sys also has its Cargo.lock file updated if needed, hence
+   the need to run the cargo update command in js/src as well. Hopefully this will
+   be resolved soon.
+
+2. Sometimes autoland tip has changed enough from mozilla-central (because of the
+   servo vcs-sync-bot, which will sync servo into m-c and often re-vendor third-
+   party rust dependencies) that trying to land an update based on mozilla-central
+   will not work well. As in, you'll get conflicts in Cargo.lock files or in the
+   third_party/rust directory. This is best handled by running your update steps
+   on top of autoland tip rather than central. (The script-based update in option A
+   has an env var you can set to do this). In theory you can get the same
+   result by resolving the conflict manually but Cargo.lock files are usually not
+   trivial to merge by hand. If it's just the third_party/rust dir that has conflicts
+   you can delete it and run |mach vendor rust| again to repopulate it.
+
+-------------------------------------------------------------------------------
+
+The version of WebRender currently in the tree is:
+6440dff485271cdfd24a22c920cea31e01e2b164