Bug 1432416 - Add comments and documentation about adding new DAMP tests;r=ochameau
MozReview-Commit-ID: LxFbX0AzJfX
--- a/devtools/docs/tests/writing-perf-tests.md
+++ b/devtools/docs/tests/writing-perf-tests.md
@@ -1,13 +1,23 @@
# Writing new performance test
See [Performance tests (DAMP)](performance-tests.md) for an overall description of our performance tests.
Here, we will describe how to write a new test with an example: track the performance of clicking inside the inspector panel.
+## Consider modifying existing tests first
+
+If a `custom` page already exists for the tool you are testing, try to modify the existing `custom` test rather than adding a new individual test.
+
+New individual tests run separately, in new tabs, and make DAMP slower than just modifying existing tests. Complexifying `custom` test pages should also help cover more scenarios and catch more regressions. For those reasons, modifying existing tests should be the preferred way of extending DAMP coverage.
+
+`custom` tests are using complex documents that should stress a particular tool in various ways. They are all named `custom.${tool}` (for instance `custom.inspector`). The test pages for those tests can be found in [pages/custom](https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/devtools/addon/content/pages/custom).
+
+If your test case requires a dedicated document or can't run next to the other tests in the current `custom` test, follow the instructions below to add a new individual test.
+
## Where is the code for the test?
For now, all the tests live in a single file [damp.js](https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/devtools/addon/content/damp.js).
There are two kinds of tests:
* The first kind is being run against two documents:
* "Simple", an empty webpage. This one helps highlighting the load time of panels,
* "Complicated", a copy of bild.be, a German newspaper website. This allows us to examine the performance of the tools when inspecting complicated, big websites.
--- a/testing/talos/talos/tests/devtools/addon/content/damp.js
+++ b/testing/talos/talos/tests/devtools/addon/content/damp.js
@@ -1127,23 +1127,26 @@ async _consoleOpenWithCachedMessagesTest
expectedText: "Math;function ga(a,b){return a.name=b}"
}));
// Run all tests against a document specific to each tool
tests["custom.inspector"] = this.customInspector;
tests["custom.debugger"] = this.customDebugger;
tests["custom.webconsole"] = this.customConsole;
- // Run individual tests covering a very precise tool feature
+ // Run individual tests covering a very precise tool feature.
tests["console.bulklog"] = this._consoleBulkLoggingTest;
tests["console.streamlog"] = this._consoleStreamLoggingTest;
tests["console.objectexpand"] = this._consoleObjectExpansionTest;
tests["console.openwithcache"] = this._consoleOpenWithCachedMessagesTest;
tests["inspector.mutations"] = this._inspectorMutationsTest;
tests["inspector.layout"] = this._inspectorLayoutTest;
+ // ⚠ Adding new individual tests slows down DAMP execution ⚠
+ // ⚠ Consider contributing to custom.${tool} rather than adding isolated tests ⚠
+ // ⚠ See http://docs.firefox-dev.tools/tests/writing-perf-tests.html ⚠
// Filter tests via `./mach --subtests filter` command line argument
let filter = Services.prefs.getCharPref("talos.subtests", "");
if (filter) {
for (let name in tests) {
if (!name.includes(filter)) {
delete tests[name];
}