This fixes a regression from bug 1454640 where urls had an extra 'docs' path inserted into them, e.g:
toolkit/components/telemetry/telemetry/index.html
became:
toolkit/components/telemetry/docs/telemetry/index.html
Differential Revision: https://phabricator.services.mozilla.com/D2079
--HG--
extra : histedit_source : 15c679e2f9366c1ea424adc4c82d7c184d80b3bb
This upgrades sphinx to version 1.7.5, which contained a couple backwards
incompatible changes that needed fixing.
This also leaves sphinx-js at version 2.1 as upgrading that to 2.5 seems to
introduce an intermittent in the Doc task.
MozReview-Commit-ID: FRUTcXs5yzb
--HG--
extra : rebase_source : e874a2e9c637b7cec710203f75f4dd989a5681a1
This was changing the project from "main" -> ("main",) which was causing the DocUp
task to upload to urls like:
"('main',)/62.0/_sources/..."
MozReview-Commit-ID: 1bL9nqiAEFE
--HG--
extra : rebase_source : 8f4148dd003dfcfb4c0ba513e4fa7e2ca540dce3
We no longer store the docs under a project name (since all the docs are now
built using the root conf.py). This mean the name and version are only used for
packaging and uploading, which typically is only used in CI.
This allows us to lazy load the package name and version, so we only read the
conf.py when we need to.
MozReview-Commit-ID: DV5Jxrbskoh
--HG--
extra : source : d03f75986f626938142bd5bde293773506a2fc14
extra : amend_source : d6d0ff5673515cac3a6c7cf03bfce4ffcd09991f
Previously, running |mach doc <subtree>| would use whatever conf.py file
happened to live in the subtree. For example, running:
./mach doc tools/lint
Would build with tools/lint/docs/conf.py. This is bad because it means the
generated docs will look different from the docs that eventually will be
published to firefox-source-docs.mozilla.com.
This patch makes sure we always use tools/docs/conf.py for building, even when
only generating a subtree.
Furthermore, this sets things up such that when you modify a file, only the
subtree containing the modified file will be re-generated. This cuts down
rebuild times from ~2 minutes to ~20 seconds.
There is one caveat. When rebuilding a subtree, the index of other trees will
be overwritten in that particular subtree. I couldn't figure out anyway around
this. This tradeoff for *much* faster rebuild times seems worth it.
MozReview-Commit-ID: Ly88mvHKpo7
--HG--
extra : source : 47fc3e2238676a40a3adc84239baed1ce873e95e
The current mechanism for reading SPHINX variables assumes we always want to
read metadata for the entire tree. Now that we have the ability to rebuild
specific subtrees, this assumption is false.
This patch allows us to specify a path that find_sphinx_variables can use to
filter down the set of moz.build variables it will traverse, yielding only
moz.builds that could potentially impact the specified path.
MozReview-Commit-ID: ALrCFLFgMLH
--HG--
extra : source : 22f2dc60e6d859d3ca411826c77002d87c1a49bd
Now that we can rebuild docs with the liveserver, there are some optimizations
we should make. One of those is processing the sphinx moz.build variables. This
patch makes sure we don't re-process moz.build if we've already done so in a
previous rebuild.
MozReview-Commit-ID: 2AIr1KeAPQV
--HG--
extra : source : 30b4083534b51213a1b9fe0d86f996cfa0e7fa54
In the mozbuild.sphinx extension, we create a new SphinxManager instance each
time. However this isn't ideal now that we can rebuild the docs within the same
interpreter using the livereload server.
This makes use of a singleton so that we can share state not only between
multiple invocations of sphinx-build, but also with the mach command. This will
be taken advantage of more heavily in future commits in this series.
MozReview-Commit-ID: 7ERYeN5BPeI
--HG--
extra : source : 8309212d820bcca29aa95b7892d39940437f2aa8
These two functions are typically only used by CI for packaging/uploading the
documentation. This is a minor re-organiztion for clarity.
MozReview-Commit-ID: 62UhQhSSkOs
--HG--
extra : source : 9942d2df371989c5dd67a75fd7a695533141dd89
We no longer store the docs under a project name (since all the docs are now
built using the root conf.py). This mean the name and version are only used for
packaging and uploading, which typically is only used in CI.
This allows us to lazy load the package name and version, so we only read the
conf.py when we need to.
MozReview-Commit-ID: DV5Jxrbskoh
--HG--
extra : rebase_source : d227b8a800715c677602cbbc9b868df5dbd31131
Previously, running |mach doc <subtree>| would use whatever conf.py file
happened to live in the subtree. For example, running:
./mach doc tools/lint
Would build with tools/lint/docs/conf.py. This is bad because it means the
generated docs will look different from the docs that eventually will be
published to firefox-source-docs.mozilla.com.
This patch makes sure we always use tools/docs/conf.py for building, even when
only generating a subtree.
Furthermore, this sets things up such that when you modify a file, only the
subtree containing the modified file will be re-generated. This cuts down
rebuild times from ~2 minutes to ~20 seconds.
There is one caveat. When rebuilding a subtree, the index of other trees will
be overwritten in that particular subtree. I couldn't figure out anyway around
this. This tradeoff for *much* faster rebuild times seems worth it.
MozReview-Commit-ID: Ly88mvHKpo7
--HG--
extra : rebase_source : 379ffb482a4d15f5f6b394f93adab2b536c7ce00
The current mechanism for reading SPHINX variables assumes we always want to
read metadata for the entire tree. Now that we have the ability to rebuild
specific subtrees, this assumption is false.
This patch allows us to specify a path that find_sphinx_variables can use to
filter down the set of moz.build variables it will traverse, yielding only
moz.builds that could potentially impact the specified path.
MozReview-Commit-ID: ALrCFLFgMLH
--HG--
extra : rebase_source : d0c26a006bb4dbc429be5eedad7825d4412dc2a4
Now that we can rebuild docs with the liveserver, there are some optimizations
we should make. One of those is processing the sphinx moz.build variables. This
patch makes sure we don't re-process moz.build if we've already done so in a
previous rebuild.
MozReview-Commit-ID: 2AIr1KeAPQV
--HG--
extra : rebase_source : d8034ef5be416975c19a473d5a13f073e95aba80
In the mozbuild.sphinx extension, we create a new SphinxManager instance each
time. However this isn't ideal now that we can rebuild the docs within the same
interpreter using the livereload server.
This makes use of a singleton so that we can share state not only between
multiple invocations of sphinx-build, but also with the mach command. This will
be taken advantage of more heavily in future commits in this series.
MozReview-Commit-ID: 7ERYeN5BPeI
--HG--
extra : rebase_source : 44aee637ea9b828b43b82e8639ddc3cc7f68c797
These two functions are typically only used by CI for packaging/uploading the
documentation. This is a minor re-organiztion for clarity.
MozReview-Commit-ID: 62UhQhSSkOs
--HG--
extra : rebase_source : c5b800734041689453f965e50ce356c2e7a623d2
This changes the default to opening a livereload webserver after doc generation
(as opposed to opening the index file). Any changes to the specified path will
result in a rebuild and refresh of the browser.
For example, if you run:
./mach doc tools/lint
The linting docs will be built, served and opened in a browser. Modifying any
file under 'tools/lint/docs' will refresh the browser with your changes.
To disable this behaviour and simply open the index file, you can pass in
'--no-serve'. The '--no-open' flag will continue to work (both with http and
the file system).
One caveat to this patch is that when generating the root docs (by running
|mach doc|), we don't watch all possible doc paths (just the root one under
'tools/docs/'). This will probably be fixed in the follow-up bug 1454640.
MozReview-Commit-ID: FQecuePM0zZ
--HG--
extra : rebase_source : 3240402d7505e99a4f64dada309b1baec78306e1
This removes the ability to specify multiple doc paths at the same time with
|mach doc|. We will be changing the default from opening index files to serving
the documentation with a webserver. Supporting multiple doc roots would mean
spinning up multiple servers in different threads.
This would add a lot of complexity for a feature which I don't think is very
useful. It's very rare that one would need to edit more than one doc location
at the same time. And if this is ever needed, the developer can just build the
entire doctree (by running |mach doc|) or run |mach doc <path>| in multiple
different terminals.
MozReview-Commit-ID: GXEZJSgLpgF
--HG--
extra : rebase_source : 2eda23274eb6c2be82f7e77ca577072386bada34
Some requirements.txt are very large and result in a lot of package already
installed messages. Would be nice to hide this.
MozReview-Commit-ID: FQecuePM0zZ
--HG--
extra : rebase_source : 58eaa7324775cfaa39077871be0be0ef39ad7c11
Previously we weren't explicitly installing sphinx. Instead, the 'sphinx-js'
package had a dependency on 'sphinx<2.0'. This caused errors when sphinx
released their backwards incompatible version 1.7.
This patch pins sphinx==1.6.7 and adds all other dependencies to the same
requirements.txt (with hashes).
Upgrading to sphinx==1.7 will happen in a follow-up.
MozReview-Commit-ID: 28fKI7T4vfa
--HG--
extra : rebase_source : a9f276586ed08f49c1a26088aae88c363a31c167
This is ancient and the team that used it (gfx) is no longer using it.
MozReview-Commit-ID: HrDgmAU9QeW
--HG--
extra : rebase_source : c4a64965c4ae1a50888893e881a6e8a9688a58b6
This enables sphinx to parse both the google and numpy style docstring
formats which tend to be more human readable than the default sphinx
format.
See:
http://www.sphinx-doc.org/en/stable/ext/napoleon.html
MozReview-Commit-ID: REmZ4IoUG8
--HG--
extra : rebase_source : 4e3e788d09a7fcc3d3e84bb94744019583e1ee5e
This reduces the time taken to upload the Firefox docs from ~30s to
~5s (per invocation).
MozReview-Commit-ID: DxOrvxvVn42
--HG--
extra : rebase_source : b170fb728267c43484573e0a57ebc96512eb0e7c
Previously, we uploaded the main Firefox tree docs to /.
In reality, there are multiple Sphinx projects in the repo. In
addition, it is sometimes desirable to access docs for an older
version of Firefox.
In this commit, we add support for specifying the S3 key prefix
for uploads. Then we change the upload code to upload to multiple
locations:
* <project>/latest (always)
* <project>/<version> (if a version is defined in the Sphinx config)
* / (for the main Sphinx docs project)
For the Firefox docs, ``version`` corresponds to a sanitized value from
``milestone.txt``. Currently, it resolves to ``57.0``.
While we're here, we add support for declaring an alternate project
name in the Sphinx conf.py file. If ``moz_project_name`` is defined,
we use that as the project name. For Firefox, we set it to ``main``.
This means our paths (local and uploaded) are now ``main`` instead of
``Mozilla_Source_Tree_Docs``. That's much more pleasant.
MozReview-Commit-ID: 8Gl6l2m6uU4
--HG--
extra : rebase_source : e56885092c12eb8cc76e5e7300f938be566e3e5a
extra : intermediate-source : 8509af1e135177a93460270b27f263c10a62d996
extra : source : 71b4f32caf209fe9dffc340c0b8ccb51ac79c7de
We now have an --upload flag to control whether upload is performed.
We don't inline it because we want to maintain a "firewall" between
regular docs and all the extra packages and imports needed for S3.
MozReview-Commit-ID: DVKhsS545gp
--HG--
extra : rebase_source : 7fee832145189be882db0f0ca057eda6158e0492
Instead of doing the file finding inside s3_upload(), the function now
takes the output of distribution_files().
The new code is much simpler.
MozReview-Commit-ID: 43i2Alvyu5i
--HG--
extra : rebase_source : 29d840c65acec794749edfda0e47193806f387f4
extra : source : 5a758defcf2f1a17de851e703388469300789ac5
We try to keep mach_commands.py files as minimal as possible.
MozReview-Commit-ID: I4kvZtDjqGd
--HG--
extra : rebase_source : 7b0af45661fc2a8b0b906a569aa95df7d3e0513b
By using mozpack, we get deterministic archives. This also makes the
task command simpler.
MozReview-Commit-ID: EPI7tuGQuso
--HG--
extra : rebase_source : babcbf3d5607480321106b40eaa4598c0faee188
An upcoming commit will want to put something outside the
format-specific output directory.
MozReview-Commit-ID: 2kYDREddpN
--HG--
extra : rebase_source : 28454f71b3abbd094048edb568b4f5e8d50881d8
I didn't fix all violations in this file. Something is better than
nothing.
MozReview-Commit-ID: BDdWhJfkPVk
--HG--
extra : rebase_source : caa0a5908c2cb37b8cbb1c5ee004d43e3a0b883a
