Skip to content

Commit 2214d1f

Browse files
chtruong814chtruong814
authored
docs: Enable docs self-publishing including nightly update (#15432)
* Add initial uv lock Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Add build docs Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Fix docs build Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Fix docstring formatting in magpietts.py Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Fix docs build Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Rename broken links files Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Add release docs jobs Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Fix docs Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Dry-run of docs publishing Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Revert "Dry-run of docs publishing" This reverts commit 1c3aa19. Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Revert "Revert "Dry-run of docs publishing"" This reverts commit 43c19ae. Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Fix dry run Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Fix broken links Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Add retries for linke check Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Fix broken link Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Fix broken link Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Revert "Revert "Revert "Dry-run of docs publishing""" This reverts commit a28f306. Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Revert "Revert "Revert "Revert "Dry-run of docs publishing"""" This reverts commit 9353dcb. Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Test nightly publish Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Increase docs broken link retry and timeout Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Revert "Test nightly publish" This reverts commit 9c8e7a4. Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Revert "Revert "Revert "Revert "Revert "Dry-run of docs publishing""""" This reverts commit 850207e. Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Fix docs footer Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Test nightly push Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Revert "Test nightly push" This reverts commit 05894f2. Signed-off-by: Charlie Truong <chtruong@nvidia.com> --------- Signed-off-by: Charlie Truong <chtruong@nvidia.com>
1 parent 15173d5 commit 2214d1f

File tree

18 files changed

+8785
-196
lines changed

18 files changed

+8785
-196
lines changed

‎.github/workflows/build-docs.yml‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
# Copyright (c) 2026, NVIDIA CORPORATION.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
name: Build docs
16+
17+
on:
18+
pull_request:
19+
branches:
20+
- main
21+
- r**
22+
paths:
23+
- "docs/**"
24+
push:
25+
branches:
26+
- main
27+
- r**
28+
paths:
29+
- "docs/**"
30+
31+
concurrency:
32+
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}-${{ github.event.label.name || 'main' }}-${{ github.event_name }}
33+
cancel-in-progress: true
34+
35+
jobs:
36+
pre-flight:
37+
uses: NVIDIA-NeMo/FW-CI-templates/.github/workflows/_cicd_preflight.yml@v0.74.0
38+
39+
build-docs:
40+
needs: [pre-flight]
41+
if: needs.pre-flight.outputs.is_deployment_workflow != 'true'
42+
uses: NVIDIA-NeMo/FW-CI-templates/.github/workflows/_build_docs.yml@v0.74.0
43+
with:
44+
docs-directory: docs/source
45+
sync-all: true
46+
47+
build-docs-summary:
48+
needs: [pre-flight, build-docs]
49+
if: |
50+
(
51+
needs.pre-flight.outputs.is_deployment_workflow == 'true'
52+
|| always()
53+
)
54+
&& !cancelled()
55+
runs-on: ubuntu-latest
56+
steps:
57+
- name: Get workflow result
58+
id: result
59+
shell: bash -x -e -u -o pipefail {0}
60+
env:
61+
GH_TOKEN: ${{ github.token }}
62+
RUN_ID: ${{ github.run_id }}
63+
SKIPPING_IS_ALLOWED: ${{ needs.pre-flight.outputs.docs_only == 'true' || needs.pre-flight.outputs.is_deployment_workflow == 'true' }}
64+
run: |
65+
FAILED_JOBS=$(gh run view $GITHUB_RUN_ID --json jobs --jq '[.jobs[] | select(.status == "completed" and .conclusion != "success")] | length') || echo 0
66+
67+
if [ "${FAILED_JOBS:-0}" -eq 0 ] || [ "$SKIPPING_IS_ALLOWED" == "true" ]; then
68+
echo "✅ All previous jobs completed successfully"
69+
exit 0
70+
else
71+
echo "❌ Found $FAILED_JOBS failed job(s)"
72+
# Show which jobs failed
73+
gh run view $GITHUB_RUN_ID --json jobs --jq '.jobs[] | select(.status == "completed" and .conclusion != "success") | .name'
74+
exit 1
75+
fi

‎.github/workflows/gh-docs.yml‎

Lines changed: 0 additions & 81 deletions
This file was deleted.

‎.github/workflows/release-docs.yml‎

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
# Copyright (c) 2026, NVIDIA CORPORATION.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
name: Release docs
15+
on:
16+
workflow_dispatch:
17+
inputs:
18+
dry-run:
19+
description: Whether to run the workflow in dry-run mode
20+
required: true
21+
type: boolean
22+
default: true
23+
publish-as-latest:
24+
description: Publish as Latest stable version.
25+
required: false
26+
type: boolean
27+
default: true
28+
docs-version-override:
29+
description: Docs version if commit is not tagged
30+
required: false
31+
type: string
32+
default: ""
33+
update-version-picker:
34+
description: Update version picker.
35+
required: false
36+
type: boolean
37+
default: true
38+
notify-emails:
39+
description: Email addresses to send the notification to. Format as "me@me.com,you@you.com".
40+
required: false
41+
type: string
42+
github-ref:
43+
description: Github ref to checkout
44+
required: false
45+
type: string
46+
default: ""
47+
workflow_call:
48+
inputs:
49+
dry-run:
50+
description: Whether to run the workflow in dry-run mode
51+
required: false
52+
type: boolean
53+
default: true
54+
publish-as-latest:
55+
description: Publish as Latest stable version.
56+
required: false
57+
type: boolean
58+
default: true
59+
docs-version-override:
60+
description: Docs version if commit is not tagged
61+
required: false
62+
type: string
63+
default: ""
64+
update-version-picker:
65+
description: Update version picker.
66+
required: false
67+
type: boolean
68+
default: true
69+
notify-emails:
70+
description: Email addresses to send the notification to. Format as "me@me.com,you@you.com".
71+
required: false
72+
type: string
73+
github-ref:
74+
description: Github ref to checkout
75+
required: false
76+
type: string
77+
default: ""
78+
79+
jobs:
80+
build-docs:
81+
uses: NVIDIA-NeMo/FW-CI-templates/.github/workflows/_build_docs.yml@v0.74.0
82+
with:
83+
ref: ${{ inputs.github-ref }}
84+
docs-directory: docs/source
85+
sync-all: true
86+
87+
publish-docs:
88+
runs-on: ubuntu-latest
89+
needs: [build-docs]
90+
steps:
91+
- uses: actions/checkout@v6
92+
with:
93+
repository: NVIDIA-NeMo/FW-CI-templates
94+
ref: v0.74.0
95+
path: FW-CI-templates
96+
97+
- uses: ./FW-CI-templates/.github/actions/publish-docs
98+
# This workflow runs either on main, or on a version tag. Any other git ref will lead
99+
# to an error.
100+
# If its on main, it will publish to "latest" directory in Akamai.
101+
# If its on a versioned tag, it will extract the version number from the tag (strip `v` prefix)
102+
# and publish to the versioned directory in Akamai.
103+
with:
104+
dry-run: ${{ inputs.dry-run }}
105+
artifacts-name: docs-html
106+
artifacts-path: _build/html
107+
emails-csv: ${{ inputs.notify-emails && format('{0},{1}', vars.docs_release_emails, inputs.notify-emails) || vars.docs_release_emails }}
108+
overwrite-latest-on-tag: ${{ inputs.publish-as-latest }}
109+
docs-version-override: ${{ inputs.docs-version-override }}
110+
update-version-picker: ${{ inputs.update-version-picker }}
111+
run-on-version-tag-only: ${{ github.ref_name != 'main' }}
112+
request-name: nemo-speech-publish-docs-${{ github.run_id }}
113+
aws-region: ${{ vars.DOCS_AWS_REGION }}
114+
aws-role-to-assume: ${{ secrets.AWS_ASSUME_ROLE_ARN }}
115+
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
116+
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
117+
akamai-host: ${{ secrets.AKAMAI_HOST }}
118+
akamai-client-token: ${{ secrets.AKAMAI_CLIENT_TOKEN }}
119+
akamai-client-secret: ${{ secrets.AKAMAI_CLIENT_SECRET }}
120+
akamai-access-token: ${{ secrets.AKAMAI_ACCESS_TOKEN }}
121+
s3-target-root: ${{ secrets.S3_BUCKET_NAME }}
122+
s3-target-path: nemo/speech

‎.github/workflows/release-nightly-docs.yml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Copyright (c) 2026, NVIDIA CORPORATION.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
name: Release Nightly Docs
16+
17+
on:
18+
schedule:
19+
- cron: "0 10 * * *"
20+
21+
jobs:
22+
call-release-docs:
23+
uses: ./.github/workflows/release-docs.yml
24+
with:
25+
dry-run: false
26+
publish-as-latest: false
27+
docs-version-override: "nightly"
28+
update-version-picker: false
29+
secrets: inherit

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,7 @@ instance/
8888

8989
# Sphinx documentation
9090
docs/build
91+
docs/source/_build
9192

9293
# PyBuilder
9394
target/

‎docs/source/_templates/layout.html‎

Lines changed: 0 additions & 14 deletions
This file was deleted.

‎docs/source/audio/configs.rst‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ NeMo Dataset Configuration
1616

1717
Training, validation, and test parameters are specified using the ``model.train_ds``, ``model.validation_ds``, and ``model.test_ds`` sections in the configuration file, respectively.
1818
Depending on the task, there may be arguments specifying the sample rate or duration of the loaded audio examples. Some fields can be left out and specified via the command-line at runtime.
19-
Refer to the `Dataset Processing Classes <./api.html#Datasets>`__ section of the API for a list of datasets classes and their respective parameters.
19+
Refer to the `Dataset Processing Classes <./api.html#datasets>`__ section of the API for a list of datasets classes and their respective parameters.
2020
An example train, validation and test datasets can be configured as follows:
2121

2222
.. code-block:: yaml
@@ -172,7 +172,7 @@ An example of a simple predictive model configuration is shown below:
172172
173173
decoder:
174174
_target_: nemo.collections.audio.modules.transforms.SpectrogramToAudio
175-
fft_length: ${model.encoder.fft_length}
175+
fft_length: ${model.encoder.fft_length}
176176
hop_length: ${model.encoder.hop_length}
177177
magnitude_power: ${model.encoder.magnitude_power}
178178
scale: ${model.encoder.scale}
@@ -184,15 +184,15 @@ An example of a simple predictive model configuration is shown below:
184184
num_res_blocks: 3 # increased number of res blocks
185185
pad_time_to: 64 # pad to 64 frames for the time dimension
186186
pad_dimension_to: 0 # no padding in the frequency dimension
187-
187+
188188
loss:
189189
_target_: nemo.collections.audio.losses.MSELoss # computed in the time domain
190190
191191
metrics:
192192
val:
193193
sisdr: # output SI-SDR
194194
_target_: torchmetrics.audio.ScaleInvariantSignalDistortionRatio
195-
195+
196196
optim:
197197
name: adam
198198
lr: 1e-4

‎docs/source/audio/datasets.rst‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ Each entry in the manifest (describing one audio file) should be bordered by ``"
2424

2525
Once there is a manifest that describes each audio file in the dataset, assign the ``JSON`` manifest file path in the experiment config file, for example, ``training_ds.manifest_filepath=<path/to/manifest.json>``.
2626

27-
For more information about the individual tarred datasets and the parameters available, including shuffling options, see the corresponding class APIs in the `Datasets <./api.html#Datasets>`_ section.
27+
For more information about the individual tarred datasets and the parameters available, including shuffling options, see the corresponding class APIs in the `Datasets <./api.html#datasets>`_ section.
2828

2929

3030
Lhotse Format
@@ -88,4 +88,4 @@ Then, create the Lhotse shar dataset.
8888
OUTPUT_DIR=/path/to/data/shar
8989
9090
# create shars, each with 2084 examples, flac audio format
91-
lhotse shar export --num-jobs 16 --verbose --shard-size 2084 --audio flac ${LHOTSE_MANIFEST} ${OUTPUT_DIR}
91+
lhotse shar export --num-jobs 16 --verbose --shard-size 2084 --audio flac ${LHOTSE_MANIFEST} ${OUTPUT_DIR}

docs/false_positives.json renamed to docs/source/broken_links_false_positives.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,3 +70,6 @@
7070
"uri": "https://nvidia.github.io/NeMo/blogs/2023/2023-08-forced-alignment/",
7171
"info": "404 Client Error: Not Found for url: https://nvidia.github.io/NeMo/blogs/2023/2023-08-forced-alignment/"
7272
}
73+
{ "filename": "audio/datasets.rst", "lineno": 27, "status": "broken", "code": 0, "uri": "./api.html#datasets", "info": "" }
74+
{ "filename": "audio/configs.rst", "lineno": 17, "status": "broken", "code": 0, "uri": "./api.html#datasets", "info": "" }
75+
{ "filename": "tts/configs.rst", "lineno": 19, "status": "broken", "code": 0, "uri": "./api.html#datasets", "info": "" }

0 commit comments

Comments
 (0)