add a very basic precompile workload by KristofferC · Pull Request #2004 · JuliaDocs/Documenter.jl

KristofferC · 2022-12-30T11:08:40Z

Before:

❯ time julia --project docs_precompile/make.jl
julia --project docs_precompile/make.jl  7.15s user 0.23s system 93% cpu 7.901 total

After

❯ time julia --project docs_precompile/make.jl
julia --project docs_precompile/make.jl  0.63s user 0.09s system 58% cpu 1.234 total

Using Julia-1.9.0-beta2.

Edit by @mortenpi: close #2002

mortenpi

Something to note is that the first run time (i.e. precompilation + makedocs) goes up with this (14s -> 20s on the docs_precompile workload). But I think that is not a particularly problematic here, as you would mostly run into this on CI.

tfiers · 2023-01-15T18:28:29Z

first run time (i.e. precompilation + makedocs) goes up with this (14s -> 20s on the docs_precompile workload). But I think that is not a particularly problematic here, as you would mostly run into this on CI.

Is it possible to cache precompilation between CI runs?
If so, that might be very worthwile to add to e.g. the PkgTemplates doc-build step

mortenpi · 2023-01-15T20:42:06Z

Is it possible to cache precompilation between CI runs?

Yes (https://github.com/julia-actions/cache#inputs), but it looks like it might create other issues.

mortenpi · 2023-01-15T21:23:56Z

It looks like the nightly failures are related to this PR (cf. https://github.com/JuliaDocs/Documenter.jl/actions/runs/3925278191 for a new master run). Any thoughts @KristofferC?

inkydragon · 2023-02-27T15:42:10Z

Maybe run CI again.
I cannot reproduce the error locally with Version 1.10.0-DEV.665 (2023-02-27)

mortenpi · 2023-02-28T00:33:28Z

CI still fails it looks like.

inkydragon · 2023-02-28T02:43:11Z

I can reproduce this issue on Version 1.10.0-DEV.667 (2023-02-27).

julia-latest --color=yes --project=@. -e 'using Pkg; Pkg.build(verbose=true);' 
julia-latest --color=yes --depwarn=yes --inline=yes --project=@. .\test\runtests.jl

Internal error: encountered unexpected error in runtime:
UndefRefError()
...

Documenter: Error During Test at E:\jl\Documenter.jl\test\runtests.jl:5
  Got exception outside of a @test
  LoadError: TypeError: in new, expected IOStream, got a value of type Base.PipeEndpoint
  Stacktrace:
...

full logs

> julia-latest --color=yes --project=@. .\test\runtests.jl
[ Info: Building example/make.jl
[ Info: Building mock package docs: HTMLWriter / deployment build
Internal error: encountered unexpected error in runtime:
UndefRefError()
getindex at .\essentials.jl:13 [inlined]
copy_exprargs at .\expr.jl:68
copy at .\expr.jl:41
copy_exprs at .\expr.jl:46
copy_exprargs at .\expr.jl:68
copy at .\expr.jl:75
inflate_ir at .\compiler/ssair\legacy.jl:60 [inlined]
codeinst_to_ir at .\compiler/ssair\irinterp.jl:129 [inlined]
abstract_call_method_with_const_args at .\compiler\abstractinterpretation.jl:978
abstract_call_method_with_const_args at .\compiler\abstractinterpretation.jl:960
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:158
abstract_call_known at .\compiler\abstractinterpretation.jl:2009
abstract_call at .\compiler\abstractinterpretation.jl:2085
abstract_call at .\compiler\abstractinterpretation.jl:2059
abstract_eval_statement_expr at .\compiler\abstractinterpretation.jl:2246
abstract_eval_statement at .\compiler\abstractinterpretation.jl:2500
abstract_eval_basic_statement at .\compiler\abstractinterpretation.jl:2767
typeinf_local at .\compiler\abstractinterpretation.jl:2976
typeinf_nocycle at .\compiler\abstractinterpretation.jl:3064
_typeinf at .\compiler\typeinfer.jl:244
typeinf at .\compiler\typeinfer.jl:215
typeinf_edge at .\compiler\typeinfer.jl:920
abstract_call_method at .\compiler\abstractinterpretation.jl:653
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:150
abstract_call_known at .\compiler\abstractinterpretation.jl:2009
abstract_call at .\compiler\abstractinterpretation.jl:2085
abstract_call at .\compiler\abstractinterpretation.jl:2059
abstract_eval_statement_expr at .\compiler\abstractinterpretation.jl:2246
abstract_eval_statement at .\compiler\abstractinterpretation.jl:2500
abstract_eval_basic_statement at .\compiler\abstractinterpretation.jl:2791
typeinf_local at .\compiler\abstractinterpretation.jl:2976
typeinf_nocycle at .\compiler\abstractinterpretation.jl:3064
_typeinf at .\compiler\typeinfer.jl:244
typeinf at .\compiler\typeinfer.jl:215
typeinf_edge at .\compiler\typeinfer.jl:920
abstract_call_method at .\compiler\abstractinterpretation.jl:653
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:150
abstract_call_known at .\compiler\abstractinterpretation.jl:2009
abstract_call at .\compiler\abstractinterpretation.jl:2085
abstract_call at .\compiler\abstractinterpretation.jl:2059
abstract_eval_statement_expr at .\compiler\abstractinterpretation.jl:2246
abstract_eval_statement at .\compiler\abstractinterpretation.jl:2500
abstract_eval_basic_statement at .\compiler\abstractinterpretation.jl:2767
typeinf_local at .\compiler\abstractinterpretation.jl:2976
typeinf_nocycle at .\compiler\abstractinterpretation.jl:3064
_typeinf at .\compiler\typeinfer.jl:244
typeinf at .\compiler\typeinfer.jl:215
typeinf_edge at .\compiler\typeinfer.jl:920
abstract_call_method at .\compiler\abstractinterpretation.jl:653
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:150
abstract_call_known at .\compiler\abstractinterpretation.jl:2009
abstract_call at .\compiler\abstractinterpretation.jl:2085
abstract_call at .\compiler\abstractinterpretation.jl:2059
abstract_eval_statement_expr at .\compiler\abstractinterpretation.jl:2246
abstract_eval_statement at .\compiler\abstractinterpretation.jl:2500
abstract_eval_basic_statement at .\compiler\abstractinterpretation.jl:2791
typeinf_local at .\compiler\abstractinterpretation.jl:2976
typeinf_nocycle at .\compiler\abstractinterpretation.jl:3064
_typeinf at .\compiler\typeinfer.jl:244
typeinf at .\compiler\typeinfer.jl:215
typeinf_edge at .\compiler\typeinfer.jl:920
abstract_call_method at .\compiler\abstractinterpretation.jl:653
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:150
abstract_call_known at .\compiler\abstractinterpretation.jl:2009
abstract_call at .\compiler\abstractinterpretation.jl:2085
abstract_call at .\compiler\abstractinterpretation.jl:2059
abstract_eval_statement_expr at .\compiler\abstractinterpretation.jl:2246
abstract_eval_statement at .\compiler\abstractinterpretation.jl:2500
abstract_eval_basic_statement at .\compiler\abstractinterpretation.jl:2791
typeinf_local at .\compiler\abstractinterpretation.jl:2976
typeinf_nocycle at .\compiler\abstractinterpretation.jl:3064
_typeinf at .\compiler\typeinfer.jl:244
typeinf at .\compiler\typeinfer.jl:215
typeinf_ext at .\compiler\typeinfer.jl:1044
typeinf_ext_toplevel at .\compiler\typeinfer.jl:1077
typeinf_ext_toplevel at .\compiler\typeinfer.jl:1073
jfptr_typeinf_ext_toplevel_20819.clone_1 at C:\Users\woclass\AppData\Local\julias\julia-latest\lib\julia\sys.dll (unknown line)
_jl_invoke at C:/workdir/src\gf.c:2725 [inlined]
ijl_apply_generic at C:/workdir/src\gf.c:2907 [inlined]
jl_apply at C:/workdir/src\julia.h:1876 [inlined]
jl_type_infer at C:/workdir/src\gf.c:317
jl_generate_fptr_impl at C:/workdir/src\jitlayers.cpp:444
jl_compile_method_internal at C:/workdir/src\gf.c:2316
jl_compile_method_internal at C:/workdir/src\gf.c:2209 [inlined]
_jl_invoke at C:/workdir/src\gf.c:2717 [inlined]
ijl_apply_generic at C:/workdir/src\gf.c:2907
jl_apply at C:/workdir/src\julia.h:1876 [inlined]
do_call at C:/workdir/src\interpreter.c:125
eval_value at C:/workdir/src\interpreter.c:222
eval_body at C:/workdir/src\interpreter.c:474
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:758
top-level scope at E:\jl\Documenter.jl\test\examples\make.jl:255
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:910
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:853
ijl_toplevel_eval at C:/workdir/src\toplevel.c:919 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:969
eval at .\boot.jl:370 [inlined]
include_string at .\loading.jl:1889
_include at .\loading.jl:1949
include at .\client.jl:478
unknown function (ip: 00000167ffe90336)
jl_apply at C:/workdir/src\julia.h:1876 [inlined]
do_call at C:/workdir/src\interpreter.c:125
eval_value at C:/workdir/src\interpreter.c:222
eval_stmt_value at C:/workdir/src\interpreter.c:173 [inlined]
eval_body at C:/workdir/src\interpreter.c:620
eval_body at C:/workdir/src\interpreter.c:529
eval_body at C:/workdir/src\interpreter.c:529
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:758
macro expansion at E:\jl\Documenter.jl\test\runtests.jl:8 [inlined]
macro expansion at C:\workdir\usr\share\julia\stdlib\v1.10\Test\src\Test.jl:1504 [inlined]
top-level scope at E:\jl\Documenter.jl\test\runtests.jl:7
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:910
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:853
ijl_toplevel_eval at C:/workdir/src\toplevel.c:919 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:969
eval at .\boot.jl:370 [inlined]
include_string at .\loading.jl:1889
_include at .\loading.jl:1949
include at .\Base.jl:467
jfptr_include_27574.clone_1 at C:\Users\woclass\AppData\Local\julias\julia-latest\lib\julia\sys.dll (unknown line)
exec_options at .\client.jl:307
_start at .\client.jl:541
jfptr__start_57110.clone_1 at C:\Users\woclass\AppData\Local\julias\julia-latest\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1876 [inlined]
true_main at C:/workdir/src\jlapi.c:573
jl_repl_entrypoint at C:/workdir/src\jlapi.c:717
mainCRTStartup at C:/workdir/cli\loader_exe.c:58
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Documenter: Error During Test at E:\jl\Documenter.jl\test\runtests.jl:5
  Got exception outside of a @test
  LoadError: TypeError: in new, expected IOStream, got a value of type Base.PipeEndpoint
  Stacktrace:
    [1] IOContext
      @ .\show.jl:296 [inlined]
    [2] IOContext
      @ .\show.jl:306 [inlined]
    [3] IOContext
      @ .\show.jl:315 [inlined]
    [4] capture(f::var"#14#16"{NamedTuple{(:edit_link,), Tuple{Symbol}}, Tuple{String, String, String}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, String, MathJax2}; rethrow::Type, color::Bool)
      @ IOCapture C:\Users\woclass\.julia\packages\IOCapture\sXY71\src\IOCapture.jl:82
    [5] _quietly(f::Function, expr::Expr, source::LineNumberNode)
      @ Main.TestUtilities E:\jl\Documenter.jl\test\TestUtilities.jl:47
    [6] macro expansion
      @ E:\jl\Documenter.jl\test\TestUtilities.jl:100 [inlined]
    [7] html_doc(build_directory::Any, mathengine::Any; htmlkwargs::Any, image_assets::Any, kwargs::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}})
      @ Main E:\jl\Documenter.jl\test\examples\make.jl:222
    [8] top-level scope
      @ E:\jl\Documenter.jl\test\examples\make.jl:255
    [9] include(fname::String)
      @ Base.MainInclude .\client.jl:478
   [10] macro expansion
      @ E:\jl\Documenter.jl\test\runtests.jl:8 [inlined]
   [11] macro expansion
      @ C:\Users\woclass\AppData\Local\julias\julia-latest\share\julia\stdlib\v1.10\Test\src\Test.jl:1504 [inlined]
   [12] top-level scope
      @ E:\jl\Documenter.jl\test\runtests.jl:7
   [13] include(mod::Module, _path::String)
      @ Base .\Base.jl:467
   [14] exec_options(opts::Base.JLOptions)
      @ Base .\client.jl:307
   [15] _start()
      @ Base .\client.jl:541
  in expression starting at E:\jl\Documenter.jl\test\examples\make.jl:253
Test Summary: | Error  Total  Time
Documenter    |     1      1  3.6s
ERROR: LoadError: Some tests did not pass: 0 passed, 0 failed, 1 errored, 0 broken.
in expression starting at E:\jl\Documenter.jl\test\runtests.jl:5

mortenpi · 2023-02-28T05:00:55Z

Are you sure it's not reproducible with .665? It also failed on CI with 1.10.0-DEV.367. Seems unlikely that there was a bug that got fixed and then re-introduced?

inkydragon · 2023-02-28T12:18:11Z

❌ Version 1.10.0-DEV.667 (2023-02-27) a23b29cb1a
❌ Version 1.10.0-DEV.665 (2023-02-27) 6bf77c0ed8
❌ Version 1.10.0-DEV.600 (2023-02-14) 1541519457
❌ Version 1.10.0-DEV.563 (2023-02-10) 500f561f72
❌ Version 1.10.0-DEV.408 (2023-01-20) ba69cbaa8e
❌ Version 1.10.0-DEV.303 (2023-01-09) d21a25d52c
❌ Version 1.10.0-DEV.226 (2022-12-29) 5c1b99a5c3
❌ Version 1.10.0-DEV.127 (2022-12-12) 4ff707227f
❌ Version 1.10.0-DEV.80 (2022-11-30) f4534d16b4
❌ Version 1.10.0-DEV.75 (2022-11-30) 70bda2cfe4
❌ Version 1.10.0-DEV.73 (2022-11-29) e06a5915a9 ❌
✅ Version 1.10.0-DEV.72 (2022-11-29) 328dd57895 ✅
✅ Version 1.10.0-DEV.68 (2022-11-29) 4fd26ba135
✅ Version 1.10.0-DEV.56 (2022-11-28) b34fe1d695
✅ Version 1.10.0-DEV.34 (2022-11-21) 23160a109d ✅
✅ Version 1.10.0-DEV.0 (2022-11-14) 7fe6b16f40 ✅

git bisect bad
There are only 'skip'ped commits left to test.
The first bad commit could be any of:
cbfdb3facd0f2ece4088f43ef97533e9e0921081
e06a5915a9e93ed5d4c25cf819275af18adf8187
We cannot bisect more!

mortenpi · 2023-02-28T22:07:06Z

It looks like it's this PR then: JuliaLang/julia#44527

But it might be good to see if we can get an MWE out. It looks like the error actually comes from IOCapture.jl.

mortenpi · 2023-02-28T23:55:51Z

I was able to reproduce something with just IOCapture.jl, which I hope will be easier to debug. The error looks a bit different, but I have a hunch it's the same underlying issue. (JuliaLang/julia#48837)

inkydragon · 2023-03-04T05:38:54Z

Version 1.10.0-DEV.699 (2023-03-04) works fine for me.

add a very basic precompile workload

5fd50a4

KristofferC mentioned this pull request Dec 30, 2022

Precompilation #2002

Closed

mortenpi reviewed Dec 30, 2022

View reviewed changes

Comment thread src/docs_precompile/make.jl

Merge branch 'master' into kc/precompile

25d8e92

mortenpi approved these changes Jan 15, 2023

View reviewed changes

mortenpi closed this Feb 28, 2023

mortenpi reopened this Feb 28, 2023

mortenpi mentioned this pull request Feb 28, 2023

SnoopPrecompile segfault regression with IOCapture.jl JuliaLang/julia#48837

Closed

mortenpi closed this Mar 6, 2023

mortenpi reopened this Mar 6, 2023

mortenpi merged commit f0d5cd2 into JuliaDocs:master Mar 6, 2023

KristofferC deleted the kc/precompile branch March 6, 2023 09:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add a very basic precompile workload#2004

add a very basic precompile workload#2004
mortenpi merged 2 commits intoJuliaDocs:masterfrom
KristofferC:kc/precompile

KristofferC commented Dec 30, 2022 •

edited by mortenpi

Loading

Uh oh!

Uh oh!

mortenpi left a comment

Uh oh!

tfiers commented Jan 15, 2023 •

edited

Loading

Uh oh!

mortenpi commented Jan 15, 2023

Uh oh!

mortenpi commented Jan 15, 2023

Uh oh!

inkydragon commented Feb 27, 2023

Uh oh!

mortenpi commented Feb 28, 2023

Uh oh!

inkydragon commented Feb 28, 2023

Uh oh!

mortenpi commented Feb 28, 2023

Uh oh!

inkydragon commented Feb 28, 2023

Uh oh!

mortenpi commented Feb 28, 2023

Uh oh!

mortenpi commented Feb 28, 2023

Uh oh!

inkydragon commented Mar 4, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

KristofferC commented Dec 30, 2022 • edited by mortenpi Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

mortenpi left a comment

Choose a reason for hiding this comment

Uh oh!

tfiers commented Jan 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mortenpi commented Jan 15, 2023

Uh oh!

mortenpi commented Jan 15, 2023

Uh oh!

inkydragon commented Feb 27, 2023

Uh oh!

mortenpi commented Feb 28, 2023

Uh oh!

inkydragon commented Feb 28, 2023

Uh oh!

mortenpi commented Feb 28, 2023

Uh oh!

inkydragon commented Feb 28, 2023

Uh oh!

mortenpi commented Feb 28, 2023

Uh oh!

mortenpi commented Feb 28, 2023

Uh oh!

inkydragon commented Mar 4, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

KristofferC commented Dec 30, 2022 •

edited by mortenpi

Loading

tfiers commented Jan 15, 2023 •

edited

Loading