Tighter rules for referencing

[barucden]

This is an attempt to clarify the matching criteria for cross-references. I am not very familiar with Documenter's code base, so I am sure the code could be improved, but I am submitting the PR as it is to see if there's broader support for it.

Essentially, the goal is to respect the following rules for references:

Reference type	Implicit link	Explicit link
Header	[Header name](@ref)	[link](@ref "Header name")
Issue	[#12345](@ref)	[link](@ref #12345)
Docstring	[`Module.func`](@ref)	[link](@ref Module.func)

It means that linking [Nonexistent header](@ref) will be recognized as a link to a header named "Nonexistent header", and never as link to a docstring for Nonexistent-header (ref. #2677).

If this direction seems promising, I'd like to get feedback on how to improve the code, especially linkcontent.

---

Resolves #2677
Resolves JuliaLang/julia#58073

[goerz]

I think that accurately describes the current situation.

It's a little bit unfortunate that the explicit link for a DocString is [link](@ref Module.func), not [link](@ref `Module.func`). That's always going to be an ambiguity, since the way this is implemented is for Documenter to look up targets in internal indices, and the fall through to the next resolver when that doesn't work. We can't get rid of the ambiguous syntax for backwards-compatibility reasons, but maybe at some point we can also add support for the non-ambiguous [link](@ref `Module.func`) and slowly push people to use that instead. That's what I did in DocumenterInterLinks.

[barucden]

It's a little bit unfortunate that the explicit link for a DocString is [link](@ref Module.func), not [link](@ref `Module.func`).

Yeah, I thought that too. However, I don't think it's ambiguous if we require that explicit header references are inside double quotes (it is supported at least:

Documenter.jl/src/cross_references.jl

Line 298 in 9bec14d

# explicit slugs that are enclosed in quotes must be further sluggified

). That is for implicit references:

• reference is inside backticks => code
• reference starts with a # => issue
• otherwise => header

And for explicit:

• reference is inside double quotes => header
• reference starts with a # => issue
• otherwise => code

The conditions are mutually exclusive this way.

[barucden]

I made a little progress, but I realized that the rules are indeed ambiguous due to referencing headers by their id:

[link](@ref someid)

[# Header](@id someid)

```@docs
someid
```

[mortenpi]

If there is an @id reference that is the same as a docstring, it should take precedence: https://documenter.juliadocs.org/dev/man/syntax/#Label-precedence-403457a5

Otherwise, the rules in #2678 (comment) seem correct to me! As far as I can tell, this matches what we have documented, even if the implementation doesn't doesn't match. I'm actually surprised that the current implementation seems to be more.. uhm.. loose.. than I though it would be.

What would be nice is to add some tests here (if feasible.. it can be tricky sometimes for this kind of stuff), and also clarify the docs (e.g. the table from the OP would be great to add to the docs probably).

What might be good is to build the Julia manual with this PR before merging, just to see if any warnings crop up.

[barucden]

Thank you for your input, Morten.

If there is an @id reference that is the same as a docstring, it should take precedence: https://documenter.juliadocs.org/dev/man/syntax/#Label-precedence-403457a5

I see. The problem is though when the user makes a mistake in the id:

See [link](@ref header-wrong)
# [Header](@id header-correct)

In this case, header-wrong is not recognized as a header id and thus is considered as code, in which case it searches for the docs of Base.:(-).

---

Implicit references are unambiguous

Type	Source	Reference
Header	# Header name	[Header name](@ref)
Issue	(none)	[#12345](@ref)
Code	```@docs Module.function ```	[`Module.function`](@ref)

That means

if startswith(text, "#")
    # process as an issue reference
elseif occursin(r"^`.+`$", text)
    # process as a code reference
else
    # process as a header reference
end

For explicit references, we can allow and prefer backticks when referring to docstrings, as Michael pointed out. Then, the explicit references are unambiguous too:

Type	Source	Reference
Header	# Header name	[link](@ref "Header name")
Header (via id)	# [Header name](@id header-id)	[link](@ref header-id)
Issue	(none)	[link](@ref #12345)
Code	```@docs Module.function ```	[link](@ref `Module.function`)

leading to the decision

if occursin(r"^\".+\"$", link)
    # process as a header reference
elseif startswith(link, "#")
    # process as an issue reference
elseif occursin(r"^`.+`$", link)
    # process as a code reference
else
    # process as a header reference via id
end

The problem is backwards compatibility of code references. Right now, [link](@ref something) can mean both

• reference to a header with id something
• reference to a docstring of something

in this order. So, we can extend the decision algorithm

if occursin(r"^\".+\"$", link)
    # process as a header reference
elseif startswith(link, "#")
    # process as an issue reference
elseif occursin(r"^`.+`$", link)
    # process as a code reference
else
    # process as a header reference via id if there is such a header
    # and otherwise, process as a code reference
end

However, there's that problem with header-wrong leading to the reference to Base.:(-). It really only matters in julia's docs, but those docs are Documenter's prominent user, so I think it warrants adding a little special case:

if occursin(r"^\".+\"$", link)
    # process as a header reference
elseif startswith(link, "#")
    # process as an issue reference
elseif occursin(r"^`.+`$", link)
    # process as a code reference
else
    if link consists of multiple dash-delimited words
        # process as a header reference via id
    else
        # process as a header reference via id if there is such a header
        # and otherwise, process as a code reference
    end
end

I don't think that many people try to use the explicit reference [link](@ref var1-var2] to refer to Base.:(-), so it shouldn't break any documentation. But if it does break any, it is easily fixable by adding backticks [link](@ref `var1-var2`]. On the other hand, all the accidentally wrong references [link](@ref header-wrong] in julia's docs will lead to a warning.

After some time, when breaking change is permitted, we can just drop this complicated logic in favor of the simple one above.

How does that sound?

[fingolfin]

Just discovered this PR.

@barucden I think your plan sounds good!

And I am not worried about someone trying to use [link](@ref var1-var2]. If despite all expectations someone actually does that, I won't feel bad to tell to switch to [link](@ref Base.:(-)] or something like that.

So if you are still willing to work on this, that'd be awesome! I see that in the meantime, a merge conflict has crept in -- hopefully it is not too bad.

[barucden]

I completely forgot about this PR, thank you for reminding me, Max.

I haven't finished the work even locally, and I am currently busy. If you or anyone else wants to push this forward, please feel free.

[fingolfin]

I've rebased it (resolving the merge conflict) and then asked Codex to complete the work, based on the nice plan sketched by @barucden above. It also tweaked the documentation to explicitly explain the new behavior. It introduced a new helper classifyxref which seems smart to me.

Overall I am happy with the code it produced; it also added tests.

But I'll wait with merging this to give @barucden a chance to react. Also perhaps @goerz or @mortenpi would like to have a look?

[barucden]

Oh wow, it looks exactly how I was going to implement it. Thank you for taking over, Max!

[goerz]

Love it!

(except for the tiny grammar nitpick)