v. 12.14.0: failure to recognize grammar types

[empwilli]

With v. 12.14.0 I am not able to build grammars that are split across multiple files (in my case separate lexer/parser grammars). It still works for v. 12.12, though. I suspect that there is some issue with the detection mechanism for the respective grammar type. (I would have opened individual issues, but I feel that all of them are cause of the same underlying bug. I hope this is ok.)

edit: for the desperate, I think there is a workaround for 12.14 below.

As far as I see all of the given examples work on 12.12

Separate Lexer/Grammar Rules w/ wildcard in .csproj

lexer grammer Lex;

CHAR: [0-9];

parser grammer Parser;

options {
    tokenVocab = Lex;
}

word: CHAR+;

and in my .csproj:

<Antlr4 Include="Grammar\*.g4">
      <Visitor>true</Visitor>
      <Listener>false</Listener>
      <Package>Package</Package>
      <Error>true</Error>
</Antlr4>

With v.12.14 building results in the following errors. (I deliberately clean up any build artifacts to be on the safe side):

> dotnet clean && rm -rf bin obj && dotnet build

Parser.g4(7,6): warning ANT01: warning(125): ERE.g4:7:5: implicit definition of token OPT in parser [Project.csproj]
Parser.g4(4,15): error ANT02: error(114): ERE.g4:4:14: cannot find tokens file Grammar/Lex.tokens [Project.csproj]
/home/some_user/.nuget/packages/antlr4buildtasks/12.14.0/build/Antlr4BuildTasks.targets(142,9): error ANT02: error(10):  warning treated as error [Project.csproj]
CSC : error CS2001: Source file 'Project/obj/Debug/net8.0/Parser.cs' could not be found. [Project.csproj]
CSC : error CS2001: Source file 'Project/obj/Debug/net8.0/ParserBaseVisitor.cs' could not be found. [Project.csproj]
CSC : error CS2001: Source file 'Project/obj/Debug/net8.0/LexBaseVisitor.cs' could not be found. [Project.csproj]
CSC : error CS2001: Source file 'Project/obj/Debug/net8.0/LexVisitor.cs' could not be found. [Project.csproj]
CSC : error CS2001: Source file 'Project/obj/Debug/net8.0/ParserVisitor.cs' could not be found. [Project.csproj])

Separate Lexer/Grammar Rules explicitly listed in .csproj

If, however, I list the build dependencies explicitly in my .csproj:

<Antrl4 Include="Grammar\Lex.g4" />
<Antlr4 Include="Grammar\Parser.g4">
      <Visitor>true</Visitor>
      <Listener>false</Listener>
      <Package>Package</Package>
      <Error>true</Error>
</Antlr4>

I get the following results:

bash 
> dotnet clean && rm -rf bin obj && dotnet build

CSC : error CS2001: Source file 'Project/obj/Debug/net8.0/LexBaseListener.cs' could not be found. [Project.csproj]
CSC : error CS2001: Source file 'Project/obj/Debug/net8.0/LexBaseVisitor.cs' could not be found. [Project.csproj]
CSC : error CS2001: Source file 'Project/obj/Debug/net8.0/LexListener.cs' could not be found. [Project.csproj]
CSC : error CS2001: Source file 'Project/obj/Debug/net8.0/LexVisitor.cs' could not be found. [Project.csproj])

No "grammar" modifier in parser rules

Finally, when adjusting the parser grammar by dropping the "grammar" modifier:

grammer Parser;

options {
    tokenVocab = Lex;
}

word: CHAR+;

The build fails with the following:

bash 
> dotnet clean && rm -rf bin obj && dotnet build

...
CSC : error CS2001: Source file 'Project/obj/Debug/net8.0/ParserLexer.cs' could not be found. [Project.csproj])

I.e., it attempts to create a lexer for the grammer, even as there are no lexer rules (and hence no ParserLexer.cs is created).

Workaround

If I understand the issue correctly, as of 12.14, to get separate parser/lexer grammars to work (which is necessary for some setups, e.g. lexer modes) do the following:

1. Prefix your lexer/parser grammars with the corresponding modifiers, i.e.:

lexer grammar Lex;
...

parser grammar Parser;

2. List them with the correct flags in your .csproj, i.e., this means: don't generate visitors/listeners for your lexer grammar:

<Antlr4 Include="Grammar\Lex.g4">
      <Visitor>false</Visitor>
      <Listener>false</Listener>
      <Package>Package</Package>
      <Error>true</Error>
</Antlr4>

<Antlr4 Include="Grammar\Parser.g4">
      <Visitor>true</Visitor>
      <Listener>false</Listener>
      <Package>Package</Package>
      <Error>true</Error>
</Antlr4>