Git generator does not support submodules

[iben12]

Checklist:

• I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
• I've included steps to reproduce the bug.
• I've pasted the output of argocd version.

Describe the bug

When using Git generator and the the repo contains submodules, the generator does not find files in submodules.

While preparing the repo it uses git submodule sync --recursive and git submodule update --init --recursive which fetches the actual submodule contents, but the file search for the generator uses git ls-files --full-name -z -- **/config.json, which does not search in submodules. The correct command would be git ls-files --full-name -z --recurse-submodules -- **/config.json.

As I researched how it works, I found the Git client util command here:

argo-cd/util/git/client.go

Line 366 in b38ff17

func (m *nativeGitClient) LsFiles(path string) ([]string, error) {

The function does not care if the submodule env is set or not, it does not search for files in submodules.

To Reproduce

Example repo structure:

├── apps
│   └── app-a (submodule
│       └── config.json
│   └── app-b (submodule
│       └── config.json
└── git-generator-files.yaml

Generator:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: submodule-generator
  namespace: argocd
spec:
  generators:
    - git:
        revision: HEAD
        repoURL: git@github.com:user/configs.git
        files:
          - path: "**/config.json"
        template:
          ...

See that the generator does not generate any applications.

Expected behavior

ApplicatioSet Controller finds files matching the pattern in Git submodlues.

Screenshots

Version

argocd: v2.6.3+e05298b
  BuildDate: 2023-02-27T14:40:19Z
  GitCommit: e05298b9c6ab8610104271fa8491f019fee3c587
  GitTreeState: clean
  GoVersion: go1.18.10
  Compiler: gc
  Platform: linux/amd64

Logs

time="2023-04-19T07:53:56Z" level=info msg="git checkout --force 6a9be03e97f4010dec9de29213fe4bde321baf03" dir=/tmp/git@github.com_********* execID=6e70b
time="2023-04-19T07:53:56Z" level=info msg=Trace args="[git checkout --force 6a9be03e97f4010dec9de29213fe4bde321baf03]" dir=/tmp/git@github.com_********* operation_name="exec git" time_ms=6.21754
time="2023-04-19T07:53:56Z" level=info msg="git submodule sync --recursive" dir=/tmp/git@github.com_******* execID=b37c5
time="2023-04-19T07:53:56Z" level=info msg=Trace args="[git submodule sync --recursive]" dir=/tmp/git@github.com_********* operation_name="exec git" time_ms=36.418946
time="2023-04-19T07:53:56Z" level=info msg="git submodule update --init --recursive" dir=/tmp/git@github.com_********* execID=96efd
time="2023-04-19T07:53:57Z" level=info msg=Trace args="[git submodule update --init --recursive]" dir=/tmp/git@github.com_********* operation_name="exec git" time_ms=1406.008223
time="2023-04-19T07:53:57Z" level=info msg="git clean -fdx" dir=/tmp/git@github.com_********* execID=7dce8
time="2023-04-19T07:53:57Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/git@github.com_********** operation_name="exec git" time_ms=2.634399
time="2023-04-19T07:53:57Z" level=info msg="git ls-files --full-name -z -- ./**/config.json" dir=/tmp/git@github.com_******** execID=bd0ff
time="2023-04-19T07:53:57Z" level=info msg=Trace args="[git ls-files --full-name -z -- ./**/config-gap-infra.json]" dir=/tmp/git@github.com_******* operation_name="exec git" time_ms=5.545562
time="2023-04-19T07:53:57Z" level=info msg="generated 0 applications" generator=..."

[Calchan]

I have made changes to #13314 which should fix this issue. Can anyone please test and confirm?