Contacts menu does not work for users with all fields set to "Local"

[nickvergessen]

Same issue happens btw with the current avatar menu.

• As user 1 go to personal settings
• Set all privacy fields to "Local"
• occ dav:sync-system-addressbook
• As user 2 open the avatar menu of user 1
• 💥

Originally posted in #29269 (comment)