DefaultUserHome: Local functionality

[erpadmin]

• AWS ParallelCluster version
  3.14.2
• Full cluster configuration without any credentials or personal data.
• Cluster name:
  rocky-pcluster

Bug description and how to reproduce:
DefaultUserHome: Local is described as: When set to Local, the head node, login nodes, and compute nodes will each have a separate local default user directory stored in local/home/.

This is true but /home is still being exported to the compute nodes and they mount it automatically. I need to use my own native NFS mount for /home on the headnode and all worker nodes via customization.

Can I disable the /home functionality and replace it like described? With enough finaggling I can probably get it working but I'd like the understand the intention of DefaultUserHome: Local. Side note: it is useful to have rocky user's home in local so i can't easily break connectivity via that user while messing with . /home so that is definitely one benefit

informational:
headnode

/home 10.95.0.0/21(rw,sync,no_root_squash)
/opt/parallelcluster/shared 10.95.0.0/21(rw,sync,no_root_squash)
/opt/parallelcluster/shared_login_nodes 10.95.0.0/21(rw,sync,no_root_squash)
/opt/intel 10.95.0.0/21(rw,sync,no_root_squash)
/mnt/ebs 10.95.0.0/21(rw,sync,no_root_squash)
/opt/slurm 10.95.0.0/21(rw,sync,no_root_squash)

$ find /local
/local
/local/home
/local/home/rocky
/local/home/rocky/.ssh
/local/home/rocky/.ssh/authorized_keys
/local/home/rocky/.ssh/id_ed25519
/local/home/rocky/.ssh/id_ed25519.pub
/local/home/rocky/.ssh/authorized_keys_cluster
/local/home/rocky/.ssh/known_hosts
/local/home/rocky/.bash_logout
/local/home/rocky/.bash_profile
/local/home/rocky/.bashrc
/local/home/rocky/.bash_history

compute

$ grep home /etc/fstab
10.95.1.129:/home /home nfs hard,_netdev,noatime 0 0

[gmarciani]

Hello @erpadmin ,

On DefaultUserHome

what you are observing is the expected behavior.
The property DefaultUserHome refers only to the home directory of the default user (rocky in your case).
So, when DefaultUserHome: Local, the home for the default cluster user is /local/home/<user>. . For any other user the home directory remains the default home directory configured in your system, which by default is /home/<user>. foir this reason /home is still shared with each cluster node.

Custom home mount

Since ParallelCluster 3.8.0, you can mount your external file system to/home using SharedStorage.
Here are some important considerations on this use case: https://docs.aws.amazon.com/parallelcluster/latest/ug/shared-storage-working-considerations-v3.html.

I hope this helps.
Let me know if there is any other clarification you may need.

Thank you,
Giacomo

[erpadmin]

Then I believe the real problem is that SharedStorage does not support regular NFS mounts. For now I am using /nfshome and overriding the user's default home in sssd.conf.