I have a kerberized NFS filesystem we're trying to access from an Ubuntu 16.04 client. The server is RHEL7. We can mount it successfully (as root):
mount -t nfs4 -o sec=krb5 colossal.ipa.localdomain:/export/home /mnt
I can run 'df' or 'ls /mnt' as root successfully, but when I try to do the same via a user account ('username@localdomain'), rpc.gssd outputs the error:
ERROR: GSS-API: error in gss_acquire_cred(): GSS_S_FAILURE (Unspecified GSS failure. Minor code may provide more information) - Can't find client principal username@localdomain in cache collection
The full debug output (including KRB5_TRACE=/dev/stdout) from rpc.gssd is:
# rpc.gssd -f -v -v -v -r -r -r Warning: rpcsec_gss library does not support setting debug level beginning poll [...]
handling gssd upcall (/run/rpc_pipefs/nfs/clnt0) handle_gssd_upcall: 'mech=krb5 uid=1388813135 enctypes=18,17,16,23,3,1,2 ' handling krb5 upcall (/run/rpc_pipefs/nfs/clnt0) process_krb5_upcall: service is '<null>'  1501554205.58843: Retrieving username@localdomain from FILE:/etc/krb5/user/0/client.keytab (vno 0, enctype 0) with result: 2/Key table file '/etc/krb5/user/0/client.keytab' not found ERROR: GSS-API: error in gss_acquire_cred(): GSS_S_FAILURE (Unspecified GSS failure. Minor code may provide more information) - Can't find client principal username@localdomain in cache collection getting credentials for client with uid 1388813135 for server colossal.ipa.localdomain WARNING: Failed to create krb5 context for user with uid 1388813135 for server colossal.ipa.localdomain doing error downcall
'username@localdomain' has a Kerberos TGT in LOCALDOMAIN:
$ klist Ticket cache: KEYRING:persistent:1388813135:1388813135 Default principal: username@LOCALDOMAIN Valid starting Expires Service principal 01/08/17 12:23:15 01/08/17 22:23:15 krbtgt/LOCALDOMAIN@LOCALDOMAIN renew until 02/08/17 12:23:13
I'm not clear about why rpc.gssd is trying to read '/etc/krb5/user/0/client.keytab'? The ccache is in a kernel keyring.
I don't know if it's significant, but the user accounts are obtained through sssd/IPA via an AD trust and have a username of 'username@localdomain' in kerberos realm LOCALDOMAIN (the AD domain).
(I'm able to do this successfully on a RHEL7 NFS client)
Any hints as to what the issue might be here?