find extremely slow on CIFS share

Question

Entry in /proc/mounts:

//OUR_ORG.file.core.windows.net/OUR_ORG /data/csv cifs rw,relatime,vers=3.0,sec=ntlmssp,cache=strict,username=OUR_ORG,domain=,uid=0,noforceuid,gid=0,noforcegid,addr=SOME_IP,file_mode=0777,dir_mode=0777,soft,persistenthandles,nounix,serverino,mapposix,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1 0 0

I'm trying to run command:

find /data/csv -mtime -1 -name '*.csv.gz'

Unfortunely, it's super slow.

ls -alh --time-style=+%D /data/csv/

This command runs fine and completes in less than 2 seconds.

I checked strace. find use newfstatat internally, but ls use lstat internally.

I would prefer to use find with some switch rather than parse ls output. Can I somehow make find command reasonably fast?

Answer 1

I had a similar find command that took close to 10mins, while ls command returned in seconds. Cache setting didn't have any effect.

By using parameter actimeo=60 (default = 1 sec), the find command returned in less than 10 seconds.

actimeo=arg

The time (in seconds) that the CIFS client caches attributes of a file or directory before it requests attribute information from a server. During this period the changes that occur on the server remain undetected until the client checks the server again. By default, the attribute cache timeout is set to 1 second. This means more frequent on-the-wire calls to the server to check whether attributes have changed which could impact performance. With this option users can make a tradeoff between performance and cache metadata correctness, depending on workload needs. Shorter timeouts mean better cache coherency, but frequent increased number of calls to the server. Longer timeouts mean a reduced number of calls to the server but looser cache coherency. The actimeo value is a positive integer that can hold values between 0 and a maximum value of 2^30 * HZ (frequency of timer interrupt) setting.

Answer 2

Change the parameters to

cache=loose

With one serious comment from the manual page:

cache=loose can cause data corruption when multiple readers and writers are working on the same files.

---

cache=

Cache mode. See the section below on CACHE COHERENCY for details. Allowed values are:

    • none: do not cache file data at all 

    • strict: follow the CIFS/SMB2 protocol strictly 

    • loose: allow loose caching semantics 

The default in kernels prior to 3.7 was "loose". As of kernel 3.7 the default is "strict".