Unfortunately, no; last access time is the only thing that can help you and it's not recorded by default on FAT filesystems (if you try with
stat, you'll find a semi-fake number, based on last mount I think):
θ64° [romano:/media/romano/PEN8G] % stat present.pdf
Size: 291235 Blocks: 576 IO Block: 4096 regular file
Device: 821h/2081d Inode: 1599 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 1153/ romano) Gid: ( 1001/ romano)
Access: 2016-04-22 00:00:00.000000000 +0200
Modify: 2016-03-08 10:31:56.000000000 +0100
Change: 2016-03-10 11:05:26.000000000 +0100
Seems that access has a 1-day granularity.
And no, there is no log of which device accessed your pen drive.
Anyhow, anyone that has copied your files can also have faked the last access time or whatever, so no, there is no way to safely say if the files have been read, copied or whatever.
Encryption is the only safe way to store things on pen drives.