Evolution does not have a feature for quickly attaching the public key like this. Instead, attaching a public-key is like attaching normal file.
You'll need to locate the public-key in your folder, and attach it to the mail.
Adding Attachments in Evolution
To attach a file to your email in the composer:
Click Add Attachment…, or click Insert ▸ Attachment, or press Ctrl+M.
Select the file you want to attach.
- You can also drag a file to the attachment bar of the composer window.
When you send the message, a copy of the attached file goes with it.