In my setup with Evince 3.36.7 on Ubuntu 20.04.1 LTS, the Evince viewer's behavior is exactly as follows: if you happen to close the document at the last page, then the next time you open the same document, it will open at the first page.
This is somewhat confusing if users don't know this behavior. This means that, if the document has n pages, Evince will only remember the last page if you close the document at page 2...n-1 and will return to the first page otherwise.
I suppose this makes at least some sense, since you can always go to the first and last page with home and end button on the keyboard.