Here is the answer that worked for me, after Hamish helped me to realize my typo.
WARNING (for older Ubuntu versions, newer (e.g. 19.04) should be bug-fixed but be careful anyway): If you only have one key and remove it before adding another, you will render your disk inaccessible after rebooting! This also means you can not add a new key anymore afterwards. Thanks waffl and khaimovmr for these helpful comments.
First, you need to find out which is the encrypted LVM partition, it may be sda3, but it can also be sda5 (default on Ubuntu LVM), sdX2, ...:
To add a new password, use
sudo cryptsetup luksAddKey /dev/sda3
To remove an existing password, use
sudo cryptsetup luksRemoveKey /dev/sda3
View currently used slots of the encrypted partition:
sudo cryptsetup luksDump /dev/sda3
Cited from this blog. Thanks.
In Ubuntu 18.04 there is another possibility, using (Gnome) Disks. Thanks for the hint, Greg Lever, after clicking around I found what Greg mentioned:
1. Open Gnome Disks.
2. Choose/Click on the main physical hard drive in the left panel.
3. Click on the LUKS encrypted partition, in this example it is Partition 3:
4. Click on the edit icon (cogs, gear wheels) and choose "Change Paraphrase".