Can't upgrade Ubuntu 16.04.5 LTS to 18.04.1 LTS because can't upgrade libbluray-bdj

Question

I am unable to upgrade from Ubuntu 16.04.5 LTS to 18.04.1 LTS because one package has not been upgraded (libbluray-bdj). When I try to upgrade it, it can't be upgraded because it depends on a package (libasm-java) that is not upgradable because it has no installation candidate. When I try to remove libbluray-bdj, it tells me it will remove ubuntu-mate-desktop which I don't want to do.

I was performing the upgrade via the terminal because I did a partial upgrade through Software Updater previously (before I ran into these problems), and I don't want to leave my system in a partial uprade state between 16.04.5 and 18.04.1.

I just want to upgrade to 18.04.1. Thanks!

Here are the commands I used and their outputs:

lsb_release -a

No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.5 LTS
Release: 16.04
Codename: xenial

sudo apt update && sudo apt upgrade

1 package can be upgraded. Run 'apt list --upgradable' to see it.

apt list --upgradable -a

libbluray-bdj/xenial,xenial 1:1.0.2-3~xenial all [upgradable from: 1:0.9.2-2]
N: There is 1 additional version. Please use the '-a' switch to see it

sudo apt-get dist-upgrade

0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

sudo apt-get upgrade -y

The following packages have been kept back:
libbluray-bdj
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.

sudo do-release-upgrade

Please install all available updates for your release before upgrading.

sudo apt-get dist-upgrade libbluray-bdj

Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Error!
Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
libbluray-bdj : Depends: libasm-java but it is not installable
E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.

sudo apt-get dist-upgrade libasm-java

Reading package lists... Done
Building dependency tree
Reading state information... Done
Package libasm-java is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or is only available from another source

E: Package 'libasm-java' has no installation candidate

 sudo apt-get remove libbluray-bdj

The following packages will be REMOVED:
libbluray-bdj ubuntu-mate-desktop

and I don't want to remove ubuntu-mate-desktop.

UPDATE:
Here are the contents of /etc/apt/sources.list:

$ cat /etc/apt/sources.list  
# deb cdrom:[Ubuntu-MATE 16.04.2 LTS _Xenial Xerus_ - Release amd64 (20170215)]/ xenial main multiverse restricted universe

# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
deb http://us.archive.ubuntu.com/ubuntu/ xenial main restricted
# deb-src http://us.archive.ubuntu.com/ubuntu/ xenial main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://us.archive.ubuntu.com/ubuntu/ xenial-updates main restricted
# deb-src http://us.archive.ubuntu.com/ubuntu/ xenial-updates main restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://us.archive.ubuntu.com/ubuntu/ xenial universe
# deb-src http://us.archive.ubuntu.com/ubuntu/ xenial universe
deb http://us.archive.ubuntu.com/ubuntu/ xenial-updates universe
# deb-src http://us.archive.ubuntu.com/ubuntu/ xenial-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu 
## team, and may not be under a free licence. Please satisfy yourself as to 
## your rights to use the software. Also, please note that software in 
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://us.archive.ubuntu.com/ubuntu/ xenial multiverse
# deb-src http://us.archive.ubuntu.com/ubuntu/ xenial multiverse
deb http://us.archive.ubuntu.com/ubuntu/ xenial-updates multiverse
# deb-src http://us.archive.ubuntu.com/ubuntu/ xenial-updates multiverse

## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
deb http://us.archive.ubuntu.com/ubuntu/ xenial-backports main restricted universe multiverse
# deb-src http://us.archive.ubuntu.com/ubuntu/ xenial-backports main restricted universe multiverse

## Uncomment the following two lines to add software from Canonical's
## 'partner' repository.
## This software is not part of Ubuntu, but is offered by Canonical and the
## respective vendors as a service to Ubuntu users.
deb http://archive.canonical.com/ubuntu xenial partner
# deb-src http://archive.canonical.com/ubuntu xenial partner

deb http://security.ubuntu.com/ubuntu xenial-security main restricted
# deb-src http://security.ubuntu.com/ubuntu xenial-security main restricted
deb http://security.ubuntu.com/ubuntu xenial-security universe
# deb-src http://security.ubuntu.com/ubuntu xenial-security universe
deb http://security.ubuntu.com/ubuntu xenial-security multiverse
# deb-src http://security.ubuntu.com/ubuntu xenial-security multiverse

Answer 1

Spend enough time really absing Ubuntu and you'll hit dependencies issues like this.

I have a whole boatload of PPAs installed at any given moment so upgrades are often quite fun. That isn't your problem —it looks like there's a missing package in the repo for whatever reason, and I have options like ppa-purge— but my typical approach to fixing is what I'd recommend to you...

1. Remove any problematic packages. Yes, even the ubuntu-mate-desktop meta-package.
2. Do your upgrade but don't do the package clean-up it offers at the end. Not a disaster if you do.
3. Reboot. Hopefully you're still on a desktop but even if you aren't, find your way to the nearest terminal and sudo apt install ubuntu-mate-desktop.
4. If you managed to hold of the stage-2 cleanup, it's time to run sudo apt autoremove to remove automatic packages that aren't required any more.

It's scary, it looks wrong, but packages can be removed and reinstalled and nothing is lost forever. Your desktop config lives in your $HOME and isn't part of the remove. System settings in /etc/ aren't removed by apt remove (that's purge's job). You might end up at a command line. You might need nmtui to get yourself connected at the other side, but it's all relatively low risk.

I think you've identified why this has happened. Some sort of fruity bug in libasm-java, or the packaging of it, that is blocking everything around it. This won't affect you in 18.04 because ubuntu-mate-desktop` no longer depends on it.

---

I'd like to think that this answer is transferable to any instance where you're up against a package conflict but be conscious of what it is you're removing.

In this case it's a crufty old Bluray library and a meta-package for a desktop environment. If you start seeing important things like network-manager or linux-image pop up in the "going to be REMOVED" warning, take a step back and start a new question :D