What is AppStream? Why is it installed? What is it doing?

Question

There's another question about AppStream that has —for better or worse— focussed on what seems to be a particular bug in AppStream but that left me realising I don't have a clue what this thing actually is.

man appstreamcli (the application taking the CPU off the deep end) throws some technical jargon at the confusion:

appstreamcli is a small helper tool to work with AppStream metadata and access the AppStream component index from the command-line. The AppStream component index contains a list of all available software components for your distribution, matched to their package names. It is generated using AppStream XML or Debian DEP-11 data, which is provided by your distributor.

So... What is it actually there for? Do all desktop (and their related package managers) use it?

By all of which I actually mean: what will break when I tear this out with my teeth?

Answer 1

The AppStream project page adds a little more jargon but does go onto say:

It provides the foundation to build software-center applications, by providing metadata necessary for an application-centric view on package repositories. AppStream additionally provides specifications for things like an unified software metadata database, screenshot services and various other things needed to create user-friendly application-centers for (Linux) distributions.

So this is basically what the new software centres are using to get their data from, rather than directly from Apt. appstreamcli refresh (the thing people are freaking out about using all the CPU) is the process that runs after an apt update to regenerate its own data. As mentioned in the question, there appears to be a bug here.

You can see what needs it based on the reverse dependencies:

$ apt rdepends appstream
appstream
Reverse Depends:
  Breaks: libapt-pkg5.0 (<< 0.9.0-3~)
  Depends: plasma-discover
  Depends: plasma-discover
  Depends: gnome-software
  Depends: plasma-discover
  Depends: isenkram-cli
  Recommends: check-all-the-things
  Depends: appstream-index (>= 0.9.4-1)
  Recommends: libappstreamqt1 (>= 0.9.4-1)
  Suggests: appstream-doc
  Depends: gnome-software

So at a base level to answer "What will break [...]?", ^^ that stuff.

Both Gnome's and KDE's software tools depend on it today. More may in the future. Most of these indirect package manager are just "recommends" so if you only ever use apt or tools that only use Apt's package library directly (too many to list), you can get rid of it without removing the whole desktop environment.

You can simulate a removal with apt -s remove appstream:

$ apt -s remove appstream
NOTE: This is only a simulation!
      apt-get needs root privileges for real execution.
      Keep also in mind that locking is deactivated,
      so don't depend on the relevance to the real current situation!
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  linux-headers-4.5-2.dmz.4-liquorix-amd64 linux-headers-4.5-3.dmz.1-liquorix-amd64 linux-image-4.5-2.dmz.4-liquorix-amd64
  linux-image-4.5-3.dmz.1-liquorix-amd64
Use 'apt autoremove' to remove them.
The following packages will be REMOVED
  appstream muon-discover plasma-discover
0 to upgrade, 0 to newly install, 3 to remove and 80 not to upgrade.
Remv muon-discover [4:5.6.4+p16.04+git20160517.1518-0]
Remv plasma-discover [5.6.4+p16.04+git20160517.1518-0]
Remv appstream [0.9.4-1]

This is under KDE so don't assume anything from that. It seems safe enough.

---

Update:

Running apt -s remove appstream in Kubuntu 18.04 is slightly different:

Remv plasma-discover [5.12.6-0ubuntu0.1]
Remv apt-config-icons-large-hidpi [0.12.0-3ubuntu1]
Remv apt-config-icons-large [0.12.0-3ubuntu1]
Remv apt-config-icons [0.12.0-3ubuntu1] [apt-config-icons-hidpi:amd64 ]
Remv appstream [0.12.0-3ubuntu1] [apt-config-icons-hidpi:amd64 ]
Remv apt-config-icons-hidpi [0.12.0-3ubuntu1]

Answer 2

I have added DefaultEnabled "false"; in section deb::DEP-11 (see /etc/apt/apt.conf.d/50appstream) to avoid the download of the DEP-11 metadata.

To disable all unwanted download of DEP-11 metadata and especially icons I have set explicitly or changed to DefaultEnabled "false"; (see 60apper and 60icons* files and deb::DEP-11-icons* sections).

The appstream cache is refreshed and stored here with 19MiB of total size on my system:

root# LC_ALL=C du -s -h /var/cache/app-info
19M     /var/cache/app-info

UPDATE: Some numbers of today's apt-get update: Reduced download size from normally approx. 50MiB down to 1,5MiB.

UPDATE-2: Shows all DEP-11 APT configs: root# apt-config dump | grep DEP-11

Note-1: Involved are the APT configs of the packages appstream, apper and apt-config-icons. Note-2: I do not use GUI based package-managers like plasma-discover and gnome-software to upgrade software which require a refresh of the DEP-11 metadata and icons.