1

I try to install DataHub together with Docker on a t2.large AWS EC2 Instance running Ubuntu.

I am following a installation guide for Docker and a guide for DataHub. The tests proposed by the guides show me that both tools have been successfully installed.

My Docker installation:

ubuntu@ip-172-31-15-72:~$ sudo docker version
Client: Docker Engine - Community
 Version:           23.0.5
 API version:       1.42
 Go version:        go1.19.8
 Git commit:        bc4487a
 Built:             Wed Apr 26 16:21:07 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          23.0.5
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.19.8
  Git commit:       94d3ad6
  Built:            Wed Apr 26 16:21:07 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

My DataHub installation:

ubuntu@ip-172-31-15-72:~$ datahub version
DataHub CLI version: 0.10.2.2
Python version: 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0]

The result if I try to use the command to start DataHub in Docker:

ubuntu@ip-172-31-15-72:~$ python3 -m datahub docker quickstart
[2023-05-12 06:27:11,109] INFO     {datahub.cli.quickstart_versioning:144} - Saved quickstart config to /home/ubuntu/.datahub/quickstart/quickstart_version_mapping.yaml.
[2023-05-12 06:27:11,109] INFO     {datahub.cli.docker_cli:643} - Using quickstart plan: composefile_git_ref='master' docker_tag='head'
Docker doesn't seem to be running. Did you start it?

My problem is that DataHub seems to not be able to use Docker. Can you please tell me how I can make DataHub find Docker?

I suspect that the quickstart command is searching for Docker in a location where Docker is not installed. I try to find and read through the code of the quickstart command in order to find the location it searches. Then I can compare that with the location where Docker is installed. However this will consume a lot of time and since this question is probably trivial and done in 5 min for a Linux Pro I humbly ask for your help.

Result of systemctl status docker:

ubuntu@ip-172-31-15-72:~$ systemctl status docker
● docker.service - Docker Application Container Engine
 Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
 Active: active (running) since Fri 2023-05-12 06:22:58 UTC; 46min ago
TriggeredBy: ● docker.socket
   Docs: https://docs.docker.com
   Main PID: 639 (dockerd)
      Tasks: 10
     Memory: 102.2M
        CPU: 764ms
     CGroup: /system.slice/docker.service
         └─639 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
8
  • You're using a development version of Docker - can this be the issue? May 12, 2023 at 6:59
  • 1
    Also what does systemctl status docker reveal? May 12, 2023 at 7:06
  • @ArturMeinlid I added the result of systemctl status docker. I am still working on downgrading to another Docker version. May 12, 2023 at 7:13
  • @ArturMeinild sorry, spelled your username wrong. May 12, 2023 at 7:23
  • 1
    @ArturMeinild Thank you, that at least confirms that it is a good idea to search the code of DataHub for how it tries to use Docker. May 12, 2023 at 8:12

1 Answer 1

1

The Slack community from DataHub found the solution.

The datahub docker quickstart command can only work if Docker can be used without sudo.

Therefore it is necessary to follow this guide. It shows how you can create a usergroup for Docker. If Docker has a usergroup the sudo command is no longer neede.

https://docs.docker.com/engine/install/linux-postinstall/

If somebody stumbles over this post who has deeper knowledge about why the usergroup is necessary a link or a short explanation would be much appreciated ;)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .