Navigating Docker like a Wizard

I have been using Docker for many years now. I still remember the times before Docker and having to manually set up servers with all the dependencies needed to run my applications.

Of course, there are ways to provision servers using scripts such as Puppet or Ansible, but these can either go wrong or if you aren't careful you can end up installing the wrong versions for things.

Using Docker images fixes a lot of these issues which is why they have become so popular. I even use Docker images for running software on my home server.

For those lucky enough to not know a time before Docker, it is easy to take it for granted. Not only that but many developers I have met treat Docker as an unknown black box.

When your application is failing to load though you haven't got any choices but to dig into your Docker container and poke around until you can find out what is causing the problem.

This is going to be a mini guide to solving issues with your Docker containers which will hopefully be useful.

I am going to assume for all these examples that your Docker container is running some flavour of Linux.

See what is running #

For this step I am going to assume you have your Docker containers running somewhere that you can at least ssh into.

Running docker ps will give you a list of all your running containers and how long they have been running for.

docker ps results

If you have your service set up to automatically restart and your docker container has a problem then you will likely see something along the lines of Restarting (1) 2 seconds ago or Up less than a second.

If you can't see your container at all then try running docker ps -a.

docker ps -a results

You can see here my Jellyfin container stopped 3 weeks ago, and it wasn't set up to restart.

Check the logs #

If your container is running or constantly restarting, as always, the first place to check is the logs.

You can check the logs of any running or stopped container by running docker logs followed by the container ID. You don't need to specify the full ID just the first few characters so that it is unique.

So to look at the logs for my caddy image I can run:

docker logs 502

Gaining access to the container #

If the logs aren't overly helpful then it is time to log into the container and see what is causing the issue.

If the container is running then you can use the following command to gain access to a shell that you can use. Again use the Container ID found from docker ps.

docker exec -it 502 /bin/bash

In some cases, bash isn't available on the version of Linux the container is using. If this is the case you will get a message such as OCI runtime exec failed: exec failed: unable to start container process: exec: "/bin/bash": stat /bin/bash: no such file or directory: unknown

Not to fear we can use sh instead:

docker exec -it 502 /bin/sh

Once you have access to the container you can run any command you would normally on a Linux system to find clues as to what might be causing the problem.

Typical things I usually try:

Find the log files for your application or the system in /var/log. You can use cat or tail to quickly view them.
Try starting your application inside the docker container if it isn't already running.
In the case of connectivity issues, try using ping or curl to see if you can contact other services outside the container. Sometimes I just try ping google.com to make sure it has internet access.

Installing diagnostic tools #

For security and to keep Docker images small, the number of packages installed is kept to a minimum.

This is unfortunately not very helpful if you are trying to diagnose why your service can't connect to something, and you don't have access to basic commands such as curl or ping.

Luckily you can still install packages in the same way you would any other Linux system. How you install packages will depend on what Linux version you are using. You can usually find out by running the following command:

cat /etc/os-release

Which should give you an output like this:

NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.18.3
PRETTY_NAME="Alpine Linux v3.18"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues"

Then it is just a case of looking at how to install packages in that Linux Distro.

In most cases, you need to be running as root in order to install packages. If the commands aren't working try exiting the container (just run exit) and then use the following command to login as root.

docker exec -it -u="root" 502 /bin/sh

There are various ways to install packages depending on your Linux version. If you don't want to look up which version just try one of these and see which works. For example if you are trying to install curl one of the following should work.

apt update
apt install curl

apk update
apk add curl

yum install curl

dnf install curl

Gaining access to a stopped container #

If the container you are trying to diagnose doesn't start up then it is a little bit more work to get it started.

Again you need to find the stopped container ID using:

docker ps -a

You then need to commit the container to a new image. It doesn't need to be called test/image:

docker commit 0488e172aa70 test/image

Then you can try running it:

docker run -it --rm test/image /bin/bash

If your container is failing to run because the entrypoint script is broken you can replace the script with just a /bin/bash or /bin/sh with the following command.

docker run -it --rm --entrypoint /bin/bash test/image

Copying files #

You have done some digging, and you have found a log file that you want to look at more closely, but you want it on your local machine.

First note the location of the file inside the container e.g. /var/log/dmesg

Then exit the container and run the following replacing the ID and file with your own.

docker cp 0c7b2063b2a2:/var/log/dmesg dmesg

You can even copy whole folders if you need to:

docker cp 0c7b2063b2a2:/var/log/. log

If you need to copy files on to the container for some reason we can do that by switching the parameters around:

docker cp dmesg 0c7b2063b2a2:/var/log/dmesg

Hopefully this mini tutorial has been helpful, if it has then let me know in the comments.

❤️ Picks of the Week #

🛠️ Tool - localsend/localsend: An open source cross-platform alternative to AirDrop. One of the reasons I use Apple products is the compatibility between all the devices. I use AirDrop a lot for transferring photos from my phone to my computer. There is now an open-source alternative that looks excellent.

📝 Article - Write more "useless" software. It is a common pattern I see amongst developers. We got into programming because we enjoyed it when we were younger. Professional programming however comes with a lot of baggage whether it be unit tests, CI/CD pipelines, issue tracking or vulnerability checks. If you make something for yourself it doesn't need any of that. Just make something for fun for a change.

🛠️ Tool - FediDevs | List of software developers on Mastodon. Twitter/X is turning into a cesspit of toxicity. That is why I am trying to encourage more people to join the Fediverse. When you first join it can be a bit daunting to find like-minded people, but this is a great too if you are looking for fellow developers to follow.

📝 Article - Hacker leaks millions more 23andMe user records on cybercrime forum. The moral of the story, giving your data to anyone is a terrible idea.

🛠️ Tool - Two Stop Bits. If you are interested in retro computing and gaming then this looks cool.

📝 Article - Relative Time Labels. If you are a designer or a frontend developer it is worth reading this. One of my pet peeves with YouTube is the fact the dates say 1 month ago. If the date isn't recent then just put the date like a normal person.

🛠️ Tool - Piped: An alternative privacy-friendly YouTube frontend which is efficient by design. YouTube have started forcing users who are using ad blockers to either disable them or pay for YouTube premium. If you don't want to see the ads then Piped is one way that you can bypass them.

On a side note, I am thinking of creating a PeerTube instance to host my own videos, so people can watch them without the ads.

📝 Article - Organizing multiple Git identities. If you are a freelancer you may have multiple git accounts that you use. If you need to switch between them, it can be a pain, but this post has a great solution for it.

👨‍💻 Latest from me #

As I mentioned last week, I am currently working on a course which is taking up most of my time at the moment. I do have a couple of announcements though.

My blog is now on the Fediverse #

I am fully embracing the Fediverse everywhere I can. I am not a big fan of feeding algorithms whose sole purpose is to make money from adverts.

Even though I have fewer followers on Mastodon then I did on Twitter, I get much higher engagement on my posts. At the end of the day, that is what really matters, connecting to real people and making a difference.

I have now set up a special @blog account for my website. If you are on Mastodon and want to keep up with my posts on there you can follow @blog@alexhyett.com.

If you have had any issues following me on Mastodon (@alex@alexhyett.com), I did fix an issue with my server that should resolve that now.

Moving to Patreon Chat #

One of the benefits of subscribing to my Patreon is getting to chat to me (or at least I think it is a benefit 😉).

I have been using Discord for this but having a Discord community with so few people is just sad. Luckily Patreon have released Patreon Chat this week, so I am switching to that instead.

If you are a patron then you can use the chat feature now from the mobile app.

💬 Quote of the Week #

Start placing a higher value on your time: Time is more valuable than money. You can always get more money, but you can never get more time.

From 6 Months to 6 Figures (affiliate link) by Peter Voogd.