Software problem: Lost network in a Podman container

I use Podman for running various stuff. It’s sometimes challenging due to problems of all categories. One special problem I experienced on one of my computers was that if a container had stopped and then was started again, it lost its network interface, only loopback interface remained available.

I searched the web and hoped it could be related to this bug. So it looked worth trying the newest Podman version. Installing a package from a development version is usually not a thing one wants to do in stable installations of Linux distributions. But on NixOS it should be harmless, so I took the exercise.

I’ve found out it’s quite easy. nixos-unstable channel must be added:

sudo nix-channel --add https://nixos.org/channels/nixos-unstable nixos-unstable
sudo nix-channel --update

And then it must be specified in /etc/nixos/configuration.nix (or wherever systemPackages are defined) and enabled for the given package:

{ config, pkgs, ... }:  
let
  unstable = import <nixos-unstable> { };
in
{
  …
  environment.systemPackages = with pkgs; [
    unstable.podman …
  ];
  …
}

So I got the latest and greatest Podman 3.2.2. Unfortunately it didn’t fix the problem.

Hmm, what now? How to find out what’s happening behind? There was no obvious error in journal and it’s not easy to debug complex tools composed of multiple components.

I could notice that slirp4netns process was running on the first container run but not on the subsequent runs. So it could be still related to the bug above. What if I restart the container with podman restart? The network was still there. OK, so how about running the container in the background and stopping and starting it using podman stop/start? It still worked. Let’s attach to the container using podman attach and exit from there. After starting the container again, the network was gone. I see.

Looks like a bug but how is it possible that nobody has noticed such a relatively noticeable problem? Probably because I couldn’t reproduce the problem on CentOS and Debian, it was present only on NixOS. Searching the web again with more specific keywords didn’t help. But stopping and starting the container again using podman restart brings the network back, which is a sufficient workaround.

I reported the behavior as a NixOS bug. I’m not sure about the software problem category, it may be any of 1, 2, 3. Let’s see what happens with the bug report.