1

I have an Ubuntu box "k3" attached to a physical network 192.168.86.0/24 and a tplink deco router on 192.168.86.1. k3's IP is 192.168.86.53.

On the ubuntu box, I created a ipvlan l3 network:

podman network create -d ipvlan --subnet 192.168.3.0/24 --opt mode=l3 ipvlan

A container is deployed on 192.168.3.3. The following communication are successful:

  • podman host to container
  • another host to container (after adding static route for 192.168.3.0/24)
  • container to host network

But the container cannot talk to the Internet. Tried pinging 1.1.1.1 & 8.8.8.8 and there was no response. A static route for 192.168.3.0/24 is already added to my deco router. Packet capture on the host network shows there are no reply.

root@k3:~# tcpdump  -nn -v icmp -i enp1s0 -c3
tcpdump: listening on enp1s0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
14:42:16.851045 IP (tos 0x0, ttl 64, id 36288, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.3.3 > 1.1.1.1: ICMP echo request, id 10, seq 130, length 64
14:42:17.851604 IP (tos 0x0, ttl 64, id 36463, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.3.3 > 1.1.1.1: ICMP echo request, id 10, seq 131, length 64
14:42:18.852261 IP (tos 0x0, ttl 64, id 36506, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.3.3 > 1.1.1.1: ICMP echo request, id 10, seq 132, length 64
3 packets captured
3 packets received by filter
0 packets dropped by kernel

I also tried setting up NAT on the podman host. net.ipv4.ip_forward is turned on and the FORWARD chain allows all traffic.

iptables -t nat -A POSTROUTING -s 192.168.3.0/24 -j MASQUERADE

Packet capture now showed the reply, but they did not actually make it into the container.

root@k3:~# tcpdump  -nn -v icmp -i enp1s0 -c3
tcpdump: listening on enp1s0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
14:46:27.439541 IP (tos 0x0, ttl 64, id 2763, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.86.53 > 1.1.1.1: ICMP echo request, id 16, seq 0, length 64
14:46:27.442535 IP (tos 0x2c, ttl 55, id 17039, offset 0, flags [none], proto ICMP (1), length 84)
    1.1.1.1 > 192.168.86.53: ICMP echo reply, id 16, seq 0, length 64
14:46:27.442642 IP (tos 0x2c, ttl 54, id 17039, offset 0, flags [none], proto ICMP (1), length 84)
    1.1.1.1 > 192.168.3.3: ICMP echo reply, id 16, seq 0, length 64
3 packets captured
3 packets received by filter
0 packets dropped by kernel

Ping output from my alpine container:

/ # ping 1.1.1.1 -c1
PING 1.1.1.1 (1.1.1.1): 56 data bytes

--- 1.1.1.1 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss

I think my deco router doesn't want to route addresses outside of the LAN range. But setting NAT on the podman host should work? What am I missing? I want my containers to be able to connect to the Internet.

muru
  • 207,228
xpk
  • 436

1 Answers1

0

I got it. All I need is replace the ipvlan driver option from l3 to l3s. Container traffic is routed to the host, NAT-ed on the host's iptables and so forth.

It didn't work on l3 because l3 driver sends egress traffic to netfilter, but ingress does not get sent back to netfilter. l3s driver puts egress and ingress traffic through netfilter [1]

[1] https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/system_design_guide/getting-started-with-ipvlan_system-design-guide#ipvlan-modes_getting-started-with-ipvlan

xpk
  • 436