主页 / server / 问题 / 1161474
Accepted
TommyPeanuts
Asked: 2024-06-25 03:49:14 +0800 CST

Docker 容器无法在多宿主主机上发送邮件

  • 772

我有一个带有两个 IP 地址的 Docker 主机（分别称为A和B）。Apache 绑定到A，解析为web.mydomain.com，而 Postfix 绑定B到mail.mydomain.com。

Docker 应用程序旨在通过 SMTP-AUTH 向 发送邮件mail.mydomain.com。它还有一个用于监听 的 Web UI localhost:3000（显示docker ps为127.0.0.1:3000->3000/tcp）。它通过 Apache 代理到 上的外部世界web.mydomain.com。

该容器似乎能够解析来自主机的A(.59) 和B(.83) 的正确 DNS 请求：

nextjs@5b3fccbf323a:/app$ getent hosts 182.xx.xx.59
182.xx.xx.59    mail.mydomain.com

nextjs@5b3fccbf323a:/app$ getent hosts 182.xx.xx.83
182.xx.xx.83    www.mydomain.com

但是当我尝试连接到邮件端口（从容器内部）时，什么也不会发生，直到超时：

openssl s_client -connect mail.mydomain.com:587 -starttls smtp

从主机连接没问题。运行其他应用程序的其他 Docker 容器可以以相同的方式发送邮件，没有问题。

我对 Docker 网络了解甚少。容器的端口 587 上的连接似乎转到 Web IP A，但我不确定。Web UIA正常运行。我如何说服容器连接到AWeb 接口和BSMTP 提交接口？

tcpdump 显示端口 587 上有来自容器的数据包，但没有从邮件服务器返回：

14:30:53.168711 br-36bedc370406 In  ifindex 102 02:42:ac:1f:00:02 ethertype IPv4 (0x0800), length 80: 172.31.0.2.55954 > 185.x.x.59.587: Flags [S], seq 1208255870, win 64240, options [mss 1460,sackOK,TS val 241634928 ecr 0,nop,wscale 7], length 0

iptables-save（省略fail2ban规则）：

# Generated by iptables-save v1.8.7 on Tue Jun 25 14:37:41 2024
*filter
:INPUT DROP [162608:8659146]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [31071143:38820437864]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
:LOGGING - [0:0]
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i eth0 -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -i eth0 -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -i docker0 -p tcp -j ACCEPT
-A INPUT -p tcp -m tcp --dport 21 -m conntrack --ctstate NEW,ESTABLISHED -m geoip --source-country GB,UG,KE,ZA  -j ACCEPT
-A INPUT -p tcp -m tcp --dport 20 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --sport 1024:65535 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m multiport --dports 12000:13000 -j ACCEPT
-A INPUT -p tcp -m multiport --dports 12000:13000 -j ACCEPT
-A INPUT -i eth0 -p tcp -m multiport --dports 25,110,143,465,587,993,995,2525,2526 -j ACCEPT
-A INPUT -i eth0 -p tcp -m multiport --dports 80,443 -j ACCEPT
-A INPUT -i eth0 -p tcp -m multiport --dports 22,873 -m geoip --source-country GB  -j ACCEPT
-A INPUT -i eth0 -p tcp -m multiport --dports 8080,3100 -m geoip --source-country GB  -j ACCEPT
-A INPUT -i eth0 -p icmp -j ACCEPT
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -o br-36bedc370406 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-36bedc370406 -j DOCKER
-A FORWARD -i br-36bedc370406 ! -o br-36bedc370406 -j ACCEPT
-A FORWARD -i br-36bedc370406 -o br-36bedc370406 -j ACCEPT
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -o docker0 -j ACCEPT
-A OUTPUT -p tcp -m tcp --dport 443 -m owner ! --uid-owner 117 -j LOGGING
-A OUTPUT -p tcp -m tcp --dport 443 -m owner ! --uid-owner 117 -j REJECT --reject-with icmp-port-unreachable
-A OUTPUT -p tcp -m tcp --dport 563 -m owner ! --uid-owner 117 -j REJECT --reject-with icmp-port-unreachable
-A OUTPUT -p tcp -m tcp --dport 21 -m conntrack --ctstate NEW -j REJECT --reject-with icmp-port-unreachable
-A DOCKER -d 172.31.0.2/32 ! -i br-36bedc370406 -o br-36bedc370406 -p tcp -m tcp --dport 3000 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i br-36bedc370406 ! -o br-36bedc370406 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-36bedc370406 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
-A LOGGING -m limit --limit 2/min -j LOG --log-prefix "IPTables-Dropped: "
COMMIT
# Completed on Tue Jun 25 14:37:41 2024
# Generated by iptables-save v1.8.7 on Tue Jun 25 14:37:41 2024
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT -p tcp -m tcp --dport 80 -m owner ! --uid-owner 117 -j REDIRECT --to-ports 8888
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 172.31.0.0/16 ! -o br-36bedc370406 -j MASQUERADE
-A POSTROUTING -s 172.31.0.2/32 -d 172.31.0.2/32 -p tcp -m tcp --dport 3000 -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
-A DOCKER -i br-36bedc370406 -j RETURN
-A DOCKER -d 127.0.0.1/32 ! -i br-36bedc370406 -p tcp -m tcp --dport 3000 -j DNAT --to-destination 172.31.0.2:3000
COMMIT

nft list ruleset显示：

table ip filter {
        chain INPUT {
                type filter hook input priority filter; policy drop;
                meta l4proto tcp tcp dport { 80,443} counter packets 1642563 bytes 190179301 jump f2b-apache-fakegooglebot
                meta l4proto tcp tcp dport { 80,443} counter packets 9116728 bytes 1069624630 jump f2b-apache-auth
                meta l4proto tcp tcp dport 22 counter packets 413620 bytes 31257328 jump f2b-sshd
                meta l4proto tcp tcp dport { 110,995,143,993,587,465,4190} counter packets 6304246 bytes 659133161 jump f2b-dovecot
                meta l4proto tcp tcp dport { 25,465,587} counter packets 1358308 bytes 1357726201 jump f2b-postfix
                meta l4proto tcp tcp dport { 80,443,25,587,110,995,143,993,4190} counter packets 15403400 bytes 2939709444 jump f2b-postfix
                iifname "lo" counter packets 45927262 bytes 22275943601 accept
                ct state related,established counter packets 23416790 bytes 5815553538 accept
                iifname "eth0" meta l4proto tcp tcp dport 53 counter packets 5885 bytes 340579 accept
                iifname "eth0" meta l4proto udp udp dport 53 counter packets 895484 bytes 69357764 accept
                iifname "docker0" meta l4proto tcp counter packets 3001 bytes 180060 accept
                meta l4proto tcp tcp dport 21 ct state new,established # -m geoip --source-country GB,UG,KE,ZA  counter packets 49 bytes 2480 accept
                meta l4proto tcp tcp dport 20 ct state related,established counter packets 0 bytes 0 accept
                meta l4proto tcp tcp sport 1024-65535 tcp dport 1024-65535 ct state established counter packets 0 bytes 0 accept
                meta l4proto tcp tcp dport 12000-13000 counter packets 2973 bytes 135532 accept
                meta l4proto tcp tcp dport 12000-13000 counter packets 0 bytes 0 accept
                iifname "eth0" meta l4proto tcp tcp dport { 25,110,143,465,587,993,995,2525,2526} counter packets 197174 bytes 11469412 accept
                iifname "eth0" meta l4proto tcp tcp dport { 80,443} counter packets 787414 bytes 46406299 accept
                iifname "eth0" meta l4proto tcp tcp dport { 22,873} # -m geoip --source-country GB  counter packets 275 bytes 15232 accept
                iifname "eth0" meta l4proto tcp tcp dport { 8080,3100} # -m geoip --source-country GB  counter packets 143 bytes 6132 accept
                iifname "eth0" meta l4proto icmp counter packets 29321 bytes 1823925 accept
                counter packets 205049 bytes 10393170 jump SpamhausIN
        }

        chain LOGGING {
                limit rate 2/minute counter packets 24372 bytes 1259527 log prefix "IPTables-Dropped: "
        }
        chain FORWARD {
                type filter hook forward priority filter; policy drop;
                counter packets 91 bytes 19822 jump DOCKER-USER
                counter packets 91 bytes 19822 jump DOCKER-ISOLATION-STAGE-1
                oifname "docker0" ct state related,established counter packets 6 bytes 636 accept
                oifname "docker0" counter packets 0 bytes 0 jump DOCKER
                iifname "docker0" oifname != "docker0" counter packets 6 bytes 390 accept
                iifname "docker0" oifname "docker0" counter packets 0 bytes 0 accept
                oifname "br-36bedc370406" ct state related,established counter packets 217 bytes 55250 accept
                oifname "br-36bedc370406" counter packets 6 bytes 360 jump DOCKER
                iifname "br-36bedc370406" oifname != "br-36bedc370406" counter packets 0 bytes 0 accept
                iifname "br-36bedc370406" oifname "br-36bedc370406" counter packets 6 bytes 360 accept
                counter packets 0 bytes 0 jump SpamhausIN
        }

        chain OUTPUT {
                type filter hook output priority filter; policy accept;
                oifname "lo" counter packets 45927236 bytes 22275941960 accept
                oifname "docker0" counter packets 53070 bytes 13526184 accept
                meta l4proto tcp tcp dport 443 skuid != 117 counter packets 137554 bytes 6981510 jump LOGGING
                meta l4proto tcp tcp dport 443 skuid != 117 counter packets 137554 bytes 6981510 reject
                meta l4proto tcp tcp dport 563 skuid != 117 counter packets 6 bytes 312 reject
                meta l4proto tcp tcp dport 21 ct state new counter packets 0 bytes 0 reject
                counter packets 32097922 bytes 38886685465 jump SpamhausOUT
        }

        chain f2b-dovecot {
                counter packets 128626 bytes 13891673 return
        }

        chain DOCKER {
        }
      chain DOCKER-ISOLATION-STAGE-1 {
                iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 jump DOCKER-ISOLATION-STAGE-2
                iifname "br-36bedc370406" oifname != "br-36bedc370406" counter packets 0 bytes 0 jump DOCKER-ISOLATION-STAGE-2
                counter packets 0 bytes 0 return
        }

        chain DOCKER-ISOLATION-STAGE-2 {
                oifname "docker0" counter packets 0 bytes 0 drop
                oifname "br-36bedc370406" counter packets 0 bytes 0 drop
                counter packets 0 bytes 0 return
        }

        chain DOCKER-USER {
                counter packets 0 bytes 0 return
        }
}
table ip nat {
        chain OUTPUT {
                type nat hook output priority -100; policy accept;
                meta l4proto tcp tcp dport 80 skuid != 117 counter packets 2 bytes 120 redirect to :8888
                ip daddr != 127.0.0.0/8 fib daddr type local counter packets 8 bytes 480 jump DOCKER
        }

        chain POSTROUTING {
                type nat hook postrouting priority srcnat; policy accept;
                oifname != "docker0" ip saddr 172.17.0.0/16 counter packets 3 bytes 195 masquerade 
                oifname != "br-36bedc370406" ip saddr 172.31.0.0/16 counter packets 0 bytes 0 masquerade 
                meta l4proto tcp ip saddr 172.31.0.2 ip daddr 172.31.0.2 tcp dport 3000 counter packets 0 bytes 0 masquerade 
        }

        chain PREROUTING {
                type nat hook prerouting priority dstnat; policy accept;
                fib daddr type local counter packets 2221 bytes 148479 jump DOCKER
        }

        chain DOCKER {
                iifname "docker0" counter packets 3 bytes 180 return
                iifname "br-36bedc370406" counter packets 7 bytes 420 return
                iifname != "br-36bedc370406" meta l4proto tcp ip daddr 127.0.0.1 tcp dport 3000 counter packets 0 bytes 0 dnat to 172.31.0.2:3000
        }
}
table ip6 nat {
        chain OUTPUT {
                type nat hook output priority -100; policy accept;
                meta l4proto tcp tcp dport 80 skuid != 117 counter packets 35400 bytes 2832000 redirect to :8888
        }

        chain DOCKER {
        }
}

host:~$ ip route（ip不在容器上）的输出：

default via 185.x.x.1 dev eth0 proto static onlink 
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 
172.31.0.0/16 dev br-36bedc370406 proto kernel scope link src 172.31.0.1 
185.x.x.0/22 dev eth0 proto kernel scope link src 185.x.x.83 

~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 02:00:00:00:40:17 brd ff:ff:ff:ff:ff:ff
    inet 185.x.x.83/22 brd 185.x.x.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet 185.x.x.59/22 brd 185.x.x.255 scope global secondary eth0
       valid_lft forever preferred_lft forever
    inet6 2001:xxx:x:xxx::3b/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 2001:xxx:x:xxx::53/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::ff:fe00:4017/64 scope link 
       valid_lft forever preferred_lft forever
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:91:dc:5d:95 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:91ff:fedc:5d95/64 scope link 
       valid_lft forever preferred_lft forever
10: vethf05d58a@if9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default 
    link/ether 6a:7e:2f:5d:df:f3 brd ff:ff:ff:ff:ff:ff link-netnsid 2
    inet6 fe80::687e:2fff:fe5d:dff3/64 scope link 
       valid_lft forever preferred_lft forever
102: br-36bedc370406: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:da:79:75:ae brd ff:ff:ff:ff:ff:ff
    inet 172.31.0.1/16 brd 172.31.255.255 scope global br-36bedc370406
       valid_lft forever preferred_lft forever
    inet6 fe80::42:daff:fe79:75ae/64 scope link 
       valid_lft forever preferred_lft forever
104: vetheec212b@if103: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-36bedc370406 state UP group default 
    link/ether 1e:4a:34:89:98:cf brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::1c4a:34ff:fe89:98cf/64 scope link 
       valid_lft forever preferred_lft forever
110: veth216a525@if109: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-36bedc370406 state UP group default 
    link/ether 4e:15:61:92:30:a0 brd ff:ff:ff:ff:ff:ff link-netnsid 1
    inet6 fe80::4c15:61ff:fe92:30a0/64 scope link 
       valid_lft forever preferred_lft forever

host:~$ docker network ls显示如下：

NETWORK ID     NAME             DRIVER    SCOPE
1f2cb04b3b54   bridge           bridge    local
c9ea0692db1b   host             host      local
a91ee892376d   none             null      local
36bedc370406   rallly_default   bridge    local

该容器使用的网络称为“rallly_default”。

host:~$ docker network inspect rallly_default

{
        "Name": "rallly_default",
        "Id": "36bedc3704069d8d2ad6cc02031296a5fadb95ca988ee1a0e9d461b5e6ba083b",
        "Created": "2024-06-24T20:59:40.356497842Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.31.0.0/16",
                    "Gateway": "172.31.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "4c41cdd942d82c1504804fbf94e3980ef96adaabbec43ae36ac8ec21c255d970": {
                "Name": "rallly-rallly_db-1",
                "EndpointID": "1f7352607a1062ddb6ae421b490bb12739a099476137396737a1a60b752e67d0",
                "MacAddress": "02:42:ac:1f:00:02",
                "IPv4Address": "172.31.0.2/16",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {
            "com.docker.compose.network": "default",
            "com.docker.compose.project": "rallly",
            "com.docker.compose.version": "2.27.1"
        }
    }
]
networking

1 个回答

  1. Best Answer

    如您所见，您的filter/INPUT策略是DROP，并且 tcp 端口 587 仅对来自 的流量开放eth0（而在关注的情况下，请求来自br-36bedc370406）。同时，所有来自默认 docker bridge 的流量docker0都被接受（这可以解释 的情况other containers，假设它们不是用 docker compose 启动的）。

    显然，您正在维护防火墙iptables（或使用该iptables程序的某个前端），而它实际上是将规则/命令转换为 nftables 规则的变体。因此，您可以继续这样做（并忽略nft/仅考虑使用 iptables 的输出）。

    不过，我不确定这里的最佳做法是什么。您可以添加一条规则，为所有接口打开 tcp 端口 587（即-i规则中没有测试），或者看看是否可以找到一种方法在撰写文件中指定固定的桥接名称。（如果我没记错的话，默认情况下，如果您执行撰写向下/向上循环，桥接名称将更改，但我真的不知道/记得。同样，私有 IP 子网不一定每次都是相同的块，但当然这有点伪确定性。）最终，这取决于您真正需要哪种过滤/“保护”。

    • 1

相关问题