我有一个 kubernetes 集群(目前只有 1 个节点)在我的根服务器(netcup)上运行,并尝试将流量路由到它。我在网络方面很糟糕,所以希望有人能帮助我。
在集群上安装了metallb和 calico 作为 CNI。Metallb 负责为 K8s-LoadBalancers 分配和公布 IP 地址,这在内部运行良好。所以我可以从服务器内部达到负载平衡服务。我只公开一个具有固定内部 IP 的服务(反向代理)
现在我想通过使用将流量直接路由到 metallb 的 IP 地址来将它暴露给外部iptables
在 iptables 中应用了以下规则:
iptables -A PREROUTING -t nat -p tcp -m tcp -j DNAT --to-destination <metallb-ip> -i eth0 --destination-port 80 -m comment --comment Redirect web traffic to cluster (80)
iptables -A PREROUTING -t nat -p tcp -m tcp -j DNAT --to-destination <metallb-ip> -i eth0 --destination-port 443 -m comment --comment Redirect web traffic to cluster (443)
iptables -A POSTROUTING -t nat -p tcp -m tcp -j SNAT --to-source <public-server-ip> -o eth0 --destination-port 80 -m comment --comment Redirect web traffic from cluster (80)
iptables -A POSTROUTING -t nat -p tcp -m tcp -j SNAT --to-source <public-server-ip> -o eth0 --destination-port 443 -m comment --comment Redirect web traffic from cluster (443)
(也许命令不是 100% 正确,因为我从我的 ansible 脚本中提取了它们。你可以在底部找到 ansible 部分)
据我所知,这应该使用 DNAT 将所有传入的 TCP 流量发送到内部 IP,并且所有传出流量都将使用 SNAT 具有公共服务器地址。那么对于连接的客户端来说,它应该看起来像是在直接与公共 IP/端口对话?
不幸的是,尝试从 Internet 访问该服务会导致以下结果:
$> curl <public-server-ip>
curl: (7) Failed to connect to <public-server-ip> port 80 after 647 ms: Network is down
从服务器访问有效:
$> curl <metallb-ip>
404 page not found # which the expected answer since its a blank Traefik reverse proxy
nmap 端口扫描当前抛出以下内容:
$> nmap -Pn -p 80,443 <public-server-ip>
Starting Nmap 7.93 ( https://nmap.org ) at 2022-12-15 15:57 CET
Strange SO_ERROR from connection to <public-server-ip> (50 - 'Network is down') -- bailing scan
QUITTING!
但自从我通过 SSH 连接后,它就启动并运行了。
目前没有ufw
启用。
我很感激任何帮助,因为正如我所说,我不太擅长网络。;)
可靠的脚本:
- name: DNAT port 80 to cluster
iptables:
table: nat
chain: PREROUTING
in_interface: eth0
protocol: tcp
match: tcp
jump: DNAT # REDIRECT
to_destination: <metallb-ip>
destination_port: 80
comment: Redirect web traffic to cluster (80)
become: yes
- name: DNAT port 443 to cluster
iptables:
table: nat
chain: PREROUTING
in_interface: eth0
protocol: tcp
match: tcp
destination_port: 443
jump: DNAT # REDIRECT
to_destination: <metallb-ip>
comment: Redirect web traffic to cluster (443)
become: yes
- name: SNAT port 80 from cluster
iptables:
table: nat
chain: POSTROUTING
out_interface: eth0
protocol: tcp
match: tcp
destination_port: 80
jump: SNAT # REDIRECT
to_source: <public-server-ip>
comment: Redirect web traffic from cluster (443)
become: yes
- name: SNAT port 443 from cluster
iptables:
table: nat
chain: POSTROUTING
out_interface: eth0
protocol: tcp
match: tcp
destination_port: 443
jump: SNAT # REDIRECT
to_source: <public-server-ip>
comment: Redirect web traffic from cluster (443)
become: yes
- name: Configure IP Masquerading
iptables:
table: nat
chain: POSTROUTING
out_interface: eth0
jump: MASQUERADE
更新:在这篇文章 的帮助下,我试图理解流程。添加了日志记录语句以调查哪里出了问题。这是我到目前为止发现的:
为了检查数据包在哪里丢失,我添加了以下规则:
iptables -t raw -A PREROUTING -p tcp --dport 80 -j LOG --log-prefix "[raw:PREROUTING]: "
iptables -t mangle -A PREROUTING -p tcp --dport 80 -j LOG --log-prefix "[mangle:PREROUTING]: "
iptables -t nat -A PREROUTING -p tcp --dport 80 -j LOG --log-prefix "[nat:PREROUTING]: "
iptables -t mangle -A POSTROUTING -p tcp --sport 80 -j LOG --log-prefix "[mangle:PREROUTING]: "
iptables -t nat -A POSTROUTING -p tcp --sport 80 -j LOG --log-prefix "[nat:PREROUTING]: "
在/var/log/kern.log
刚刚出现的raw
和mangle
日志语句下。所以似乎nat
没有触发 -step 。使用以下方法检查 mangle 规则:
sudo iptables -t mangle -L
但无法在此处发现任何问题。
IP 转发已启用:
$> sudo sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1
那么为什么数据包没有到达nat
-step?
$> sudo iptables -t mangle -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
cali-PREROUTING all -- anywhere anywhere /* cali:6gwbT8clXdHdC1b1 */
LOG tcp -- anywhere anywhere tcp dpt:http LOG level warning prefix "[mangle:PREROUTING]: "
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
cali-POSTROUTING all -- anywhere anywhere /* cali:O3lYWMrLQYEMJtB5 */
Chain KUBE-IPTABLES-HINT (0 references)
target prot opt source destination
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
Chain KUBE-PROXY-CANARY (0 references)
target prot opt source destination
Chain cali-from-host-endpoint (1 references)
target prot opt source destination
Chain cali-to-host-endpoint (1 references)
target prot opt source destination
Chain cali-PREROUTING (1 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere /* cali:6BJqBjBC7crtA-7- */ ctstate RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere /* cali:KX7AGNd6rMcDUai6 */ mark match 0x10000/0x10000
cali-from-host-endpoint all -- anywhere anywhere /* cali:wNH7KsA3ILKJBsY9 */
ACCEPT all -- anywhere anywhere /* cali:Cg96MgVuoPm7UMRo */ /* Host endpoint policy accepted packet. */ mark match 0x10000/0x10000
Chain cali-POSTROUTING (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere /* cali:NX-7roTexQ3fGRfU */ mark match 0x10000/0x10000
MARK all -- anywhere anywhere /* cali:nnqPh8lh2VOogSzX */ MARK and 0xfff0ffff
cali-to-host-endpoint all -- anywhere anywhere /* cali:nquN8Jw8Tz72pcBW */ ctstate DNAT
RETURN all -- anywhere anywhere /* cali:jWrgvDQ0xEZHmta3 */ /* Host endpoint policy accepted packet. */ mark match 0x10000/0x10000