主页 / server / 问题

问题[kubernetes](server)

Martin Hope
Sagar
Asked: 2024-11-16 00:30:50 +0800 CST
  • 5

我定义了以下 ClusterRole：

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: deployer-system
rules:
- apiGroups:
  - '*'
  resources:
  - '*'
  verbs:
  - list
- nonResourceURLs:
  - '*'
  verbs:
  - list

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: deployer-nonsystem
rules:
- apiGroups:
  - '*'
  resources:
  - secrets
  - PersistentVolumes
  - Role
  - RoleBinding
  verbs:
  - list
- apiGroups:
  - '*'
  resources:
  - deployments
  - pods
  verbs:
  - get
  - list
  - watch
  - create
  - update
  - patch
- nonResourceURLs:
  - '*'
  verbs:
  - list

我使用以下 RoleBindings 为用户分配上述 ClusterRole：

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: theuser-nonsystem-role-binding
  namespace: nonsystem-namespace
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: deployer-nonsystem
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: theuser

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: theuser-kube-system-role-binding
  namespace: system-namespace
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: deployer-system
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: theuser

但是，他们可以运行kubectl -n [NAMESPACE] get secrets -o yaml并查看所有机密。根据上面的 ClusterRole 规范，我预计此调用将被禁止。

我是不是遗漏了什么或者误解了什么？为什么用户可以“获取”机密？

更新：请注意：我的问题不是他们可以列出秘密。我的问题是用户可以“获取”秘密（不同的动词！）

kubernetes
Martin Hope
Sotiris Sotiriou
Asked: 2024-10-22 02:33:11 +0800 CST
  • 6

我已经使用 Helm 在我的 AKS 集群中安装了 Istio Ingress Gateway（图表使用的istio/base、istio/istiod、istio/gateway来自https://istio-release.storage.googleapis.com/charts）。

现在我想安装一个Istio Egress Gateway，但我不知道是否还有另一个 Helm 图表，或者我是否必须再次安装 istio/gateway 并将其配置为 Egress。

有人可以帮助我吗，因为 istio 文档中没有使用 Helm 的清晰安装指南？

我应该安装和配置什么来通过此 Egress Gateway 路由所有集群流量并限制每个工作负载对特定主机的访问。例如

apiVersion: networking.istio.io/v1
kind: ServiceEntry
metadata:
  name: external-svc-https
  namespace: testing
spec:
  hosts:
  - api.dropboxapi.com
  - www.googleapis.com
  - api.facebook.com
  location: MESH_EXTERNAL
  exportTo: "."
  ports:
  - number: 443
    name: https
    protocol: TLS
  resolution: NONE

我已经找到了一种方法，使用 ServiceEntry 仅允许每个命名​​空间访问特定主机，并将以下配置更改为 istiod 安装。

meshConfig:
  outboundTrafficPolicy:
    mode: REGISTRY_ONLY
kubernetes
Martin Hope
Сергей Дударев
Asked: 2024-10-07 20:28:45 +0800 CST
  • 5

下午好，我遇到了无法将 MetalLB 连接到 K3S 的问题。我有一个包含 3 个主服务器和 3 个工作服务器的集群

NAME            STATUS   ROLES                       AGE    VERSION
master-node-1   Ready    control-plane,etcd,master   4d2h   v1.30.5+k3s1
master-node-2   Ready    control-plane,etcd,master   4d1h   v1.31.1+k3s1
master-node-3   Ready    control-plane,etcd,master   4d1h   v1.31.1+k3s1
worker-node-1   Ready    <none>                      4d1h   v1.31.1+k3s1
worker-node-2   Ready    <none>                      4d     v1.31.1+k3s1
worker-node-3   Ready    <none>                      4d     v1.31.1+k3s1

最初，我尝试以这种方式安装 MetalLB

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/namespace.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/metallb.yaml

配置

# config.yml
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: default
      protocol: layer2
      addresses:
      - 192.168.0.170-192.168.0.180

这种方法虽然有效，但负载均衡器在水龙头上不稳定，有 50% 的概率无法进入网站。后来我发现这是一种过时的方法。我尝试了一种新方法

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.5/config/manifests/metallb-native.yaml

配置

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: k3s-lb-pool
  namespace: metallb
spec:
  addresses:
  - 192.168.0.170-192.168.0.180
  autoAssign: true
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: k3s-lb-pool
  namespace: metallb

但我收到错误

Error from server (InternalError): error when creating "metalLB-config.yaml": Internal error occurred: failed calling webhook "ipaddresspoolvalidationwebhook.metallb.io": failed to call webhook: Post "https://metallb-webhook-service.metallb.svc:443/validate-metallb-io-v1beta1-ipaddresspool?timeout=10s": no endpoints available for service "metallb-webhook-service"
Error from server (InternalError): error when creating "metalLB-config.yaml": Internal error occurred: failed calling webhook "l2advertisementvalidationwebhook.metallb.io": failed to call webhook: Post "https://metallb-webhook-service.metallb.svc:443/validate-metallb-io-v1beta1-l2advertisement?timeout=10s": no endpoints available for service "metallb-webhook-service"

但命名空间仍然正常工作

root@master-node-1:~# kubectl get pod -n=metallb-system
NAME                          READY   STATUS    RESTARTS        AGE
controller-86f5578878-9ddsr   1/1     Running   2 (3m31s ago)   4m57s
speaker-5j47x                 1/1     Running   0               4m56s
speaker-6drpd                 1/1     Running   0               4m56s
speaker-ccpmq                 1/1     Running   0               4m56s
speaker-jfpch                 1/1     Running   1 (3m12s ago)   4m56s
speaker-mps8b                 1/1     Running   0               4m57s
speaker-t9lx2                 1/1     Running   0               4m56s

（我提前道歉，英语不是我的母语）

kubernetes
Martin Hope
Carlos Dámaso
Asked: 2024-10-05 00:35:45 +0800 CST
  • 5

我的 Talos Linux 集群遇到了一些问题。

我目前在 Virtualbox 上运行 2 台机器（1 台控制平面和 1 台工作器），两台机器都有 4GB 内存和 4 个 CPU 单元。我还有一个 TrueNas 服务器来提供 NFS 共享，TrueNas 上启用了 nfsv4，并且可以在其他 Linux 机器上访问共享（在 ubuntu 20.04 和 Kali Linux 上测试）。

所有机器都已连接。我已按照 Kubernetes 文档的建议安装了 NFS-CSI 驱动程序。我正在尝试从 bitnami 部署 wordpress helm chart，但出现此错误：

 Error: failed to create subPath directory for volumeMount "wordpress-data" of container "wordpress"

NFS 共享设置了 777 个权限。您认为这可能是什么？

kubernetes
Martin Hope
McLayn
Asked: 2024-10-03 03:05:57 +0800 CST
  • 6

我正在尝试运行podman由 kubernetes YML 文件定义的 quadlet。我希望它在主机上的用户systemd下以无根方式运行： 。websystemctl --user status pod-web

在容器内部，PHP 进程以33:33用户身份运行。我希望/home/web/data主机上的目录归用户所有web，但同时在33:33容器内部，该目录可由用户读取和写入，容器被挂载到/var/www/html。

在评论中，你可以看到我尝试的一些方法。不幸的是，我对用户命名空间和subuid/subgid了解不够，无法使其与文档配合使用，而 Copilot 一直在产生幻觉，因此任何来自真人的帮助都将不胜感激。

我的问题与这个类似，但我想使用podman kubeYML 文件中定义的四元组。

环境：

  • AlmaLinux release 9.4 (Seafoam Ocelot)
  • podman version 4.9.4-rhel
  • systemd 252 (252-32.el9_4.7)
  • SELinux 启用强制
  • chown -R web:web /home/web
  • usermod --add-subuids 100000-165535 --add-subgids 100000-165535 web

/home/web/pod-web.yml：

apiVersion: v1
kind: Pod
metadata:
  name: pod-web
#  annotations:
#    io.podman.annotations.userns: "keep-id"
spec:
  containers:
  - name: pod-web
    image: docker.io/library/php:8.3-apache
#    securityContext:
#      runAsUser: 33
#      runAsGroup: 33
#      supplementalGroups: [65536]
#    ports:
#    - containerPort: 80
#      hostPort: 8000
    volumeMounts:
    - mountPath: /var/www/html
      name: web-data
  volumes:
  - name: web-data
    hostPath:
      path: /home/web/data
  restartPolicy: Always

使用runAsUser: 33容器记录：

(13)Permission denied: AH00072: make_sock: could not bind to address [::]:80
(13)Permission denied: AH00072: make_sock: could not bind to address 0.0.0.0:80

容器runAsUser: 0似乎可以运行，但如果我理解正确的话，这是一种不好的做法，因为它赋予了太多特权。

/home/web/.config/containers/systemd/pod-web.kube：

[Unit]
Description=Podman Quadlet: %p

[Service]
# ExecStartPre=/usr/bin/podman unshare -- /bin/bash -c 'chown -R 33:33 /home/web/data'

[Kube]
Yaml=/home/web/%p.yml
LogDriver=journald
#UserNS=keep-id:uid=33,gid=33
#UserNS=auto

[Install]
WantedBy=multi-user.target default.target
kubernetes
Martin Hope
Seididieci
Asked: 2024-09-19 21:35:45 +0800 CST
  • 6

我已经使用 Fedora 的本指南创建了一个 kubernetes 集群：https ://docs.fedoraproject.org/en-US/quick-docs/using-kubernetes/

我设置了 1 个控制面板和 1 个工作节点（2 个 libvirt VM），并且能够部署应用程序、入口、运行等等……

我唯一想念的是获取一些指标（kubectl top nodes/ ），所以我尝试使用官方 yaml 文件（ https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yamlkubectl top pods ）安装指标服务器。

然后我编辑了部署以修复添加--kubelet-insecure-tls到参数列表中的 x509 证书错误（这不是生产集群，所以我保留了默认的自签名证书）。

我的问题是度量服务器不工作：在日志中它显示抓取节点错误，因为它在请求资源时得到 403（禁止）：“

I0919 08:32:37.415029       1 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/tmp/apiserver.crt::/tmp/apiserver.key"
I0919 08:32:37.415090       1 secure_serving.go:213] Serving securely on [::]:10250
I0919 08:32:37.415119       1 tlsconfig.go:240] "Starting DynamicServingCertificateController"
E0919 08:32:37.417167       1 scraper.go:149] "Failed to scrape node" err="request failed, status: \"403 Forbidden\"" node="f40-work1"
E0919 08:32:37.421847       1 scraper.go:149] "Failed to scrape node" err="request failed, status: \"403 Forbidden\"" node="f40main"

我被困在那里，无法弄清楚如何解决这个问题，谷歌也帮不了我......

节点版本：

NAME        STATUS   ROLES                                            AGE     VERSION   INTERNAL-IP       EXTERNAL-IP   OS-IMAGE                           KERNEL-VERSION            CONTAINER-RUNTIME
f40-work1   Ready    metrics-server-aggregated-reader                 2d19h   v1.29.8   192.168.124.128   <none>        Fedora Linux 40 (Server Edition)   6.10.10-200.fc40.x86_64   cri-o://1.28.2
f40main     Ready    control-plane,metrics-server-aggregated-reader   2d20h   v1.29.8   192.168.124.146   <none>        Fedora Linux 40 (Server Edition)   6.10.9-200.fc40.x86_64    cri-o://1.28.2

我用角色标记了两个节点metrics-server-aggregate-reader并尝试修复错误但没有成功......

kubernetes
Martin Hope
Raphael10
Asked: 2024-09-13 02:11:58 +0800 CST
  • 6

我定义并应用了 ServiceAccount“service-account-token” Vault-Config/service-account-token.yaml：：

apiVersion: v1
kind: ServiceAccount
metadata:
  name: service-account-token
automountServiceAccountToken: false

root@k8s-eu-1-control-plane-node-1:~# kubectl apply -f Vault-Config/service-account- 
token.yaml 
serviceaccount/service-account-token created

root@k8s-eu-1-control-plane-node-1:~# kubectl get ServiceAccount
NAME                       SECRETS   AGE
default                    0         10d
issuer                     0         20h
secrets-store-csi-driver   0         2d9h
service-account-token      0         22s   // <----------------------
webapp-sa                  0         2d1h

我定义并应用了保险库发行者秘密：

root@k8s-eu-1-control-plane-node-1:~# nano Vault-Config/cert-manager-vault-issuer-
secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: issuer-token-abcde
  #namespace: nats
  annotations:
    kubernetes.io/service-account.name: issuer
type: kubernetes.io/service-account-token # https://developer.hashicorp.com/vault
/docs/auth/kubernetes#continue-using-long-lived-tokens

->

root@k8s-eu-1-control-plane-node-1:~# kubectl apply -f Vault-Config/cert-manager-vault-
issuer-secret.yaml 
secret/issuer-token-abcde created

->

root@k8s-eu-1-control-plane-node-1:~# kubectl get secrets
NAME                         TYPE                                  DATA   AGE
issuer-token-abcde           kubernetes.io/service-account-token   3      8s  // <------------
nats-box-contexts            Opaque                                1      6d
sh.helm.release.v1.csi.v1    helm.sh/release.v1                    1      2d9h
sh.helm.release.v1.nats.v1   helm.sh/release.v1                    1      6d

当我申请这个保险库发行人时 Vault-Config/vault-issuer-cert-manager.yaml：：

# https://developer.hashicorp.com/vault/tutorials/archive/kubernetes-cert-   
manager#configure-an-issuer-and-generate-a-certificate

    apiVersion: cert-manager.io/v1
    kind: Issuer
    metadata:
      name: vault-issuer
      #namespace: nats
    spec:
      vault:
        server: http://vault.default // <---- as suggested here: https://cert-manager.io/docs/configuration/vault/#deployment
        path: pki_int/sign/nats
        auth:
          kubernetes:
            mountPath: /v1/auth/kubernetes
            role: issuer
            secretRef:
              name: issuer-token-abcde
              #key: token

->：

root@k8s-eu-1-control-plane-node-1:~# kubectl apply -f Vault-Config/vault-issuer-cert-
manager.yaml 
issuer.cert-manager.io/vault-issuer created

我收到此错误：

root@k8s-eu-1-control-plane-node-1:~# kubectl describe issuer vault-issue
Failed to initialize Vault client: while requesting a Vault token using the Kubernetes auth:
error calling Vault server: Post "https://vault.default/v1/auth/kubernetes/login": dial tcp: 
lookup vault.default on 10.96.0.10:53: no such host

对于 Vault 配置，我通过 helm 应用了以下值：

root@k8s-eu-1-control-plane-node-1:~# nano Vault-Config/overrides.yaml ：

global:
   enabled: true
   tlsDisable: false
injector:
   enabled: true
server:
   extraEnvironmentVars:
      VAULT_CACERT: /vault/userconfig/vault-ha-tls/vault.ca
      VAULT_TLSCERT: /vault/userconfig/vault-ha-tls/vault.crt
      VAULT_TLSKEY: /vault/userconfig/vault-ha-tls/vault.key
   dataStorage:
       enabled: true
   volumes:
      - name: userconfig-vault-ha-tls
        secret:
         defaultMode: 420
         secretName: vault-ha-tls
   volumeMounts:
      - mountPath: /vault/userconfig/vault-ha-tls
        name: userconfig-vault-ha-tls
        readOnly: true
   standalone:
      enabled: false
   affinity: ""
   readinessProbe:
     enabled: true
     path: "/v1/sys/health?standbyok=true&sealedcode=204&uninitcode=204"
   ha:
      enabled: true
      replicas: 3
      raft:
         enabled: true
         setNodeId: true
         config: |
            cluster_name = "vault-integrated-storage"
            ui = true
            listener "tcp" {
               tls_disable = 0
               address = "[::]:8200"
               cluster_address = "[::]:8201"
               tls_cert_file = "/vault/userconfig/vault-ha-tls/vault.crt"
               tls_key_file  = "/vault/userconfig/vault-ha-tls/vault.key"
               tls_client_ca_file = "/vault/userconfig/vault-ha-tls/vault.ca"
            }

            # https://developer.hashicorp.com/vault/tutorials/kubernetes/kubernetes-raft-deployment-guide#vault-storage-configuration

            storage "raft" {
               path = "/vault/data"

               retry_join {
                 leader_api_addr = "https://vault-0.vault-internal:8200"
                 leader_ca_cert_file = "/vault/userconfig/tls-ca/ca.crt"
                 leader_client_cert_file = "/vault/userconfig/tls-server/tls.crt"
                 leader_client_key_file = "/vault/userconfig/tls-server/tls.key"
               }

               retry_join {
                 leader_api_addr = "https://vault-1.vault-internal:8200"
                 leader_ca_cert_file = "/vault/userconfig/tls-ca/ca.crt"
                 leader_client_cert_file = "/vault/userconfig/tls-server/tls.crt"
                 leader_client_key_file = "/vault/userconfig/tls-server/tls.key"
               }

               retry_join {
                 leader_api_addr = "https://vault-2.vault-internal:8200"
                 leader_ca_cert_file = "/vault/userconfig/tls-ca/ca.crt"
                 leader_client_cert_file = "/vault/userconfig/tls-server/tls.crt"
                 leader_client_key_file = "/vault/userconfig/tls-server/tls.key"
               }

               retry_join {
                 leader_api_addr = "https://vault-3.vault-internal:8200"
                 leader_ca_cert_file = "/vault/userconfig/tls-ca/ca.crt"
                 leader_client_cert_file = "/vault/userconfig/tls-server/tls.crt"
                 leader_client_key_file = "/vault/userconfig/tls-server/tls.key"
               }

               retry_join {
                 leader_api_addr = "https://vault-4.vault-internal:8200"
                 leader_ca_cert_file = "/vault/userconfig/tls-ca/ca.crt"
                 leader_client_cert_file = "/vault/userconfig/tls-server/tls.crt"
                 leader_client_key_file = "/vault/userconfig/tls-server/tls.key"
               }

               autopilot {
                 server_stabilization_time = "10s"
                 last_contact_threshold = "10s"
                 min_quorum = 5
                 cleanup_dead_servers = false
                 dead_server_last_contact_threshold = "10m"
                 max_trailing_logs = 1000
                 disable_upgrade_migration = false
               }


            }
            disable_mlock = true
            service_registration "kubernetes" {}

我必须将哪个服务器地址放入 vault-issuer 配置文件中 Vault-Config/vault-issuer-cert-manager.yaml：：

# https://developer.hashicorp.com/vault/tutorials/archive/kubernetes-cert-manager#configure-an-issuer-and-generate-a-certificate

apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: vault-issuer
  #namespace: nats
spec:
  vault:
    server: https://vault-0.vault-internal:8200/    // <----------------- ????????
    path: pki_int/sign/nats
    auth:
      kubernetes:
        mountPath: /v1/auth/kubernetes
        role: issuer
        secretRef:
          name: issuer-token-abcde
          key: token

--> ：

root@k8s-eu-1-control-plane-node-1:~# kubectl describe issuer     vault-issue

Message:               Failed to initialize Vault client: while  
requesting a Vault token using the Kubernetes auth: error calling 
Vault server: Post "http://vault.default:8200/v1/auth/kubernetes
/login": dial tcp: lookup vault.default on 10.96.0.10:53: no such 
host

？

kubernetes
Martin Hope
Luke Cousins
Asked: 2024-09-03 17:47:28 +0800 CST
  • 5

我们最初使用静态清单安装了cert-manager。现在我们需要自定义设置（webhook 超时，请参阅 digitalocean/clusterlint#167），所以我认为最好的选择是迁移到 helm 安装，但我不确定如何最好地做到这一点，非常感谢您的帮助。

我已阅读卸载说明，但不想删除证书或（集群/）颁发者。我可以跳过删除步骤吗？因为我知道稍后我会使用 Helm 重新安装，还是我需要遵循其他流程？

任何帮助都太棒了。谢谢🙏

kubernetes
Martin Hope
Bob5421
Asked: 2024-09-01 14:59:19 +0800 CST
  • 5

看看这个 yaml 清单：

apiVersion: v1
kind: ServiceAccount
metadata:
  name: myserviceaccount
  namespace: default
---
apiVersion: v1
kind: Secret
metadata:
  name: myserviceaccount-token
  namespace: default
  annotations:
    kubernetes.io/service-account.name: myserviceaccount
type: kubernetes.io/service-account-token

我将此清单应用于我的 k8s 集群：

$ kubectl apply -f myserviceaccount.yaml

当我尝试这个命令时：

$ kubectl -n default get service account/myserviceaccount

-> 我看到秘密数量为 0

我应该至少看到 1 个 myserviceaccount-token 的密钥。为什么我看不到这个密钥？

谢谢

kubernetes