主页 / server / 问题 / 1065321
Accepted
I. Shm
Asked: 2021-06-02 06:11:14 +0800 CST

Kibana 不信任 Kubernetes Helm 设置中的 Let's Encrypt CA

  • 772

我有一个配置了 calico 作为网络系统的 k8s 集群。我正在运行启用了安全性的 helm elastic/elasticsearch 和 elastic/kibana 的略微定制版本。为了安全起见，我使用 Let's Encrypt 证书。启动 kibana 时，与 elasticsearch 实例的连接失败并出现此错误

{"type":"log","@timestamp":"2021-06-01T13:09:55+00:00","tags":["debug","elasticsearch","query","data"],"pid":952,"message":"[ConnectionError]: unable to get issuer certificate"}

我可以通过在 kibana 中禁用证书验证来解决此问题。任何人都可以看到，为什么它会失败？

密钥库创建

cat cert1.pem > store.pem
cat privkey1.pem >> store.pem
cat chain1.pem >> store.pem
cat fullchain1.pem >> store.pem
openssl pkcs12 -export -in store.pem -out keystore.pkcs12

values_elastic.yaml

replicas: 1
minimumMasterNodes: 1

esConfig:
   elasticsearch.yml: |
     xpack.security.transport.ssl.enabled: true
     xpack.security.transport.ssl.verification_mode: certificate
     xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12
     xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12
     xpack.security.http.ssl.enabled: true
     xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12
     xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12
     xpack.security.enabled: true
extraEnvs:
  - name: ELASTIC_PASSWORD
    valueFrom:
      secretKeyRef:
        name: elastic-credentials
        key: password
  - name: ELASTIC_USERNAME
    valueFrom:
      secretKeyRef:
        name: elastic-credentials
        key: username
secretMounts:
  - name: elastic-certificates
    secretName: elastic-certificates
    path: /usr/share/elasticsearch/config/certs-gen/
protocol: https
service:
  labels: {}
  labelsHeadless: {}
  type: NodePort
  nodePort: 30001
  annotations: {}
  httpPortName: http
  transportPortName: transport
  loadBalancerIP: ""
  loadBalancerSourceRanges: []
  externalTrafficPolicy: ""
clusterHealthCheckParams: "wait_for_status=yellow&timeout=1s"

values_kibana.yaml

elasticsearchHosts: "redacted its a TLD with appropriate port"

extraEnvs:
  - name: "NODE_OPTIONS"
    value: "--max-old-space-size=1800"
  - name: 'ELASTICSEARCH_USERNAME'
    valueFrom:
      secretKeyRef:
        name: elastic-credentials
        key: username
  - name: 'ELASTICSEARCH_PASSWORD'
    valueFrom:
      secretKeyRef:
        name: elastic-credentials
        key: password
  - name: 'KIBANA_ENCRYPTION_KEY'
    valueFrom:
      secretKeyRef:
        name: kibana
        key: encryptionkey
  - name: 'LOGGING_VERBOSE'
    value: "true"


secretMounts:
  - name: elastic-certificates
    secretName: elastic-certificates
    path: /usr/share/kibana/config/certs-gen/


kibanaConfig:
  kibana.yml: |
    server.ssl:
      enabled: true
      key: /usr/share/kibana/config/certs-gen/privkey1.pem
      certificate: /usr/share/kibana/config/certs-gen/fullchain1.pem
    elasticsearch.ssl:
      certificateAuthorities: /usr/share/kibana/config/certs-gen/fullchain1.pem
      verificationMode: certificate
    xpack.reporting.encryptionKey: ${KIBANA_ENCRYPTION_KEY}
    xpack.security.encryptionKey: ${KIBANA_ENCRYPTION_KEY}
    xpack.encryptedSavedObjects.encryptionKey: ${KIBANA_ENCRYPTION_KEY}

protocol: https

service:
  type: NodePort
  loadBalancerIP: ""
  port: 5601
  nodePort: 30002
  labels: {}
  annotations: {}
    # cloud.google.com/load-balancer-type: "Internal"
    # service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
    # service.beta.kubernetes.io/azure-load-balancer-internal: "true"
    # service.beta.kubernetes.io/openstack-internal-load-balancer: "true"
    # service.beta.kubernetes.io/cce-load-balancer-internal-vpc: "true"
  loadBalancerSourceRanges: []
    # 0.0.0.0/0
  httpPortName: HTTP

kubectl 获取 pv,pvc,nodes,pods,svc

NAME                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                                 STORAGECLASS   REASON   AGE
persistentvolume/elk-data   30Gi       RWO            Retain           Bound    default/elasticsearch-master-elasticsearch-master-0                           40m

NAME                                                                STATUS   VOLUME     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/elasticsearch-master-elasticsearch-master-0   Bound    elk-data   30Gi       RWO                           32m

NAME                 STATUS   ROLES                  AGE   VERSION
node/kubeloadbalan   Ready    control-plane,master   28h   v1.21.1

NAME                                    READY   STATUS    RESTARTS   AGE
pod/elasticsearch-master-0              1/1     Running   0          13m
pod/kibana-kibana-7fdbd7c66d-bg5xb      0/1     Running   0          7m1s
pod/nginx-deployment-868c6bb874-tsbg4   1/1     Running   0          40m

NAME                                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                         AGE
service/elasticsearch-master            NodePort    10.97.47.66     <none>        9200:30001/TCP,9300:32185/TCP   13m
service/elasticsearch-master-headless   ClusterIP   None            <none>        9200/TCP,9300/TCP               13m
service/kibana-kibana                   NodePort    10.101.78.90    <none>        5601:30002/TCP                  7m1s
service/kubernetes                      ClusterIP   10.96.0.1       <none>        443/TCP                         28h
service/nginx-service                   NodePort    10.100.231.27   <none>        80:30000/TCP                    40m

从容器内部：

kubectl exec pod/kibana-kibana-7fdbd7c66d-bg5xb -it bash

curl -k -u redacted:redacted https://redacted:30001

{
  "name" : "elasticsearch-master-0",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "iXEuqB7iQ9abptIZ_Gp1yg",
  "version" : {
    "number" : "7.13.0",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "5ca8591c6fcdb1260ce95b08a8e023559635c6f3",
    "build_date" : "2021-05-19T22:22:26.081971330Z",
    "build_snapshot" : false,
    "lucene_version" : "8.8.2",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}


curl  -u redacted:redacted https://redacted:30001
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

这表明，即使 curl 也无法从容器内部验证合法性。这可能表明问题也出在 elasticsearch 上。由于详细程度增加，完整的 Kibana 日志完全是一团糟。我可以根据要求发布。

我为此沉迷了好几天。我筋疲力尽，不知道从哪里继续。

kubernetes elasticsearch lets-encrypt kibana

1 个回答

  1. Best Answer

    通过在评论中讨论问题的可能根本原因（缺少CA），我自己找到了解决方案。该问题是由添加错误的 CA 引起的

    elasticsearch.ssl:
  certificateAuthorities: /usr/share/kibana/config/certs-gen/fullchain1.pem
  verificationMode: certificate

    删除这些条目可以解决问题。

    • 2

相关问题