我有一个配置了 calico 作为网络系统的 k8s 集群。我正在运行启用了安全性的 helm elastic/elasticsearch 和 elastic/kibana 的略微定制版本。为了安全起见，我使用 Let's Encrypt 证书。启动 kibana 时，与 elasticsearch 实例的连接失败并出现此错误
{"type":"log","@timestamp":"2021-06-01T13:09:55+00:00","tags":["debug","elasticsearch","query","data"],"pid":952,"message":"[ConnectionError]: unable to get issuer certificate"}
我可以通过在 kibana 中禁用证书验证来解决此问题。任何人都可以看到，为什么它会失败？
密钥库创建
cat cert1.pem > store.pem
cat privkey1.pem >> store.pem
cat chain1.pem >> store.pem
cat fullchain1.pem >> store.pem
openssl pkcs12 -export -in store.pem -out keystore.pkcs12
values_elastic.yaml
replicas: 1
minimumMasterNodes: 1
esConfig:
elasticsearch.yml: |
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12
xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12
xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12
xpack.security.enabled: true
extraEnvs:
- name: ELASTIC_PASSWORD
valueFrom:
secretKeyRef:
name: elastic-credentials
key: password
- name: ELASTIC_USERNAME
valueFrom:
secretKeyRef:
name: elastic-credentials
key: username
secretMounts:
- name: elastic-certificates
secretName: elastic-certificates
path: /usr/share/elasticsearch/config/certs-gen/
protocol: https
service:
labels: {}
labelsHeadless: {}
type: NodePort
nodePort: 30001
annotations: {}
httpPortName: http
transportPortName: transport
loadBalancerIP: ""
loadBalancerSourceRanges: []
externalTrafficPolicy: ""
clusterHealthCheckParams: "wait_for_status=yellow&timeout=1s"
values_kibana.yaml
elasticsearchHosts: "redacted its a TLD with appropriate port"
extraEnvs:
- name: "NODE_OPTIONS"
value: "--max-old-space-size=1800"
- name: 'ELASTICSEARCH_USERNAME'
valueFrom:
secretKeyRef:
name: elastic-credentials
key: username
- name: 'ELASTICSEARCH_PASSWORD'
valueFrom:
secretKeyRef:
name: elastic-credentials
key: password
- name: 'KIBANA_ENCRYPTION_KEY'
valueFrom:
secretKeyRef:
name: kibana
key: encryptionkey
- name: 'LOGGING_VERBOSE'
value: "true"
secretMounts:
- name: elastic-certificates
secretName: elastic-certificates
path: /usr/share/kibana/config/certs-gen/
kibanaConfig:
kibana.yml: |
server.ssl:
enabled: true
key: /usr/share/kibana/config/certs-gen/privkey1.pem
certificate: /usr/share/kibana/config/certs-gen/fullchain1.pem
elasticsearch.ssl:
certificateAuthorities: /usr/share/kibana/config/certs-gen/fullchain1.pem
verificationMode: certificate
xpack.reporting.encryptionKey: ${KIBANA_ENCRYPTION_KEY}
xpack.security.encryptionKey: ${KIBANA_ENCRYPTION_KEY}
xpack.encryptedSavedObjects.encryptionKey: ${KIBANA_ENCRYPTION_KEY}
protocol: https
service:
type: NodePort
loadBalancerIP: ""
port: 5601
nodePort: 30002
labels: {}
annotations: {}
# cloud.google.com/load-balancer-type: "Internal"
# service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
# service.beta.kubernetes.io/azure-load-balancer-internal: "true"
# service.beta.kubernetes.io/openstack-internal-load-balancer: "true"
# service.beta.kubernetes.io/cce-load-balancer-internal-vpc: "true"
loadBalancerSourceRanges: []
# 0.0.0.0/0
httpPortName: HTTP
kubectl 获取 pv,pvc,nodes,pods,svc
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/elk-data 30Gi RWO Retain Bound default/elasticsearch-master-elasticsearch-master-0 40m
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/elasticsearch-master-elasticsearch-master-0 Bound elk-data 30Gi RWO 32m
NAME STATUS ROLES AGE VERSION
node/kubeloadbalan Ready control-plane,master 28h v1.21.1
NAME READY STATUS RESTARTS AGE
pod/elasticsearch-master-0 1/1 Running 0 13m
pod/kibana-kibana-7fdbd7c66d-bg5xb 0/1 Running 0 7m1s
pod/nginx-deployment-868c6bb874-tsbg4 1/1 Running 0 40m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/elasticsearch-master NodePort 10.97.47.66 <none> 9200:30001/TCP,9300:32185/TCP 13m
service/elasticsearch-master-headless ClusterIP None <none> 9200/TCP,9300/TCP 13m
service/kibana-kibana NodePort 10.101.78.90 <none> 5601:30002/TCP 7m1s
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 28h
service/nginx-service NodePort 10.100.231.27 <none> 80:30000/TCP 40m
从容器内部：
kubectl exec pod/kibana-kibana-7fdbd7c66d-bg5xb -it bash
curl -k -u redacted:redacted https://redacted:30001
{
"name" : "elasticsearch-master-0",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "iXEuqB7iQ9abptIZ_Gp1yg",
"version" : {
"number" : "7.13.0",
"build_flavor" : "default",
"build_type" : "docker",
"build_hash" : "5ca8591c6fcdb1260ce95b08a8e023559635c6f3",
"build_date" : "2021-05-19T22:22:26.081971330Z",
"build_snapshot" : false,
"lucene_version" : "8.8.2",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
curl -u redacted:redacted https://redacted:30001
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
这表明，即使 curl 也无法从容器内部验证合法性。这可能表明问题也出在 elasticsearch 上。由于详细程度增加，完整的 Kibana 日志完全是一团糟。我可以根据要求发布。
我为此沉迷了好几天。我筋疲力尽，不知道从哪里继续。
