在使用 promtail、loki 和 grafana 的设置中，grafana explore 中没有显示任何数据。所有三个部分都在同一台机器上运行。

问题是：在哪里查找设置中的错误？

普罗姆泰尔

promtail.yaml：

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /var/db/promtail/positions.yaml

clients:
  - url: http://127.0.0.1:3100/loki/api/v1/push

scrape_configs:
- job_name: system
  static_configs:
  - targets:
      - localhost
    labels:
      job: varlogs
      __path__: /var/log/**/*log

这似乎没问题。测试：

promtail --config.file=promtail.yaml --log.level=info -print-config-stderr

---
# Loki Config
# (version=, branch=, revision=unknown)
global:
  file_watch_config:
    min_poll_frequency: 250ms
    max_poll_frequency: 250ms
server:
  http_listen_network: tcp
  http_listen_address: ""
  http_listen_port: 9080
[...]
level=info ts=2023-12-13T16:00:24.711575585Z caller=promtail.go:133 msg="Reloading configuration file" md5sum=169d8cc02367d196d7ba24694ea90c5d
level=info ts=2023-12-13T16:00:24.712927375Z caller=server.go:322 http=[::]:9080 grpc=[::]:32729 msg="server listening on addresses"
level=info ts=2023-12-13T16:00:24.713264874Z caller=main.go:174 msg="Starting Promtail" version="(version=, branch=, revision=unknown)"
level=warn ts=2023-12-13T16:00:24.713474024Z caller=promtail.go:263 msg="enable watchConfig"
level=info ts=2023-12-13T16:00:29.714075081Z caller=filetargetmanager.go:361 msg="Adding target" key="/var/log/**/*log:{job=\"varlogs\"}"
level=info ts=2023-12-13T16:00:29.716662721Z caller=filetarget.go:313 msg="watching new directory" directory=/var/log/loki
level=info ts=2023-12-13T16:00:29.716703277Z caller=filetarget.go:313 msg="watching new directory" directory=/var/log/promtail
level=info ts=2023-12-13T16:00:29.716812088Z caller=filetarget.go:313 msg="watching new directory" directory=/var/log/gitea
level=info ts=2023-12-13T16:00:29.716857702Z caller=filetarget.go:313 msg="watching new directory" directory=/var/log/grafana
level=info ts=2023-12-13T16:00:29.717132151Z caller=filetarget.go:313 msg="watching new directory" directory=/var/log/redis
level=info ts=2023-12-13T16:00:29.717201082Z caller=filetarget.go:313 msg="watching new directory" directory=/var/log/caddy
level=info ts=2023-12-13T16:00:29.717280021Z caller=filetarget.go:313 msg="watching new directory" directory=/var/log
ts=2023-12-13T16:00:29.720424446Z caller=log.go:168 level=info msg="Seeked /var/log/caddy/caddy.log - &{Offset:606936482 Whence:0}"
level=info ts=2023-12-13T16:00:29.720534804Z caller=tailer.go:145 component=tailer msg="tail routine: started" path=/var/log/caddy/caddy.log
ts=2023-12-13T16:00:29.720631282Z caller=log.go:168 level=info msg="Seeked /var/log/gitea/gitea.log - &{Offset:8194 Whence:0}"
level=info ts=2023-12-13T16:00:29.72066379Z caller=tailer.go:145 component=tailer msg="tail routine: started" path=/var/log/gitea/gitea.log
ts=2023-12-13T16:00:29.720728343Z caller=log.go:168 level=info msg="Seeked /var/log/grafana/grafana.log - &{Offset:38901 Whence:0}"
level=info ts=2023-12-13T16:00:29.720756759Z caller=tailer.go:145 component=tailer msg="tail routine: started" path=/var/log/grafana/grafana.log
[...]

洛基

洛基.yaml

auth_enabled: false

server:
  http_listen_port: 3100
  grpc_listen_port: 9096

common:
  instance_addr: 127.0.0.1
  path_prefix: /var/db/loki
  storage:
    filesystem:
      chunks_directory: /var/db/loki/chunks
      rules_directory: /var/db/loki/rules
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory

query_range:
  results_cache:
    cache:
      embedded_cache:
        enabled: true
        max_size_mb: 100

schema_config:
  configs:
    - from: 2020-10-24
      store: boltdb-shipper
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 24h

loki 应用程序/服务已启动并正在运行。我还可以在这里测试什么吗？

格拉法纳

创建的 loki 数据源：

• 类型：洛基
• URL：（http://127.0.0.1:3100 其他所有内容：保留默认设置）

即使没有任何过滤条件，仅查看 grafana/explore也不会显示任何数据。光是按Run query好像没有什么效果。

grafana 日志对我来说看起来不错：

尾部 /var/log/grafana/grafana.log

logger=grafana.update.checker t=2023-12-13T16:31:16.759743533+01:00 level=info msg="Update check succeeded" duration=486.885735ms
logger=cleanup t=2023-12-13T16:41:16.24610575+01:00 level=info msg="Completed cleanup jobs" duration=13.483743ms
logger=plugins.update.checker t=2023-12-13T16:41:16.617294501+01:00 level=info msg="Update check succeeded" duration=334.848905ms
logger=grafana.update.checker t=2023-12-13T16:41:16.657803737+01:00 level=info msg="Update check succeeded" duration=387.307337ms
logger=cleanup t=2023-12-13T16:51:16.240282272+01:00 level=info msg="Completed cleanup jobs" duration=7.750284ms
logger=grafana.update.checker t=2023-12-13T16:51:16.292110778+01:00 level=info msg="Update check succeeded" duration=21.261137ms
logger=plugins.update.checker t=2023-12-13T16:51:16.313163122+01:00 level=info msg="Update check succeeded" duration=31.295959ms
logger=cleanup t=2023-12-13T17:01:16.220217724+01:00 level=info msg="Completed cleanup jobs" duration=4.195714ms
logger=plugins.update.checker t=2023-12-13T17:01:16.312587838+01:00 level=info msg="Update check succeeded" duration=30.754461ms
logger=grafana.update.checker t=2023-12-13T17:01:16.333881656+01:00 level=info msg="Update check succeeded" duration=63.366615ms

感谢您指出这里可能缺少的内容。

Caddy v2.2.1 似乎不尊重给定地址的端口Caddyfile：

myfinedomain.com:9999 {
    respond "Nothing to see here. Srsly."
}

启动 caddy 只会导致以下错误（取自日志）：

{"level":"info","ts":1614777947.500496,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1614777947.5004952,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0002d2380"}
run: loading initial config: loading new config: http app module: start: tcp: listening on :80: listen tcp :80: bind: address already in use
start: caddy process exited with error: exit status 1
{"level":"info","ts":1614778157.9187949,"msg":"using provided configuration","config_file":"/usr/local/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"info","ts":1614778157.9210048,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["[::1]:2019","127.0.0.1:2019","localhost:2019"]}
{"level":"info","ts":1614778157.9213488,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1614778157.92195,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0002be3f0"}
run: loading initial config: loading new config: http app module: start: tcp: listening on :80: listen tcp :80: bind: address already in use
start: caddy process exited with error: exit status 1

因此，caddy 似乎尝试绑定到端口 80，这是不应该的。

该文档将端口列为地址的有效选项。

对这里可能缺少的东西的想法表示赞赏。

当我有一个具有多个控制节点的 kubernetes 集群并删除其中一个时，整个 API 服务器似乎不再可用。

在此设置中，我想从两个控制节点缩小到一个控制节点，但最终导致集群无法使用：

$ kubectl get nodes
NAME      STATUS   ROLES    AGE     VERSION
master1   Ready    master   5d20h   v1.18.6
worker1   Ready    <none>   5d19h   v1.18.6
master2   Ready    master   19h     v1.18.6
$ kubectl drain master2 --ignore-daemonsets
node/master2 cordoned
WARNING: ignoring DaemonSet-managed Pods: kube-system/calico-node-hns7p, kube-system/kube-proxy-vk6t7
node/master2 drained
$ kubectl get nodes
NAME      STATUS                     ROLES    AGE     VERSION
master1   Ready                      master   5d20h   v1.18.6
worker1   Ready                      <none>   5d20h   v1.18.6
master2   Ready,SchedulingDisabled   master   19h     v1.18.6
$ kubectl delete node master2
node "master2" deleted
$ kubectl get nodes
NAME      STATUS   ROLES    AGE     VERSION
master1   Ready    master   5d20h   v1.18.6
worker1   Ready    <none>   5d20h   v1.18.6
$ ssh master2
$ sudo kubeadm reset
[reset] Reading configuration from the cluster...
[reset] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
W0811 10:24:49.750898    7159 reset.go:99] [reset] Unable to fetch the kubeadm-config ConfigMap from cluster: failed to get node registration: failed to get corresponding node: nodes "master2" not found
[reset] WARNING: Changes made to this host by 'kubeadm init' or 'kubeadm join' will be reverted.
[reset] Are you sure you want to proceed? [y/N]: y
[preflight] Running pre-flight checks
W0811 10:24:51.487912    7159 removeetcdmember.go:79] [reset] No kubeadm config, using etcd pod spec to get data directory
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in "/var/lib/kubelet"
[reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
[reset] Deleting contents of stateful directories: [/var/lib/etcd /var/lib/kubelet /var/lib/dockershim /var/run/kubernetes /var/lib/cni]

The reset process does not clean CNI configuration. To do so, you must remove /etc/cni/net.d

The reset process does not reset or clean up iptables rules or IPVS tables.
If you wish to reset iptables, you must do so manually by using the "iptables" command.

If your cluster was setup to utilize IPVS, run ipvsadm --clear (or similar)
to reset your system's IPVS tables.

The reset process does not clean your kubeconfig files and you must remove them manually.
Please, check the contents of the $HOME/.kube/config file.
$ exit
$ kubectl get nodes
Error from server: etcdserver: request timed out
$ kubectl cluster-info

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
The connection to the server master1:6443 was refused - did you specify the right host or port?

这里缺少什么？或者，删除控制平面节点与删除工作节点有何不同？指针表示赞赏。