神秘的“无法找到可执行文件”

在这篇文章之后，我一直在使用 Promtail、Loki 和 Grafana 为各种服务器设置监控解决方案。我有一台运行 Loki 和 Grafana（在 Rocky Linux 9.3 上）的监控机器和一堆运行 Promtail 的 Ubuntu 服务器，它将日志铲入 Loki。效果很好。

然而，我想做的最后一步是为监控机器设置 Promtail。我按照上面的步骤操作 - 这些步骤在大约 20 台服务器上有效 - 突然间，我不断收到“无法找到可执行文件”的消息：

[root@localhost ~]# systemctl status promtail.service
× promtail.service - Promtail for Loki
     Loaded: loaded (/etc/systemd/system/promtail.service; enabled; preset: disabled)
     Active: failed (Result: exit-code) since Tue 2024-01-09 05:21:23 EST; 5s ago
   Duration: 22ms
    Process: 3633351 ExecStart=/usr/local/bin/promtail-linux-amd64 -config.file /etc/loki/promtail.yaml (code=exited, status=203/EXEC)
   Main PID: 3633351 (code=exited, status=203/EXEC)
        CPU: 21ms

Jan 09 05:21:23 localhost.localdomain systemd[1]: Started Promtail for Loki.
Jan 09 05:21:23 localhost.localdomain systemd[3633351]: promtail.service: Failed to locate executable /usr/local/bin/promtail-linux-amd64: Permission denied
Jan 09 05:21:23 localhost.localdomain systemd[3633351]: promtail.service: Failed at step EXEC spawning /usr/local/bin/promtail-linux-amd64: Permission denied
Jan 09 05:21:23 localhost.localdomain systemd[1]: promtail.service: Main process exited, code=exited, status=203/EXEC
Jan 09 05:21:23 localhost.localdomain systemd[1]: promtail.service: Failed with result 'exit-code'.

但可执行文件位于正确的位置，并且所有者是 promtail 用户：

[root@localhost ~]# ls -al /usr/local/bin/
total 165048
drwxr-xr-x+  3 root     root          128 Jan  2 12:35 .
drwxr-xr-x. 12 root     root          131 May 30  2023 ..
-rwxr-xr-x+  1 loki     loki     59424768 May  3  2023 loki-linux-amd64
-rw-r--r--+  1 root     root     18930096 May 31  2023 loki-linux-amd64.zip
-rwxr-xr-x.  1 root     root          233 Nov  6 11:53 normalizer
-rwxrwxr--+  1 promtail promtail 90640576 May  3  2023 promtail-linux-amd64
drwxr-xr-x.  7 root     root         4096 Jan  4 06:47 server_heartbeat

ACL 看起来像这样：

[root@localhost ~]# getfacl /usr/local/bin/promtail-linux-amd64 
getfacl: Removing leading '/' from absolute path names
# file: usr/local/bin/promtail-linux-amd64
# owner: promtail
# group: promtail
user::rwx
group::r-x
other::r--

服务文件指定 promtail 作为用户：

[Unit]

Description=Promtail for Loki

After=network.target

[Service]

Type=simple

User=promtail

ExecStart=/usr/local/bin/promtail-linux-amd64 -config.file /etc/loki/promtail.yaml

Restart=on-abort

NoNewPrivileges=true

PrivateTmp=yes

RestrictNamespaces=uts ipc pid user cgroup

ProtectKernelTunables=yes

ProtectKernelModules=yes

ProtectControlGroups=yes

#ProtectSystem=strict

#PrivateUsers=strict

#CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_DAC_READ_SEARCH

[Install]

WantedBy=multi-user.target

我缺少什么？

编辑：根据 @gerald-schneider 的评论，相关的 SELinux 上下文如下所示：

[root@localhost ~]# ls -Z /usr/local/bin/promtail-linux-amd64 
unconfined_u:object_r:admin_home_t:s0 /usr/local/bin/promtail-linux-amd64

[root@localhost ~]# ls -Z /etc/loki/promtail.yaml 
unconfined_u:object_r:admin_home_t:s0 /etc/loki/promtail.yaml

[root@localhost ~]# ls -Z /tmp/positions.yaml 
unconfined_u:object_r:user_tmp_t:s0 /tmp/positions.yaml