问题[bash](coding)

问题

我（拼命地）尝试在 podman 中复制此 mlflow/minio 设置：https://github.com/minio/blog-assets/blob/main/mlflow-minio-setup/docker-compose.yml。本教程在 VM 中使用 docker-compose 运行良好。

然而，在 podman 中，我根本无法弄清楚如何执行脚本以在 minio 中自动创建存储桶。我尝试了 --entrypoint 选项的许多可能组合，但总是失败：Error: crun: executable file [/bin/sh,-c,/create_bucket.sh] not found in $PATH: No such file or directory: OCI runtime attempted to invoke a command that was not found...

我的配置

创建存储桶：

#!/bin/sh
sleep 5;
/usr/bin/mc config host add mlflow_minio_storage http://mlflow_minio_storage:9000 "$MINIO_ACCESS_KEY" "$MINIO_SECRET_ACCESS_KEY" --api S3v4;
/usr/bin/mc ls mlflow_minio_storage | grep -q challenge || /usr/bin/mc mb mlflow_minio_storage/mlflow;
/usr/bin/mc policy download mlflow_minio_storage/mlflow;
exit 0;

Dockerfile：

FROM docker.io/minio/mc

COPY create_bucket.sh /create_bucket.sh
RUN chmod +x /create_bucket.sh

podman 运行：

podman build -t minio_client:latest -f Containerfile
podman run \
--name=mlflow_minio_client \
--detach \
--secret=MINIO_ACCESS_KEY,type=env,target=MINIO_ACCESS_KEY \
--secret=MINIO_SECRET_ACCESS_KEY,type=env,target=MINIO_SECRET_ACCESS_KEY \
--net mlflow_net \
--network-alias mlflow_minio_client \
--restart always \
--requires mlflow_minio_storage \
--entrypoint ["/bin/sh","-c","/create_bucket.sh"] \
localhost/minio_client:latest

最小示例

即使这种简单的情况也会导致相同的错误。我是不是误解了什么？

podman run \
--entrypoint ["/bin/sh","-c","/usr/bin/mc version"] \
docker.io/minio/mc:latest
>>> Error: crun: executable file `[/bin/sh,-c,/usr/bin/mc version]` not found in $PATH: No such file or directory: OCI runtime attempted to invoke a command that was not found

我试图理解为什么我似乎无法成功实现用于flock避免竞争条件的计数器。

我flock从这个 SO 答案中复制了代码。

计数器似乎会随机重置，尽管我不明白为什么它会这样做。counter inc应该永远自动计数。

我的代码旨在让我能够使用同时处理多个文件GNU parallel，但是使用包含递增计数器的每个处理文件来更新状态行。

该计数器保存到由 创建的 RAM 中的临时文件中mktemp。

我在这里把它剪短了，所以它只是一个应该永远计数的无限循环，但它会不断重置。

谁能向我解释为什么计数器有时会重置？

#!/bin/bash

clear_line () {
    echo -ne "\r\033[K"
}

counter () {
  {
    flock -s 200
    read numfiles < "$countfile"
    if [ "$1" = "inc" ]; then
        ((numfiles++))
    fi
    if [ "$1" = "rst" ]; then
        numfiles=0
    fi
    if [ "$1" = "get" ]; then
        echo "$numfiles"
    fi
    echo "$numfiles" > "$countfile"
  } 200> "$countlockfile"
}

process () {
    counter inc
    currfileno=$(counter get)

    clear_line
    echo -n "$currfileno"
}

# from https://unix.stackexchange.com/a/29918/52247
tmpdir=
cleanup () {
  trap - EXIT
  if [ -n "$tmpdir" ] ; then rm -rf "$tmpdir"; fi
  if [ -n "$1" ]; then trap - $1; kill -$1 $$; fi
}
tmpdir=$(mktemp -dt "$(basename $0).XXXXXXXX" --tmpdir=/run/user/$(id -u)) || exit 1
countfile=$(mktemp -t "counter.XXXXXXXX" --tmpdir=$tmpdir) || exit 1
countlockfile=$(mktemp -t "countlockfile.XXXXXXXX" --tmpdir=$tmpdir) || exit 1

trap 'cleanup' EXIT
trap 'cleanup HUP' HUP
trap 'cleanup TERM' TERM
trap 'cleanup INT' INT

export -f clear_line
export -f process
export -f counter
export countfile
export countlockfile

counter rst
while :
do
    echo whatever
done | parallel process

我有这个脚本，它似乎printf '%s\0%s\0'以代码 1 退出，我不明白为什么。

我甚至强迫它使用 bash 版本 5.2 - 结果相同。

以下是示例脚本及其输出

#!/usr/bin/env bash
echo $BASH_VERSION
set -x
set -Eeuo pipefail

function prompt_creds {
  local username password
  read -rp "username " username
  read -rsp "password " password 
  printf '%s\0%s\0' "${username}" "${password}"
}

function main() {
  IFS=$'\0' read -r username password < <(prompt_creds )
  echo "name name is ${username}" "my password is ${password}"
}

main

使用和输出

./sample.sh 
3.2.57(1)-release
+ set -Eeuo pipefail
+ main
+ IFS=
+ read -r username password
++ prompt_creds
++ local username password
++ read -rp 'username ' username
username user
++ read -rsp 'password ' password
password ++ printf '%s\0%s\0' user pass
bash-5.2$ echo $?
1

我甚至尝试在 CLI 上简单运行它，并且它可以工作：

bash-5.2$ IFS=$'\0' read -r u p < <(printf '%s\0%s\0' user pass)
bash-5.2$ echo $u $p
userpass

我需要使用 bash 修改 csv 文件。

输入（csv 文件）：

firstletter="s"
surname="houston"
emaildomain"@zzz.com"
input=$(cat 1.csv)

1.csv：

1,1,Susan houston,Director of Services,,
2,1,Christina Gonzalez,Director,,
3,2,Brenda brown,"Director, Second Career Services",,

如何使用 Linux bash 在最后两个逗号之间添加文本？​​我尝试了以下方法：

for i in $(cat $input);do
        sed -i "s/,$/${firstletter}${surname}${emaildomain},/g" $i;
done

但是，这会导致错误： sed: -e expression #1, char 5: unterminated `s' command

预期输出：

1,1,Susan houston,Director of Services,[email protected],
2,1,Christina Gonzalez,Director,[email protected],
3,2,Brenda brown,"Director, Second Career Services",[email protected],

在 ubuntu 上我运行以下脚本（为了测试目的，该脚本应该随机失败或成功）：

if [ "$RANDOM" -gt 16384 ]; then
    echo "Success!"
    exit 0
else
    echo "Failure!"
    exit 1
fi

和

sh random.sh

但它失败了

random.sh: 1: [: Illegal number: 
Failure!

问题的原因是什么？

我正在尝试做以下事情：

文件1：

x.x.x.5
x.x.x.6

文件2：

router
switch

尝试循环遍历这两个文件来创建两个附加文件，每个文件在 file2 中的名称如下：

ping路由器

sudo /sbin/mtr -rn -c 30 --interval 1 -Z 1 x.x.x.6 | sed 's/Start: //g' |  sed -e '2,$ s/^/     /'

ping_交换机

sudo /sbin/mtr -rn -c 30 --interval 1 -Z 1 x.x.x.7 | sed 's/Start: //g' |  sed -e '2,$ s/^/     /'

脚本：

for x in $(cat file1); do; for z in $(cat file2); do; echo "sudo /sbin/mtr -rn -c 30 --interval 1 -Z 1 $x | sed 's/Start: //g' |  sed -e '2,$ s/^/     /'" >>mtr_$z.sh; done;done

^^ 它创建了两个文件，但包含两个 IP

我怎样才能实现上述目标？

!/bin/bash
cd /home/pi/dockertest
sudo -S <<< Adminpassword docker-compose up -d --scale chrome=2 --scale firefox=0

我想向 pm2 添加一个脚本来启动 SeleniumGrid。因此，我使用上述代码创建了 .sh 文件，并将其添加到 pm2 中

pm2 start '/home/user/dockertest/startGrid.sh' --name SeleniumGrid

Docker 和 Grid 现已启动，但在 pm2 monit 控制台中，我看到命令不断重复。如何防止这种情况？

在此处输入图片描述

在 bash 中，我尝试查找包含特定文件的目录；但是，这些目录位于不同的级别，例如，如果我正在寻找 dir03/myfile.txt：

dir01/dir02/dir03/myfile.txt
dir01/dir03/ ##no target file in it
dir01/dir04/dir05/dir03/myfile.txt
dir01/dir04/dir05/dir06/myfile.txt ##I do not want this

我可以这样做：

find ./dir01 -type -d -iname "dir03"

但随后我会得到所有目录，甚至没有目标文件的目录

如果我这样做：

find ./dir01/*/dir03 -iname "myfile.txt" 

我不会看到“dir01/dir04/dir05/dir03/myfile.txt”

这不起作用：

find ./dir01 -iname "dir03/myfile.txt"

##find: warning: Unix filenames usually don't contain slashes (though pathnames do).  
That means that '-iname ‘dir03/myfile.txt’' will probably evaluate to false all the time on this system.  You might find the '-wholename' test more useful, or perhaps '-samefile'.  Alternatively, if you are using GNU grep, you could use 'find ... -print0 | grep -FzZ ‘dir03/myfile.txt’'

我可以这样做：

find ./dir01 -iname "myfile.txt" -print0 | grep -FzZ 'dir03' | sed "s/myfile\.txt/\n/g"

它找到 myfile.txt 所在的所有匹配项，并仅获取目标目录中包含的匹配项；之后，我用新行拆分结果（上面的行给出了连接的结果，例如./dir01/dir02/dir03/myfile.txt./dir01/dir04/dir05/dir03/myfile.txt 但如果我有一个复杂的子目录结构，它可能会使搜索更通用、更长。我想知道是否有更简单的解决方案来告诉 find 在任何级别找到“dir03/myfile.txt”

请注意，目录名称仅供参考，它们不共享相同的前缀。

编辑：我想我可以做到

find ./dir01 -iname "myfile.txt" > myres0.txt
grep 'dir03' myres0.txt > myres1.txt

考虑到我正在寻找多个父目录而不仅仅是 dir03，它只会进行一次搜索，然后我可以使用包含目标目录的 txt 文件并执行以下操作：

grep -f mytargetDirs.txt myres0.txt > myres1.txt

仍然想知道是否有更好的方法。

鉴于：

MY_JSON=$(cat <<EOF
{
  "schema": "my.schema",
  "properties": [
    {
      "type": "new.data",
      "value": {
        "labels": {
          "ofasd.io/arch.amd64": "supported",
          "ofasd.io/arch.arm64": "supported",
          "ofasd.io/arch.ppc64le": "supported",
          "ofasd.io/arch.s390x": "unsupported"
        }
      }
    }
  ]
}
EOF
)

我正在尝试仅过滤“受支持的”架构：

arches=$(echo "$MY_JSON" | tr -d '\000-\031' | jq -r '
.properties[] 
| select(.type == "new.data") 
| .value.labels 
| to_entries 
| map(select(.key | test("^ofasd\\.io/arch\\.") and .value == "supported")) 
| map(.key | sub("^ofasd\\.io/arch\\."; "")) 
| join("\n")')

预期输出：

amd64
arm64
ppc64le

实际产量：

jq: error (at <stdin>:0): Cannot index string with string "value"

有人能帮我理解我做错了什么吗？问题似乎是 ofasd.io/arch.* 过滤不起作用。当我运行：

echo "$MY_JSON" | jq -r '
.properties[] 
| select(.type == "new.data") 
| .value.labels 
| to_entries 
| map(select(.key | test("^ofasd.io/arch\\.") and .value == "supported"))'

我得到：

echo "$MY_JSON" | jq -r '' failed with status 5

我写了一个bash脚本：

• 获取最新的GithubCI/CD 运行
• 检查下载的工件的摘要
• 如果摘要发生变化，则生成一个进程来（并行）下载和提取每个工件
• 等待下载和提取过程完成，然后重新运行脚本

我遇到了一个问题 - 脚本可以毫无问题地生成并行任务，但它每 10 秒执行一次。出于某种原因，wait没有得到尊重。

为什么会这样？我该如何解决？

这是我的脚本：

#!/bin/bash

# GitHub repository details
REPO="RiceKX/Hydra"            # Format: owner/repo, e.g., "octocat/Hello-World"
BRANCH="development"           # The branch you want to check for the latest build
GITHUB_TOKEN="${GITHUB_TOKEN:-$(cat /path/to/your/token)}"  # GitHub token with necessary permissions
GITHUB_API="https://api.github.com"
ARTIFACTS_DIR="latest_dev_build"    # Directory to store downloaded artifacts

# Ensure the artifacts directory exists
mkdir -p "$ARTIFACTS_DIR"

# Run the script in an infinite loop
while true; do
    # Check if repository, branch, and GitHub token are provided
    if [ -z "$REPO" ] || [ -z "$BRANCH" ] || [ -z "$GITHUB_TOKEN" ]; then
      echo "Please set the repository, branch, and GitHub token in the script."
      exit 1
    fi

    # Step 1: Fetch the latest successful run for the specified branch
    echo "Fetching the latest successful build for branch '$BRANCH'..."

    RUN_ID=$(curl -s \
      -H "Authorization: token $GITHUB_TOKEN" \
      "$GITHUB_API/repos/$REPO/actions/runs?branch=$BRANCH&status=success&per_page=1" | \
      jq -r '.workflow_runs[0].id')

    if [ "$RUN_ID" == "null" ] || [ -z "$RUN_ID" ]; then
      echo "No successful builds found for branch '$BRANCH'."
      exit 1
    fi

    echo "Latest successful run ID: $RUN_ID"

    # Step 2: Fetch the artifacts associated with this run
    echo "Fetching artifacts for the latest run..."

    ARTIFACTS=$(curl -s \
      -H "Authorization: token $GITHUB_TOKEN" \
      "$GITHUB_API/repos/$REPO/actions/runs/$RUN_ID/artifacts")

    # Check if we successfully got the artifacts
    if [ -z "$ARTIFACTS" ] || [ "$(echo "$ARTIFACTS" | jq '.artifacts | length')" -eq 0 ]; then
      echo "No artifacts found for this run."
      exit 1
    fi

    # Step 3: Extract artifact details and download them in parallel
    PIDS=()  # Store background process PIDs to wait for them later
    echo "$ARTIFACTS" | jq -r '.artifacts[] | "\(.name) \(.archive_download_url) \(.digest)"' | \
    while read ARTIFACT_NAME ARTIFACT_DOWNLOAD_URL ARTIFACT_DIGEST; do
        if [ -z "$ARTIFACT_NAME" ] || [ -z "$ARTIFACT_DOWNLOAD_URL" ] || [ -z "$ARTIFACT_DIGEST" ]; then
          echo "Error: Artifact name, download URL, or digest missing for artifact."
          continue
        fi

        ARTIFACT_PATH="$ARTIFACTS_DIR/$ARTIFACT_NAME.zip"

        # Check if the artifact already exists
        if [ -f "$ARTIFACT_PATH" ]; then
            # Calculate the local digest of the downloaded artifact
            LOCAL_DIGEST=$(sha256sum "$ARTIFACT_PATH" | cut -d' ' -f1)

            # If the local digest matches the GitHub digest, skip downloading
            if [ "$LOCAL_DIGEST" == "$ARTIFACT_DIGEST" ]; then
                echo "Artifact $ARTIFACT_NAME is already up to date (digest matches), skipping download."
                continue
            else
                echo "Artifact $ARTIFACT_NAME exists but the digest does not match. Re-downloading..."
            fi
        fi

        # Download and extract each artifact in parallel
        (
            echo "Downloading and extracting artifact: $ARTIFACT_NAME from $ARTIFACT_DOWNLOAD_URL"

            curl -L -o "$ARTIFACT_PATH" \
              -H "Authorization: token $GITHUB_TOKEN" \
              "$ARTIFACT_DOWNLOAD_URL"
            
            if [ $? -eq 0 ]; then
                echo "Artifact $ARTIFACT_NAME downloaded successfully."

                # Extract the downloaded artifact
                unzip -o "$ARTIFACT_PATH" -d "$HOME"
                echo "Artifact $ARTIFACT_NAME extracted to $HOME."
            else
                echo "Failed to download $ARTIFACT_NAME."
            fi
        ) &  # Run this process in the background

        PIDS+=($!)
    done

    # Wait for all background processes to finish before continuing
    echo "Waiting for background processes to finish..."
    for PID in "${PIDS[@]}"; do
        wait "$PID"
    done

    echo "All artifacts downloaded and extracted successfully."

    # Sleep for 10 seconds before running the script again
    echo "Waiting for 10 seconds before checking again..."
    sleep 10
done