主页 / unix / 问题 / 787020
Accepted
wobtax
Asked: 2024-11-22 05:36:00 +0800 CST

仅在匹配的行上运行程序

  • 772

假设我有一个程序blackbox和一个包含以下内容的文件：

in this file
  this line contains =TAG=
    so does =TAG= this one
      as =TAG= does this other line
    this line does not
nor does this line
  =TAG= here again
    gone again

我如何blackbox仅在包含的行上运行=TAG=？

注 1：一种方法是使用while read循环，但这被认为是不好的做法。那么，什么是规范的、正确的方法来做到这一点（如果有的话）？

注 2：当然，如果我只是编辑文本，使用AWK或 的解决方案sed是合适的——但blackbox可能会产生预期的副作用。这个问题适用于我需要执行另一个进程的情况。

注 3：您可能会问，如果blackbox是类似nl或的情况，会发生什么情况sort— — 在多行上同时运行它与在每行上运行一个新进程的结果不同。在这种情况下，我希望能够通过以下三种方式中的每一种来做到这一点：

  • 按块：=TAG=用该块上的结果替换包含的每个连续行的块blackbox。

    预期输出为blackbox= nl：

    in this file
1  this line contains =TAG=
2    so does =TAG= this one
3      as =TAG= does this other line
    this line does not
nor does this line
1  =TAG= here again
    gone again

  • 逐行：将包含的每一行替换为该行=TAG=的结果。blackbox

    预期输出为blackbox= nl：

    in this file
1  this line contains =TAG=
1    so does =TAG= this one
1      as =TAG= does this other line
    this line does not
nor does this line
1  =TAG= here again
    gone again

  • 连续：将包含的所有行发送=TAG=给一个进程，并在接收下一个块blackbox之前用新打印的行替换每个块。blackbox

    预期输出为blackbox= nl：

    in this file
1  this line contains =TAG=
2    so does =TAG= this one
3      as =TAG= does this other line
    this line does not
nor does this line
4  =TAG= here again
    gone again

    （如果我们使用sort，所有匹配的行最终都会排在最后一个匹配的块中，因为它们直到最后才会被打印。）

我在这里没有发现任何关于一般问题的问题，但这些都是该问题的特殊情况：

shell

2 个回答

  1. 这是执行您要求的第一件事的一种方法（按块执行），假设blackbox是nl：

    $ cat tst.sh
#!/usr/bin/env bash

awk '
    BEGIN { cmd = "nl" }
    /=TAG=/ {
        print | cmd
        next
    }
    {
        close(cmd)
        print
    }
' "${@:--}"


    $ ./tst.sh file
in this file
     1    this line contains =TAG=
     2      so does =TAG= this one
     3        as =TAG= does this other line
    this line does not
nor does this line
     1    =TAG= here again
    gone again

    如果您想要像第二个请求那样一次执行一行（逐行），只需移动close()：

    $ cat tst.sh
#!/usr/bin/env bash

awk '
    BEGIN { cmd = "nl" }
    /=TAG=/ {
        print | cmd
        close(cmd)
        next
    }
    {
        print
    }
' "${@:--}"


    $ ./tst.sh file
in this file
     1    this line contains =TAG=
     1      so does =TAG= this one
     1        as =TAG= does this other line
    this line does not
nor does this line
     1    =TAG= here again
    gone again

    如果您需要更好地控制外部命令的输出在整体输出中出现的位置，或者在打印之前需要任何其他附加处理，GNU awk 还支持协同处理。

    说到这......这里有一种使用 GNU awk 进行协同处理来连续实现你的第三个请求的方法：

    $ cat tst.sh
#!/usr/bin/env bash

awk '
    BEGIN { cmd = "nl" }
    { lines[NR] = $0 }
    /=TAG=/ {
        print |& cmd
        tbds[NR]
    }
    END {
        close(cmd, "to")
        for ( i=1; i<=NR; i++ ) {
            if ( i in tbds ) {
                if ( (cmd |& getline line) > 0 ) {
                    lines[i] = line
                }
            }
            print lines[i]
        }
        close(cmd)
    }
' "${@:--}"


    $ ./tst.sh file
in this file
     1    this line contains =TAG=
     2      so does =TAG= this one
     3        as =TAG= does this other line
    this line does not
nor does this line
     4    =TAG= here again
    gone again

    这假设您的输入不是太大以至于无法放入内存。如果不是这种情况，那么您可以使用 2 遍方法执行相同的操作，即读取输入两次，而不是将其存储在数组中lines[]。

    如果管道已满，上述操作可能会中断（感谢@Stéphane Chazelas 指出这一点），如果您的系统已满stdbuf，那么您可以执行以下操作以使nl输出行缓冲：

    $ cat tst.sh
#!/usr/bin/env bash

awk '
    BEGIN { cmd = "stdbuf -oL nl" }
    /=TAG=/ {
        print |& cmd
        if ( (cmd |& getline line) > 0 ) {
            $0 = line
        }
    }
    { print }
' "${@:--}"


    $ ./tst.sh file
in this file
     1    this line contains =TAG=
     2      so does =TAG= this one
     3        as =TAG= does this other line
    this line does not
nor does this line
     4    =TAG= here again
    gone again

    使用非 GNU awk，您可以写入临时文件而不是通过管道传输到进程，然后从临时文件读取而不是从进程读取，例如：

    $ cat tst.sh
#!/usr/bin/env bash

tmp=$(mktemp) || exit 1
trap 'rm -f "$tmp"; exit' EXIT

tmp="$tmp" awk '
    BEGIN {
        tmp = ENVIRON["tmp"]
        cmd = "nl > \047" tmp "\047"
    }
    { lines[NR] = $0 }
    /=TAG=/ {
        print | cmd
        tbds[NR]
    }
    END {
        close(cmd)
        for ( i=1; i<=NR; i++ ) {
            if ( i in tbds ) {
                if ( (getline line < tmp) > 0 ) {
                    lines[i] = line
                }
            }
            print lines[i]
        }
    }
' "${@:--}"


    $ ./tst.sh file
in this file
     1    this line contains =TAG=
     2      so does =TAG= this one
     3        as =TAG= does this other line
    this line does not
nor does this line
     4    =TAG= here again
    gone again

    或者，再次stdbuf以与我们对协同处理解决方案相同的方式使用。

    如果您正在考虑使用，getline请务必先阅读http://awk.freeshell.org/AllAboutGetline。

    • 3
  2. Best Answer

    和perl：

    $ perl -ne '
  if (/=TAG=/) {
    STDOUT->flush;
    open $cmd, "|-", "nl" unless $cmd;
    print $cmd $_;
  } else {
    close $cmd;
    undef $cmd;
    print;
  }' file
in this file
     1    this line contains =TAG=
     2      so does =TAG= this one
     3        as =TAG= does this other line
    this line does not
nor does this line
     1    =TAG= here again
    gone again

    $ perl -ne '
  if (/=TAG=/) {
    STDOUT->flush;
    open $cmd, "|-", "nl";
    print $cmd $_;
    close $cmd;
  } else {
    print;
  }' file
in this file
     1    this line contains =TAG=
     1      so does =TAG= this one
     1        as =TAG= does this other line
    this line does not
nor does this line
     1    =TAG= here again
    gone again

    $ perl -MTime::HiRes=usleep -ne '
  if (/=TAG=/) {
    STDOUT->flush;
    unless ($cmd) {
      open $cmd, "|-", "stdbuf", "-oL", "nl";
      $cmd->autoflush;
    }
    print $cmd $_;
    usleep(10000)
  } else {print}' file
in this file
     1    this line contains =TAG=
     2      so does =TAG= this one
     3        as =TAG= does this other line
    this line does not
nor does this line
     4    =TAG= here again
    gone again

    后者假设命令不缓冲其输出（我们避免使用nl--actually ask for Line output buffering Instead-- as stdbuf -oLfound on GNU or FreeBSD systems），并且在读取一行输入后，在接下来的 10,000 微秒（0.1 秒）内产生相应的输出行。

    此处，命令在代码中被硬编码为"nl"或"stdbuf", "-oL", "nl"（您也可以写为qw(stdbuf -oL nl)）perl。对于要作为额外参数传递的命令及其参数，perl改为：

    perl -ne '
  BEGIN {@cmd = @ARGV; @ARGV = ()}
  # rest of the code where the open call would become:
  open $cmd, "|-", @cmd
  ' stdbuf -oL nl < file

    除了使用简单命令外，您还可以使用上面的代码将其sh -c 'any shell code'作为简单命令或使用open $cmd, "|any shell code"语法¹来使其成为任何 shell 代码

    ¹ 实际上，在 现代版本中perl，perl如果命令足够简单，将跳过执行 shell。

    • 1

相关问题