Curious Sam's questions

Ubuntu 16.04 GNU bash，版本 4.4.0

我想格式化这个包含 1,264,524 行的邮政编码归档的文本文件。

#!/bin/bash

wget http://download.geonames.org/export/zip/allCountries.zip
unzip allCountries.zip
mv -f allCountries.txt .allCountries.txt
rm -f allCountries.zip

这是未格式化的文件的一部分。

AD AD100 Canillo                    42.5833  1.6667   6
AD AD200 Encamp                     42.5333  1.6333   6
AD AD300 Ordino                     42.6  1.55  6
AD AD400 La Massana                    42.5667  1.4833   6
AD AD500 Andorra la Vella                    42.5  1.5   6
AD AD600 Sant Julià de Lòria                    42.4667  1.5   6
AD AD700 Escaldes-Engordany                     42.5  1.5667   6
AR 3636  POZO CERCADO (EL CHORRO (F), DPTO. RIVADAVIA (S))  Salta A              -23.4933 -61.9267 3
AR 4123  LAS SALADAS Salta A              -25.7833 -64.5 4
AR 4126  BARADERO Salta A              -26.0833 -65.263  3
AR 4126  EL CUIBAL   Salta A              -26.0833 -65.263  3
AR 4126  LA ASUNCION Salta A              -26.0833 -65.263  3
AR 4126  MIRAFLORES (TALA, DPTO. CANDELARIA) Salta A              -26.0833 -65.263  3
AR 4141  TOLOMBON Salta A              -26.2 -65.9167 4
AR 4141  QUISCA GRANDE  Salta A              -26.4367 -65.97   3
AR 4141  LA CIENEGUITA  Salta A              -26.4367 -65.97   3
AR 4141  MACHO RASTROJO Salta A              -26.4367 -65.97   3
AR 4190  ROSARIO DE LA FRONTERA  Salta A              -25.8 -64.9667 4
AR 4190  OVANDO   Salta A              -25.8 -65.1 4
AR 4190  SAN ESTEBAN Salta A              -25.8 -65.0333 3
AR 4190  LA BANDA (R. DE LA FRONTERA, DPTO. ROSARIO DE LA FRONTERA)  Salta A              -25.8 -65.0333 3
AR 4190  LA MATILDE  Salta A              -25.8 -65.0333 3
AR 4190  LAS PIEDRITAS  Salta A              -25.8 -65.0333 3
AR 4190  LOS POCITOS Salta A              -25.8 -65.0333 3
AR 4190  OJO DE AGUA (ROSARIO DE LA FRONTERA, DPTO. R.DE LA FRONTERA)   Salta A              -25.8 -65.0333 3
AR 4190  POTRERILLO (R. DE LA FRONTERA, DPTO. ROSARIO DE LA FRONTERA)   Salta A              -25.8 -65.0333 3

这应该是最终结果：

AD AD100 Canillo                                                                          42.5833     1.6667      6
AD AD200 Encamp                                                                           42.5333     1.6333      6
AD AD300 Ordino                                                                           42.6        1.56        6
AD AD400 La Massana                                                                       42.5667     1.4833      6
AD AD500 Andorra la Vella                                                                 42.5        1.6         6
AD AD600 Sant Julià de Lòria                                                              42.4667     1.5         6
AD AD700 Escaldes-Engordany                                                               42.5        1.5667      6
AR 3636  POZO CERCADO (EL CHORRO (F), DPTO. RIVADAVIA (S))  Salta A                       -23.4933    -61.9267    3
AR 4123  LAS SALADAS Salta A                                                              -25.7833    -64.5       4
AR 4126  BARADERO Salta A                                                                 -26.0833    -65.263     3
AR 4126  EL CUIBAL   Salta A                                                              -26.0833    -65.263     3
AR 4126  LA ASUNCION Salta A                                                              -26.0833    -65.263     3
AR 4126  MIRAFLORES (TALA, DPTO. CANDELARIA) Salta A                                      -26.0833    -65.263     3
AR 4141  TOLOMBON Salta A                                                                 -26.2       -65.9167    4
AR 4190  OJO DE AGUA (ROSARIO DE LA FRONTERA, DPTO. R.DE LA FRONTERA)   Salta A           -25.8       -65.0333    3
AR 4190  POTRERILLO (R. DE LA FRONTERA, DPTO. ROSARIO DE LA FRONTERA)   Salta A           -25.8       -65.0333    3

所以从左边第 3 列到最后一列的第 30 个制表符或 88 个空格应该开始。最后 3 列的第一个字符相隔 12 个空格。

每行都有数据，所以我尝试将最后 3 列剪切到另一个文件中。然后我尝试从原始文件中删除所有空格，然后将其格式化为列。第三列给我带来了麻烦，因为如果这样做

awk 'BEGIN{ OFS="\t"}{ print $1, $2, NR }' .allCountries.txt

AT      4873    Pehigen 34069
AT      4873    Hofberg 34070
AT      4873    Wiederhals      34071
AT      4873    Oberedt 34072
AT      4873    Oberegg 34073
AT      4873    Raitenberg      34074
AT      4873    Redltal 34075
AT      4873    Friedhalbing    34076
AT      4873    Unterhaselbach  34077
AT      4873    Redltal 34078
AT      4873    Erkaburgen      34079
AT      4873    Mayrhof 34080
AT      4873    Erdpries        34081
AT      4873    Grünbergsiedlung        34082
AT      4873    Brunnhölzl      34083
AT      4873    Seibrigen       34084
AT      4873    Kinast  34085
AT      4873    Stöckert        34086
AT      4873    Frankenburg     34087
AT      4873    Fischeredt      34088
AT      4873    Marigen 34089
AT      4873    Oberhaselbach   34090
AT      4873    Ottokönigen     34091
AT      4873    Fischigen       34092
AT      4873    Endriegl        34093

任何帮助都会很重要。

• Ubuntu 16.04
• 重击版本 4.4.0
• nginx版本：nginx/1.14.0

如何在 Bash 脚本中测试 Nginx 配置文件？目前我在 shell 中使用 -t ：

$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

但我想在脚本中做到这一点？

Ubuntu 16.04

bash -version  
GNU bash, version 4.4.0(1)-release (x86_64-unknown-linux-gnu)

我想要grep2 种模式，然后将它们并排列出。目前，这就是我所拥有的：

root@tires ~ # grep -e tire_id -e appID /path/to/*/vehicle/production.json
/path/to/000001_000002/vehicle/production.json:    "tire_id": "1305436516186552",
/path/to/000001_000002/vehicle/production.json:        "appID": "1164562920689523",
/path/to/000001_000079/vehicle/production.json:    "tire_id": "1815123428733289",
/path/to/000001_000079/vehicle/production.json:        "appID": "18412365908966538",
/path/to/000001_000088/vehicle/production.json:    "tire_id": "138477888324",

这就是我想要的，尽管任何类似的东西实际上都可以。

root@tires ~ # grep -e tire_id -e appID /path/to/*/vehicle/production.json
/path/to/000001_000002/vehicle/production.json:    tire_id: 1305436516186552, appID: 1164562920689523
/path/to/000001_000079/vehicle/production.json:    tire_id: 1815123428733289, appID: 18412365908966538

此处的文件示例：

{
    "socal": "https://xxx.xxxxx.xxx",
    "ip": "xxx.xxx.xxx.xxx",
    "tire_id": "213275925375485",
    "client": {
        "platform": "xx",
        "clientID": "xxxxx",
        "serviceID": "xxxxx",
        "service_id": XXXX,
        "vendor": "default"
    },
    "locale": "en_US",
    "cdc": {
        "appID": "233262274090443",
        "isdel": "ORdiZBMAQS2ZBCnTwZDZD",
    },
    "attachments": {
        "output": "attachments",
        "public": false,
    },
}

Ubuntu 16.04

我想备份所有 .txt、.cfg 和 .ini 文件，同时将它们的文件夹结构保留为备份文件夹中的 zip 文件。

我知道有一种更简单的方法可以完成整个操作，但这就是我所拥有的。

#!/bin/bash

wdir="/home/files"
backup_dir="/home/files/backup"
allcfg=$(find suan -name '*.cfg')
alltxt=$(find suan -name '*.txt')
allini=$(find suan -name '*.ini')
timeStamp="$(date +%Y--%b-%d--%k:%M--%P)"
backupfilename="${backup_dir}/Backup-${timeStamp}.zip"

#-- set some arrays ...
backupfiles=( "$allcfg" "$alltxt" "$allini" )

cd "$wdir"
zip -r "$backupfilename" "$backupfiles"

所以我可以看到它归档的文件的一个很好的输出，但是当它到达压缩部分时它会出错。

zip error: Nothing to do! (try: zip -r /home/files/backup/Backup-2018--Jul-04--21:37--pm.zip . -i suan/cfg/360controller.cfg

但这也行不通。

我正在尝试从 csv 文件的第 31 列中删除第一个值。这些列用引号括起来，并用逗号分隔，就像这样。

"first_name","last_name","home_address"
"Frank",Russel","123 Hampton road"

第 31 列是图像列，图像用逗号分隔

"http://ddd.com/124dfr3f.jpg,http://ddd.com/hjasgx37trg.jpg"

必须从第 31 列中删除第一个图像和尾部逗号

添加- 这是一个示例。

"one","two","three","four","five","six","seven","eight","nine","ten","eleven","twelve","three","four","five","six","seven","eight","nine","ten","twenty-one","twenty-two","three","four","five","six","seven","eight","nine","ten","thirty-one","thirty-two","three","four","five","six","seven","eight","nine","forty"
"one","two","three","four","five","six","seven","eight","nine","ten","eleven","twelve","three","four","five","six","seven","eight","nine","ten","twenty-one","twenty-two","three","four","five","six","seven","eight","nine","http://pictures.domain.com/e/josequervo/1183/eeeffcc7927sssssssssss5d26c0eb19e1dc53548x.jpg","http://pictures.domain.com/e/josequervo/0083/eeef8cc7927bf1245d26c0eb19e1dc53545x.jpg,http://pictures.domain.com/e/josequervo/0099/eeefscc7927bf1245d26c0eb19e1dc53546x.jpg,http://pictures.domain.com/e/josequervo/0012/eeefdcc7927bf1245d26c0eb19e1dc53547x.jpg,http://pictures.domain.com/e/josequervo/1183/eeeffcc7927bf1245d26c0eb19e1dc53548x.jpg,http://pictures.domain.com/e/josequervo/1234/eeefgcc7927bf1245d26c0eb19e1dc53549x.jpg,http://pictures.domain.com/e/josequervo/asee/eeefhcc7927bf1245d26c0eb19e1dc53540x.jpg,http://pictures.domain.com/e/josequervo/2341/eeefjcc7927bf1245d26c0eb19e1dc53544x.jpg","two",three","four","five","six","seven","eight","nine","forty"

此网址和逗号必须删除

http://pictures.domain.com/e/josequervo/0083/eeef8cc7927bf1245d26c0eb19e1dc53545x.jpg,

我只发布了 csv 文件的前 2 行，但该文件是 300 - 400 行，并且必须从第 2 行删除该图像到最后一行。我们不能改变第一行。

Ubuntu 16.04

我习惯于使用这个命令行来杀死任何进程匹配说 COMMAND

for i in $(ps -ef | grep COMMAND | awk '{ print $2 }'); do kill -9 "$i" &>/dev/null;done

shellcheck 不太喜欢它并建议

root@singa /home # shellcheck run

In run line 4:
for i in $(ps -ef | grep COMMAND | awk '{ print $2 }'); do kill -9 "$i" &>/dev/null;done
           ^-- SC2009: Consider using pgrep instead of grepping ps output.

在这里完成此操作需要哪些标志来替换 XXX

for i in $(pgrep XXX | grep COMMAND | awk '{ print $2 }'); do kill -9 "$i" &>/dev/null;done

我已经检查了 pgrep 的手册页。

Ubuntu 16.04 GNU bash，版本 4.3.48(1)-release (x86_64-pc-linux-gnu)

我们必须每天向服务器添加几次 ips。我发现自己根据 ip 块创建了同一文件的不同变体。

#-- 100.100.100.160/29 - Sun 04/22/18 02-38-pm
#-- -----------------------------------------
auto eth0:0
iface eth0:0 inet static
    address 100.100.100.160
    netmask 255.255.255.255

auto eth0:1
iface eth0:1 inet static
    address 100.100.100.161
    netmask 255.255.255.255

auto eth0:2
iface eth0:2 inet static
    address 100.100.100.162
    netmask 255.255.255.255

auto eth0:3
iface eth0:3 inet static
    address 100.100.100.163
    netmask 255.255.255.255

auto eth0:4
iface eth0:4 inet static
    address 100.100.100.164
    netmask 255.255.255.255

auto eth0:5
iface eth0:5 inet static
    address 100.100.100.165
    netmask 255.255.255.255

auto eth0:6
iface eth0:6 inet static
    address 100.100.100.166
    netmask 255.255.255.255

auto eth0:7
iface eth0:7 inet static
    address 99.100.100.167
    netmask 255.255.255.255

所以我想创建一个脚本让我尽可能接近那个文件，所以我写了一个脚本，这里是它的一部分......

#!/bin/bash
#
#-- bash add_ips.sh "eth0" "100.100.100.160" "255.255.255.255" "29"

wDir="/scripts/tools/ips"
ifaceFile="/etc/network/interfaces"
ipFile="${wDir}/ipFile.txt"
nic="${1}"
address="${2}"
netmask="${3}"
block="${4}"
timeStamp="$(date '+%a %D %I-%M-%P')"

if [ ! -n "$4" ]; then
   echo 'add_ips.sh nic address mask block ...';
   exit 1;
fi

#-- echo the address block on a header line
{
   echo "";
   echo "#-- IP Block ${address}/${block} - ${timeStamp}";
   echo "#-- ----------------------------------------------";
} > "$ipFile"

#-- If a 30/block
if [[ "$block" == "30" ]]; then
   start="0"
   end="3"
   for ipnum in $(seq "$start" "$end"); do
      {
         echo "auto ${nic}:${ipnum}";
         echo "iface ${nic}:${ipnum} inet static";
         echo -e "\t address ${address}";
         echo -e "\t netmask ${netmask}";
         echo "";
      } >> "$ipFile"
   done

输出接近于我正在寻找的内容，剩下要做的就是更改 IP 地址的最后一个数字。

#-- 100.100.100.160/29 - Sun 04/22/18 02-38-pm
#-- -----------------------------------------
auto eth0:0
iface eth0:0 inet static
    address 100.100.100.160
    netmask 255.255.255.255

auto eth0:1
iface eth0:1 inet static
    address 100.100.100.160
    netmask 255.255.255.255

auto eth0:2
iface eth0:2 inet static
    address 100.100.100.160
    netmask 255.255.255.255

如何在该行上添加当前循环的 $seq 值时更改 ip 地址的最后一个八位字节？

我试过这个

#-- If a 30/block
if [[ "$block" == "30" ]]; then
   start="0"
   end="3"
   for ipnum in $(seq "$start" "$end"); do
      ipaddress=$(expr "$address" + "$ipnum")
      {
         echo "auto ${nic}:${ipnum}";
         echo "iface ${nic}:${ipnum} inet static";
         echo -e "\t address ${ipaddress}";
         echo -e "\t netmask ${netmask}";
         echo "";
      } >> "$ipFile"
   done
fi

地址不是整数，所以它失败了。

Ubuntu 16.04

我想将原始命令回显到日志文件。我的日志文件应如下所示：

Mon 04/16/18 04-24-pm - Executing command: sudo /home/editini.sh "one" "two" "three" "four" "five" "six"

在满足拼写检查的同时完成此任务的最简单方法是什么。

#!/bin/bash


hello="${1}"
my="$2"
friend="$3"
are="$4"
you="$5"
safe="$6"
timeStamp="$(date '+%a %D %m-%S-%P')"

rm -rf report*; touch report.txt;
{
   echo "$timeStamp - Executing command: sudo /home/editini.sh \"$hello\" \"$my\" \"$friend\" \"$are\" \"$you\" \"$safe\""
   echo ""
   echo "$timeStamp" -  Executing command: sudo /home/editini.sh "$*"
   echo "";
   echo "$timeStamp" -  Executing command: sudo /home/editini.sh \""$*"\"
   echo "";
} > report.txt
cat report.txt

我不能使用第一行，因为我必须提前知道这些论点。

这就是我运行上述命令时控制台所说的内容。

root@me /home/scripts/vent-commands # sh one.sh one two three four five six
Mon 04/16/18 04-05-pm - Executing command: sudo /home/editini.sh "one" "two" "three" "four" "five" "six"

Mon 04/16/18 04-05-pm - Executing command: sudo /home/editini.sh one two three four five six

Mon 04/16/18 04-05-pm - Executing command: sudo /home/editini.sh "one two three four five six"

这是一个两部分的问题。

场景：这个脚本在一个 cronjob 上。如果文件夹不存在，系统会向我们发送一封电子邮件，其中会打开一张票，通知我们该文件夹不可用。我们必须手动登录并删除前面的文件夹 atm。

我希望我们能够手动执行脚本并通过按“Y”删除前面的文件夹，或者在我们登录并手动执行脚本时按“Enter”键继续。

这是我到目前为止...

   #-- check to see if cache folder exists
   { echo "Checking to see if ...";
   echo "${wDir}/${client%/}/.ftp-vendor-scripts/cache exists ... "; echo ""; } >> "$log"

   if [ ! -d "${wDir}"/"${client%/}"/.ftp-vendor-scripts/cache ]; then
      echo "Directory - ${wDir}/${client%/}/.ftp-vendor-scripts/cache DOES NOT exists - Failed ..." >> "$log";

      if [ ******** this script is being executed manually ******* ]; then
         echo "Would you like to delete the ${wDir}/${client%/}/.ftp-vendor-scripts folder?"
         echo "Press \"Y\" to delete the ${wDir}/${client%/}/.ftp-vendor-scripts."
         echo "Press \"Enter\" to continue without deleting the .ftp-vendor-scripts folder."
      else
         echo "Directory - ${wDir}/${client%/}/.ftp-vendor-scripts/cache DOES NOT exists - Failed ..." | mail -s "${wDir}/${client%/}/.ftp-vendor-scripts/ca$
      fi

   else
      echo "Directory - ${wDir}/${client%/}/.ftp-vendor-scripts/cache exists - Success ..." >> "$log";
   fi

Ubuntu 16.04 - 尝试微调

这是我的代码

#!/bin/bash

# if the .md5sum file doesn't exists
# or if the .md5sum file exists && doesn't match, recreate it
# but if the .md5sum file exists && matches then break and log

csvfile=inventory.csv

if [ ! -f .md5sum ] || [ -f .md5sum ] && [ ! md5sum -c .md5sum >/dev/null ]; then
   md5sum "$csvfile" > .md5sum
else
   echo "$csvfile file matched md5sum ..."
   break
fi

我在构建会导致新的 .md5sum 的条件语句时遇到问题。我试图找出条件语句的哪一部分是错误的，这就是 shellcheck 告诉我的。

root@me ~ # shellcheck run.sh

In run.sh line 8:
if [ ! -f .md5sum ] || [ -f .md5sum ] && [ ! md5sum -c .md5sum >/dev/null ]; then
^-- SC1009: The mentioned parser error was in this if expression.
                                         ^-- SC1073: Couldn't parse this test expression.
                                                               ^-- SC1072: Expected "]". Fix any mentioned problems and try again.

我可以通过一些比较来完成整个过程，但我认为如果我加入比较，我会获得一个更快、更清晰的脚本。

我还检查了 Jesse_P 的代码

#!/bin/bash

csvfile=inventory.csv

echo "hello" > inventory.csv
md5sum "$csvfile" > .md5sum
echo "well hello" > inventory.cvs

if [[ ! -f .md5sum ]] || [[ -f .md5sum && ! md5sum -c .md5sum >/dev/null ]]; then
        echo "All conditiions have been met to generate a new md5sum for inventory.csv ..."
        md5sum inventory.csv > .mds5sum
fi
exit 0

然后我用了shellcheck

root@0003 ~ # shellcheck run.sh
    In run.sh line 8:
    if [[ ! -f .md5sum ]] || [[ -f .md5sum && ! md5sum -c .md5sum >/dev/null ]]; then
    ^-- SC1009: The mentioned parser error was in this if expression.
                             ^-- SC1073: Couldn't parse this test expression.
                                                                  ^-- SC1072: Expected "]". Fix any mentioned problems and try again.

Ubuntu 16.04

即使 for 的输出client in */; do不会产生斜杠，如果我在循环内对文件执行时间测试时回显变量 $client ，也会出现斜杠。

cd "$wDir"
for client in */; do
   cd "$wDir"/"$client";

   #-- check to see if any .csv files exists
   if ls *.csv &>/dev/null; then
      for csvfile in *.csv; do
         if test $(find "$csvfile" -mmin +2880); then
            echo "$client has files older than 2 days ..." >> "staleFtpAccts"
         fi
      done
   fi
done

当我执行脚本时，在 $client 变量之后放置一个 / ，如下所示：

root@me /home/frank # bash script.sh
Start ...
000029_000020/ has files older than 2 days ...
#--
Finished ...

Start ...
000033_000040/ has files older than 2 days ...
#--
Finished ...

Start ...
000033_000041/ has files older than 2 days ...
#--
Finished ...

Start ...
000033_000042/ has files older than 2 days ...
#--
Finished ...

Start ...
000033_000043/ has files older than 2 days ...
#--
Finished ...

这是我追求的结果...

root@me /home/frank # bash script.sh
Start ...
000029_000020 has files older than 2 days ...
#--
Finished ...

Start ...
000033_000040 has files older than 2 days ...
#--
Finished ...

Start ...
000033_000041 has files older than 2 days ...
#--
Finished ...

Start ...
000033_000042 has files older than 2 days ...
#--
Finished ...

Start ...
000033_000043 has files older than 2 days ...
#--
Finished ...

Ubuntu 16.04

#!/bin/bash

site="hello"
wDir="/home/websites/${site}/httpdocs/"

for file in $(find "${wDir}" -name "*.css")
do
   echo "$file";
done
exit 0;

即使我定义了开始目录，shellcheck 也会警告我，但脚本工作得很好。

root@me /scripts/ # shellcheck test.sh

In test.sh line 6:
for file in $(find "${wDir}" -name "*.css")
            ^-- SC2044: For loops over find output are fragile. Use find -exec or a while read loop.