Alex's questions

Temos um conjunto de replicação do MongoDB composto por três membros:

        "members" : [
                {
                        "_id" : 6,
                        "host" : "10.0.0.17:27017",
                        "arbiterOnly" : false,
                        "buildIndexes" : true,
                        "hidden" : false,
                        "priority" : 2,
                        "tags" : {
                        },
                        "slaveDelay" : NumberLong(0),
                        "votes" : 1
                },
                {
                        "_id" : 7,
                        "host" : "10.0.0.18:27017",
                        "arbiterOnly" : false,
                        "buildIndexes" : true,
                        "hidden" : false,
                        "priority" : 2,
                        "tags" : {
                        },
                        "slaveDelay" : NumberLong(0),
                        "votes" : 1
                },
                {
                        "_id" : 8,
                        "host" : "10.0.0.19:27017",
                        "arbiterOnly" : false,
                        "buildIndexes" : true,
                        "hidden" : false,
                        "priority" : 2,
                        "tags" : {
                        },
                        "slaveDelay" : NumberLong(0),
                        "votes" : 1
                }
        ],

O cluster está sob carga moderada, não mais do que algumas dezenas de solicitações por segundo. db.serverStatus()em um relatório principal que quase todas as transações são revertidas:

"transaction begins" : 2625009877,
"transaction checkpoint currently running" : 0,
"transaction checkpoint generation" : 22618,
"transaction checkpoint max time (msecs)" : 5849,
"transaction checkpoint min time (msecs)" : 153,
"transaction checkpoint most recent time (msecs)" : 1869,
"transaction checkpoint scrub dirty target" : 0,
"transaction checkpoint scrub time (msecs)" : 0,
"transaction checkpoint total time (msecs)" : 11017082,
"transaction checkpoints" : 22617,
"transaction checkpoints skipped because database was clean" : 0,
"transaction failures due to cache overflow" : 0,
"transaction fsync calls for checkpoint after allocating the transaction ID" : 22617,
"transaction fsync duration for checkpoint after allocating the transaction ID (usecs)" : 354402,
"transaction range of IDs currently pinned" : 0,
"transaction range of IDs currently pinned by a checkpoint" : 0,
"transaction range of IDs currently pinned by named snapshots" : 0,
"transaction range of timestamps currently pinned" : 8589934583,
"transaction range of timestamps pinned by the oldest timestamp" : 8589934583,
"transaction sync calls" : 0,
"transactions committed" : 30213144,
"transactions rolled back" : 2594972913,
"update conflicts" : 578

Basicamente, minhas perguntas são: O que está acontecendo aqui? É normal ter tantas transações e tantos rollbacks? Se não, qual é a causa raiz e quente para corrigi-lo?

Atualizado : Atualizamos para 3.6.8-2.0(este foi o pacote Percona mais recente na série 3.6) e o problema persistiu.

Estou tentando estabelecer uma conexão entre os serviços firewalld e docker. Estendi o arquivo de unidade firewalld.service padrão adicionando o seguinte:

[Service]
ExecStartPost=-/bin/bash -c '/usr/bin/test -f /etc/default/docker && /bin/systemctl stop docker && /bin/systemctl start docker'

para /etc/systemd/system/firewalld.service.d/docker.conf.

Mas isso não funcionou, o serviço firewalld expirou no início:

root@ubuntu-xenial:~# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/firewalld.service.d
           └─docker.conf
   Active: failed (Result: timeout) since Sat 2017-04-08 14:39:45 UTC; 1min 35s ago
  Process: 26050 ExecStartPost=/bin/bash -c /usr/bin/test -f /etc/default/docker && /bin/systemctl stop docker && /bin/systemctl start docker (code=killed, signal=TERM)
  Process: 26000 ExecStart=/usr/sbin/firewalld --nofork --nopid (code=exited, status=0/SUCCESS)
 Main PID: 26000 (code=exited, status=0/SUCCESS)

Apr 08 14:38:10 ubuntu-xenial systemd[1]: Starting firewalld - dynamic firewall daemon...
Apr 08 14:39:41 ubuntu-xenial systemd[1]: firewalld.service: Start-post operation timed out. Stopping.
Apr 08 14:39:45 ubuntu-xenial systemd[1]: Failed to start firewalld - dynamic firewall daemon.
Apr 08 14:39:45 ubuntu-xenial systemd[1]: firewalld.service: Unit entered failed state.
Apr 08 14:39:45 ubuntu-xenial systemd[1]: firewalld.service: Failed with result 'timeout'.

Estou ciente disso BindTo, Requiresmas gostaria de reiniciar o serviço docker a cada mudança de estado do firewalld para que essas diretivas não resolvam meu problema.

Upd.: strace mostra que systemctl startestá esperando em um soquete unix:

root@ubuntu-xenial:~# lsof -p 27898
COMMAND     PID USER   FD   TYPE             DEVICE SIZE/OFF   NODE NAME
systemctl 27898 root  cwd    DIR                8,1     4096      2 /
systemctl 27898 root  rtd    DIR                8,1     4096      2 /
systemctl 27898 root  txt    REG                8,1   659848    124 /bin/systemctl
systemctl 27898 root  mem    REG                8,1    80496   2057 /lib/x86_64-linux-gnu/libgpg-error.so.0.17.0
systemctl 27898 root  mem    REG                8,1    14608   2060 /lib/x86_64-linux-gnu/libdl-2.23.so
systemctl 27898 root  mem    REG                8,1   456632   2144 /lib/x86_64-linux-gnu/libpcre.so.3.13.2
systemctl 27898 root  mem    REG                8,1  1864888   2066 /lib/x86_64-linux-gnu/libc-2.23.so
systemctl 27898 root  mem    REG                8,1   138696   2082 /lib/x86_64-linux-gnu/libpthread-2.23.so
systemctl 27898 root  mem    REG                8,1   919168   2037 /lib/x86_64-linux-gnu/libgcrypt.so.20.0.5
systemctl 27898 root  mem    REG                8,1   137400   2067 /lib/x86_64-linux-gnu/liblzma.so.5.0.0
systemctl 27898 root  mem    REG                8,1    31712   2077 /lib/x86_64-linux-gnu/librt-2.23.so
systemctl 27898 root  mem    REG                8,1   130224   2154 /lib/x86_64-linux-gnu/libselinux.so.1
systemctl 27898 root  mem    REG                8,1   162632   2069 /lib/x86_64-linux-gnu/ld-2.23.so
systemctl 27898 root    0r   CHR                1,3      0t0      6 /dev/null
systemctl 27898 root    1w   CHR                1,3      0t0      6 /dev/null
systemctl 27898 root    2w   CHR                1,3      0t0      6 /dev/null
systemctl 27898 root    3u  unix 0xffff880003155400      0t0 177152 type=STREAM
root@ubuntu-xenial:~# strace -p 27898
strace: Process 27898 attached
ppoll([{fd=3, events=POLLIN}], 1, NULL, NULL, 8 <unfinished ...>
+++ killed by SIGKILL +++
root@ubuntu-xenial:~#

Atualizar 2: de acordo com gdb, systemctlestá bloqueado em uma chamada DBus:

root@ubuntu-xenial:~# gdb -ex "set pagination 0" -ex "thread apply all bt" --batch -p 18145
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007f99b3edebcd in __GI_ppoll (fds=0x7ffc098bb800, nfds=1, timeout=<optimized out>, sigmask=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:50
50  ../sysdeps/unix/sysv/linux/ppoll.c: No such file or directory.

Thread 1 (Thread 0x7f99b4f0d880 (LWP 18145)):
#0  0x00007f99b3edebcd in __GI_ppoll (fds=0x7ffc098bb800, nfds=1, timeout=<optimized out>, sigmask=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:50
#1  0x000056094741272e in ppoll () at /usr/include/x86_64-linux-gnu/bits/poll2.h:71
#2  bus_poll.lto_priv.198 (bus=0x560948d86040, need_more=<optimized out>, timeout_usec=<optimized out>) at ../src/libsystemd/sd-bus/sd-bus.c:2874
#3  0x00005609473b435e in bus_process_wait (bus=0x560948d86040) at ../src/shared/bus-util.c:1993
#4  bus_wait_for_jobs (extra_args=0x0, quiet=false, d=0x560948d86910) at ../src/shared/bus-util.c:2136
#5  start_unit.lto_priv.362 (argc=<optimized out>, argv=<optimized out>, userdata=<optimized out>) at ../src/systemctl/systemctl.c:2787
#6  0x00005609473b0716 in dispatch_verb (verbs=0x56094743f760 <verbs>, userdata=0x0, argv=0x7ffc098bbb38, argc=<optimized out>) at ../src/basic/verbs.c:92
#7  systemctl_main () at ../src/systemctl/systemctl.c:7447
#8  main (argc=<optimized out>, argv=0x7ffc098bbb38) at ../src/systemctl/systemctl.c:7713
root@ubuntu-xenial:~#