Meus contêineres CentOS LXC não estão mais iniciando em uma máquina Ubuntu 14.10. Acho que o problema começou após uma reinicialização, mas não tenho certeza.
Eu tive um problema semelhante após uma atualização do yum, quando os scripts init foram substituídos por outros padrão que não reconhecem o LXC. Eles estavam tentando iniciar o udev, etc... Mas desta vez eu tenho esse problema para todas as instâncias do CentOS, mesmo para aquelas recém-criadas.
Sistema operacional host: Ubuntu14.10 64 bits Sistema
operacional convidado: Centos 6.5 64 bits
root@ubuntu-mvutcovici:~# lxc-start --logfile stash-lxc.log --logpriority DEBUG -dn stash
lxc-start: lxc_start.c: main: 337 The container failed to start.
lxc-start: lxc_start.c: main: 339 To get more details, run the container in foreground mode.
lxc-start: lxc_start.c: main: 341 Additional information can be obtained by setting the --logfile and --logpriority options.
root@ubuntu-mvutcovici:~#
Aqui está o conteúdo do arquivo stash-lxc.log:
lxc-start 1416596262.928 INFO lxc_start_ui - lxc_start.c:main:265 - using rcfile /var/lib/lxc/stash/config
lxc-start 1416596262.928 WARN lxc_confile - confile.c:config_pivotdir:1685 - lxc.pivotdir is ignored. It will soon become an error.
lxc-start 1416596262.928 WARN lxc_log - log.c:lxc_log_init:316 - lxc_log_init called with log already initialized
lxc-start 1416596262.929 INFO lxc_start - start.c:lxc_check_inherited:209 - closed inherited fd 4
lxc-start 1416596262.934 INFO lxc_lsm - lsm/lsm.c:lsm_init:48 - LSM security driver AppArmor
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:285 - processing: .[all].
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:285 - processing: .kexec_load errno 1.
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:358 - Adding non-compat rule for kexec_load action 327681
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:369 - Adding compat rule for kexec_load action 327681
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:382 - Really adding compat rule bc nr1 == nr2 (283, 246)
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:285 - processing: .open_by_handle_at errno 1.
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:358 - Adding non-compat rule for open_by_handle_at action 327681
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:369 - Adding compat rule for open_by_handle_at action 327681
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:382 - Really adding compat rule bc nr1 == nr2 (342, 304)
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:285 - processing: .init_module errno 1.
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:358 - Adding non-compat rule for init_module action 327681
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:369 - Adding compat rule for init_module action 327681
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:382 - Really adding compat rule bc nr1 == nr2 (128, 175)
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:285 - processing: .finit_module errno 1.
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:358 - Adding non-compat rule for finit_module action 327681
lxc-start 1416596262.934 WARN lxc_seccomp - seccomp.c:do_resolve_add_rule:196 - Seccomp: got negative # for syscall: finit_module
lxc-start 1416596262.934 WARN lxc_seccomp - seccomp.c:do_resolve_add_rule:197 - This syscall will NOT be blacklisted
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:369 - Adding compat rule for finit_module action 327681
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:377 - Adding non-compat rule bc nr1 == nr2 (-10085, -10085)
lxc-start 1416596262.934 WARN lxc_seccomp - seccomp.c:do_resolve_add_rule:196 - Seccomp: got negative # for syscall: finit_module
lxc-start 1416596262.934 WARN lxc_seccomp - seccomp.c:do_resolve_add_rule:197 - This syscall will NOT be blacklisted
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:285 - processing: .delete_module errno 1.
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:358 - Adding non-compat rule for delete_module action 327681
lxc-start 1416596262.934 INFO lxc_seccomp - seccomp.c:parse_config_v2:369 - Adding compat rule for delete_module action 327681
lxc-start 1416596262.935 INFO lxc_seccomp - seccomp.c:parse_config_v2:382 - Really adding compat rule bc nr1 == nr2 (129, 176)
lxc-start 1416596262.935 INFO lxc_seccomp - seccomp.c:parse_config_v2:390 - Merging in the compat seccomp ctx into the main one
lxc-start 1416596262.935 DEBUG lxc_conf - conf.c:lxc_create_tty:3504 - allocated pty '/dev/pts/2' (5/6)
lxc-start 1416596262.935 DEBUG lxc_conf - conf.c:lxc_create_tty:3504 - allocated pty '/dev/pts/4' (7/8)
lxc-start 1416596262.935 DEBUG lxc_conf - conf.c:lxc_create_tty:3504 - allocated pty '/dev/pts/5' (9/10)
lxc-start 1416596262.935 DEBUG lxc_conf - conf.c:lxc_create_tty:3504 - allocated pty '/dev/pts/7' (11/12)
lxc-start 1416596262.935 INFO lxc_conf - conf.c:lxc_create_tty:3515 - tty's configured
lxc-start 1416596262.935 DEBUG lxc_start - start.c:setup_signal_fd:247 - sigchild handler set
lxc-start 1416596262.935 DEBUG lxc_console - console.c:lxc_console_peer_default:536 - no console peer
lxc-start 1416596262.935 INFO lxc_start - start.c:lxc_init:443 - 'stash' is initialized
lxc-start 1416596262.936 DEBUG lxc_start - start.c:__lxc_start:1061 - Not dropping cap_sys_boot or watching utmp
lxc-start 1416596262.936 INFO lxc_start - start.c:lxc_check_inherited:209 - closed inherited fd 4
lxc-start 1416596262.940 INFO lxc_monitor - monitor.c:lxc_monitor_sock_name:177 - using monitor sock name lxc/ad055575fe28ddd5//var/lib/lxc
lxc-start 1416596262.943 DEBUG lxc_conf - conf.c:instanciate_veth:2842 - instanciated veth 'vethF4JUT8/vethVOPS0P', index is '11'
lxc-start 1416596262.943 INFO lxc_cgroup - cgroup.c:cgroup_init:62 - cgroup driver cgmanager initing for stash
lxc-start 1416596262.948 INFO lxc_cgmanager - cgmanager.c:cgm_setup_limits:1241 - cgroup limits have been setup
lxc-start 1416596262.977 DEBUG lxc_conf - conf.c:lxc_assign_network:3259 - move '(null)' to '11664'
lxc-start 1416596262.978 DEBUG lxc_conf - conf.c:setup_rootfs:1536 - mounted '/var/lib/lxc/stash/rootfs' on '/usr/lib/x86_64-linux-gnu/lxc'
lxc-start 1416596262.978 INFO lxc_conf - conf.c:setup_utsname:896 - 'stash' hostname has been setup
lxc-start 1416596263.005 DEBUG lxc_conf - conf.c:setup_hw_addr:2392 - mac address 'fe:fb:95:37:ac:3c' on 'eth0' has been setup
lxc-start 1416596263.005 DEBUG lxc_conf - conf.c:setup_netdev:2619 - 'eth0' has been setup
lxc-start 1416596263.005 INFO lxc_conf - conf.c:setup_network:2640 - network has been setup
lxc-start 1416596263.005 INFO lxc_conf - conf.c:setup_ttydir_console:1688 - created /usr/lib/x86_64-linux-gnu/lxc/dev/lxc
lxc-start 1416596263.005 INFO lxc_conf - conf.c:setup_ttydir_console:1734 - console has been setup on lxc/console
lxc-start 1416596263.006 INFO lxc_conf - conf.c:setup_tty:1023 - 4 tty(s) has been setup
lxc-start 1416596263.006 INFO lxc_conf - conf.c:do_tmp_proc_mount:3809 - I am 1, /proc/self points to '1'
lxc-start 1416596263.029 DEBUG lxc_conf - conf.c:setup_rootfs_pivot_root:1078 - pivot_root syscall to '/usr/lib/x86_64-linux-gnu/lxc' successful
lxc-start 1416596263.029 INFO lxc_conf - conf.c:setup_pts:1605 - created new pts instance
lxc-start 1416596263.029 INFO lxc_conf - conf.c:setup_personality:1622 - set personality to '0x0'
lxc-start 1416596263.029 DEBUG lxc_conf - conf.c:setup_caps:2303 - drop capability 'mac_admin' (33)
lxc-start 1416596263.029 DEBUG lxc_conf - conf.c:setup_caps:2303 - drop capability 'mac_override' (32)
lxc-start 1416596263.029 DEBUG lxc_conf - conf.c:setup_caps:2303 - drop capability 'sys_time' (25)
lxc-start 1416596263.029 DEBUG lxc_conf - conf.c:setup_caps:2303 - drop capability 'sys_module' (16)
lxc-start 1416596263.029 DEBUG lxc_conf - conf.c:setup_caps:2303 - drop capability 'setfcap' (31)
lxc-start 1416596263.029 DEBUG lxc_conf - conf.c:setup_caps:2303 - drop capability 'setpcap' (8)
lxc-start 1416596263.029 DEBUG lxc_conf - conf.c:setup_caps:2303 - drop capability 'sys_nice' (23)
lxc-start 1416596263.029 DEBUG lxc_conf - conf.c:setup_caps:2303 - drop capability 'sys_pacct' (20)
lxc-start 1416596263.029 DEBUG lxc_conf - conf.c:setup_caps:2303 - drop capability 'sys_rawio' (17)
lxc-start 1416596263.029 DEBUG lxc_conf - conf.c:setup_caps:2312 - capabilities have been setup
lxc-start 1416596263.029 NOTICE lxc_conf - conf.c:lxc_setup:4144 - 'stash' is setup.
lxc-start 1416596263.029 DEBUG lxc_cgmanager - cgmanager.c:cgm_setup_limits:1237 - cgroup 'devices.deny' set to 'a'
lxc-start 1416596263.029 DEBUG lxc_cgmanager - cgmanager.c:cgm_setup_limits:1237 - cgroup 'devices.allow' set to 'c *:* m'
lxc-start 1416596263.030 DEBUG lxc_cgmanager - cgmanager.c:cgm_setup_limits:1237 - cgroup 'devices.allow' set to 'b *:* m'
lxc-start 1416596263.030 DEBUG lxc_cgmanager - cgmanager.c:cgm_setup_limits:1237 - cgroup 'devices.allow' set to 'c 1:3 rwm'
lxc-start 1416596263.030 DEBUG lxc_cgmanager - cgmanager.c:cgm_setup_limits:1237 - cgroup 'devices.allow' set to 'c 1:5 rwm'
lxc-start 1416596263.030 DEBUG lxc_cgmanager - cgmanager.c:cgm_setup_limits:1237 - cgroup 'devices.allow' set to 'c 1:7 rwm'
lxc-start 1416596263.031 DEBUG lxc_cgmanager - cgmanager.c:cgm_setup_limits:1237 - cgroup 'devices.allow' set to 'c 5:0 rwm'
lxc-start 1416596263.031 DEBUG lxc_cgmanager - cgmanager.c:cgm_setup_limits:1237 - cgroup 'devices.allow' set to 'c 5:1 rwm'
lxc-start 1416596263.031 DEBUG lxc_cgmanager - cgmanager.c:cgm_setup_limits:1237 - cgroup 'devices.allow' set to 'c 5:2 rwm'
lxc-start 1416596263.031 DEBUG lxc_cgmanager - cgmanager.c:cgm_setup_limits:1237 - cgroup 'devices.allow' set to 'c 1:8 rwm'
lxc-start 1416596263.031 DEBUG lxc_cgmanager - cgmanager.c:cgm_setup_limits:1237 - cgroup 'devices.allow' set to 'c 1:9 rwm'
lxc-start 1416596263.031 DEBUG lxc_cgmanager - cgmanager.c:cgm_setup_limits:1237 - cgroup 'devices.allow' set to 'c 136:* rwm'
lxc-start 1416596263.031 INFO lxc_cgmanager - cgmanager.c:cgm_setup_limits:1241 - cgroup limits have been setup
lxc-start 1416596263.031 ERROR lxc_apparmor - lsm/apparmor.c:mount_feature_enabled:61 - Permission denied - Error mounting securityfs
lxc-start 1416596263.032 WARN lxc_apparmor - lsm/apparmor.c:apparmor_process_label_set:184 - Incomplete AppArmor support in your kernel
lxc-start 1416596263.032 ERROR lxc_apparmor - lsm/apparmor.c:apparmor_process_label_set:186 - If you really want to start this container, set
lxc-start 1416596263.032 ERROR lxc_apparmor - lsm/apparmor.c:apparmor_process_label_set:187 - lxc.aa_allow_incomplete = 1
lxc-start 1416596263.032 ERROR lxc_apparmor - lsm/apparmor.c:apparmor_process_label_set:188 - in your container configuration file
lxc-start 1416596263.032 ERROR lxc_sync - sync.c:__sync_wait:51 - invalid sequence number 1. expected 4
lxc-start 1416596263.032 ERROR lxc_start - start.c:__lxc_start:1087 - failed to spawn 'stash'
lxc-start 1416596263.032 WARN lxc_commands - commands.c:lxc_cmd_rsp_recv:172 - command get_init_pid failed to receive response
lxc-start 1416596263.032 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:503 - call to cgmanager_remove_sync failed: invalid request
lxc-start 1416596263.032 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:505 - Error removing name=systemd:lxc/stash-3
lxc-start 1416596263.032 WARN lxc_cgmanager - cgmanager.c:cgm_get:946 - do_cgm_get exited with error
lxc-start 1416596263.032 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:503 - call to cgmanager_remove_sync failed: invalid request
lxc-start 1416596263.032 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:505 - Error removing perf_event:lxc/stash-3
lxc-start 1416596263.033 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:503 - call to cgmanager_remove_sync failed: invalid request
lxc-start 1416596263.033 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:505 - Error removing net_prio:lxc/stash-3
lxc-start 1416596263.033 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:503 - call to cgmanager_remove_sync failed: invalid request
lxc-start 1416596263.033 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:505 - Error removing net_cls:lxc/stash-3
lxc-start 1416596263.033 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:503 - call to cgmanager_remove_sync failed: invalid request
lxc-start 1416596263.033 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:505 - Error removing memory:lxc/stash-3
lxc-start 1416596263.033 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:503 - call to cgmanager_remove_sync failed: invalid request
lxc-start 1416596263.033 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:505 - Error removing hugetlb:lxc/stash-3
lxc-start 1416596263.033 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:503 - call to cgmanager_remove_sync failed: invalid request
lxc-start 1416596263.033 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:505 - Error removing freezer:lxc/stash-3
lxc-start 1416596263.034 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:503 - call to cgmanager_remove_sync failed: invalid request
lxc-start 1416596263.034 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:505 - Error removing devices:lxc/stash-3
lxc-start 1416596263.034 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:503 - call to cgmanager_remove_sync failed: invalid request
lxc-start 1416596263.034 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:505 - Error removing cpuset:lxc/stash-3
lxc-start 1416596263.034 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:503 - call to cgmanager_remove_sync failed: invalid request
lxc-start 1416596263.034 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:505 - Error removing cpuacct:lxc/stash-3
lxc-start 1416596263.034 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:503 - call to cgmanager_remove_sync failed: invalid request
lxc-start 1416596263.034 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:505 - Error removing cpu:lxc/stash-3
lxc-start 1416596263.035 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:503 - call to cgmanager_remove_sync failed: invalid request
lxc-start 1416596263.035 ERROR lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:505 - Error removing blkio:lxc/stash-3
lxc-start 1416596268.038 ERROR lxc_start_ui - lxc_start.c:main:337 - The container failed to start.
lxc-start 1416596268.038 ERROR lxc_start_ui - lxc_start.c:main:339 - To get more details, run the container in foreground mode.
lxc-start 1416596268.038 ERROR lxc_start_ui - lxc_start.c:main:341 - Additional information can be obtained by setting the --logfile and --logpriority options.
Para criar todas as instâncias do CentOS que usei:
root@ubuntu-mvutcovici:~# lxc-create -t centos -f lxc-mircea.conf -n stash
root@ubuntu-mvutcovici:~# cat lxc-mircea.conf
lxc.network.type = veth
lxc.network.link = br0
lxc.network.flags = up
EDIT : Parece que adicionar lxc.aa_allow_incomplete = 1ao arquivo /var/lib/lxc/stash/config é uma solução alternativa para o problema de inicialização. Como posso fazer com que o App Armor coexista novamente com o CentOS LXC?
Na página do manual lxc.container.conf:
lxc.aa_allow_incomplete
Apparmor profiles are pathname based. Therefore many file restrictions require mount restrictions to be effective against a determined attacker. However, these mount restrictions are not yet implemented in the
upstream kernel. Without the mount restrictions, the apparmor profiles still protect against accidental damager.
If this flag is 0 (default), then the container will not be started if the kernel lacks the apparmor mount features, so that a regression after a kernel upgrade will be detected. To start the container under partial
apparmor protection, set this flag to 1.
EDIT2 : adicionado arquivo /var/lib/lxc/stash/config original:
# Template used to create this container: /usr/share/lxc/templates/lxc-centos
# Parameters passed to the template:
# For additional config options, please look at lxc.container.conf(5)
lxc.network.type = veth
lxc.network.link = br0
lxc.network.hwaddr = fe:98:41:37:ca:3d
lxc.network.flags = up
lxc.rootfs = /var/lib/lxc/stash/rootfs
# Include common configuration
lxc.include = /usr/share/lxc/config/centos.common.conf
lxc.arch = x86_64
lxc.utsname = stash
lxc.autodev = 0
# When using LXC with apparmor, uncomment the next line to run unconfined:
#lxc.aa_profile = unconfined
# example simple networking setup, uncomment to enable
#lxc.network.type = veth
#lxc.network.flags = up
#lxc.network.link = lxcbr0
#lxc.network.name = eth0
# Additional example for veth network type
# static MAC address,
#lxc.network.hwaddr = 00:16:3e:77:52:20
# persistent veth device name on host side
# Note: This may potentially collide with other containers of same name!
#lxc.network.veth.pair = v-stash-e0
A solução era anexar
lxc.aa_allow_incomplete = 1ao/var/lib/lxc/[container-name]/configarquivo.Essa configuração diminuirá a segurança oferecida pelo apparmor. Este é um trecho da
lxc.container.conf(5)página de manual.Na verdade, parece que você se deparou com um bug . O link referenciado direciona para um patch que ajuda a evitar essas falhas do AppArmor. No entanto, você precisará saber como compilar o LXC a partir do código-fonte para usá-lo. Não tenho certeza se esse patch chegou aos binários ainda.
Depois de atualizar o Ubuntu 14.4 para 16.x, siga as etapas de atualização e atualização do sistema. Isso me permite iniciar meus contêineres lxc mais uma vez. atualização do apt-get atualização do apt-get