主页 / unix / 问题 / 458441
Accepted
Satish
Asked: 2018-07-26 11:47:50 +0800 CST

openstack 实时迁移问题

  • 772

我有两个带有Ceph共享存储 (RBD) 的计算节点 1 和节点 2，我正在尝试设置实时迁移，但它失败并出现以下错误，不确定出了什么问题。

我正在使用 Openstack pike 16.0.16

[root@compute-01 instances]# cat /etc/libvirt/libvirtd.conf
# Ansible managed

listen_tls = 0
listen_tcp = 1
unix_sock_group = "libvirt"
unix_sock_ro_perms = "0777"
unix_sock_rw_perms = "0770"
auth_unix_ro = "none"
auth_unix_rw = "none"
auth_tcp = "none"

nova.log 中出现以下错误

如果第一次进行虚拟机实时迁移，那么它可以工作，但会引发以下错误并且虚拟机进入Error状态

C1 ----> C2 （第一次工作但有错误）

lt] [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32] Received unexpected event network-vif-unplugged-251b70a9-2118-4f95-8b35-e9e52f4392e7 for instance
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [req-e0cd3865-151e-4d07-8b94-3a8943dafb57 78ca592a663a487d982a4c412ce4d52e d0d9c227e2d34d3a941e3cd16dea06ed - default default] [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32] Live migration failed.: AttributeError: 'Guest' object has no attribute 'migrate_configure_max_speed'
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32] Traceback (most recent call last):
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 5580, in _do_live_migration
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32]     block_migration, migrate_data)
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6436, in live_migration
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32]     migrate_data)
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6944, in _live_migration
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32]     guest.migrate_configure_max_speed(
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32] AttributeError: 'Guest' object has no attribute 'migrate_configure_max_speed'
2018-07-25 17:32:39.573 2833 ERROR nova.compute.manager [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32]
2018-07-25 17:32:41.646 2833 WARNING nova.compute.manager [req-eb9e883f-08c3-427d-89c3-cdcf012e7c8b 78ca592a663a487d982a4c412ce4d52e d0d9c227e2d34d3a941e3cd16dea06ed - default default] [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32] Received unexpected event network-vif-plugged-251b70a9-2118-4f95-8b35-e9e52f4392e7 for instance
2018-07-25 17:32:49.516 2833 WARNING nova.compute.manager [req-d70c12f9-42fc-43be-ae8e-6dd6b21b1b1f 78ca592a663a487d982a4c412ce4d52e d0d9c227e2d34d3a941e3cd16dea06ed - default default] [instance: 4f4009ee-902d-4ee9-ae99-e9bc55267b32] Received unexpected event network-vif-plugged-251b70a9-2118-4f95-8b35-e9e52f4392e7 for instance

要修复 VM 的错误状态，我必须做

[root@ostack-infra-01-utility-container-a8dbff46 ~]# nova list
+--------------------------------------+------+--------+------------+-------------+-----------------------+
| ID                                   | Name | Status | Task State | Power State | Networks              |
+--------------------------------------+------+--------+------------+-------------+-----------------------+
| 4f4009ee-902d-4ee9-ae99-e9bc55267b32 | d1   | ERROR  | -          | NOSTATE     | net-vlan31=10.31.1.10 |
+--------------------------------------+------+--------+------------+-------------+-----------------------+

nova reset-state --active 4f4009ee-902d-4ee9-ae99-e9bc55267b32

即使在成功迁移后，我的 Horizo​​n 仪表板仍然显示C1

现在 VM 在 C2 上运行，我必须将其移回 C1 我收到以下错误，看起来 nova 在迁移 VM 后没有清理文件，或者可能是因为它处于错误状态并且它没有清理以前的文件..

2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [req-285a7f42-99a1-47a2-86c1-79140fcfca31 040960f6067d42c2b52c3fcac9ebde6d 2349c3efbf8a4c6ba6dc3b961160c81b - default default] [instance: aa58095d-7027-488e-901e-f3259353de0d] Pre live migration failed at ostack-compute-02.v1v0x.net: DestinationDiskExists_Remote: The supplied disk path (/var/lib/nova/instances/aa58095d-7027-488e-901e-f3259353de0d) already exists, it is expected not to exist.
Traceback (most recent call last):

  File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
    res = self.dispatcher.dispatch(message)

  File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
    return self._do_dispatch(endpoint, method, ctxt, args)

  File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
    result = func(ctxt, **new_args)

  File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/exception_wrapper.py", line 76, in wrapped
    function_name, call_dict, binary)

  File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
    self.force_reraise()

  File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)

  File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/exception_wrapper.py", line 67, in wrapped
    return f(self, context, *args, **kw)

  File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/utils.py", line 880, in decorated_function
    return function(self, context, *args, **kwargs)

  File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 218, in decorated_function
    kwargs['instance'], e, sys.exc_info())

  File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
    self.force_reraise()

  File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)

  File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 206, in decorated_function
    return function(self, context, *args, **kwargs)

  File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 5507, in pre_live_migration
    migrate_data)

  File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7072, in pre_live_migration
    raise exception.DestinationDiskExists(path=instance_dir)

DestinationDiskExists: The supplied disk path (/var/lib/nova/instances/aa58095d-7027-488e-901e-f3259353de0d) already exists, it is expected not to exist.
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d] Traceback (most recent call last):
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 5562, in _do_live_migration
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     block_migration, disk, dest, migrate_data)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/rpcapi.py", line 745, in pre_live_migration
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     disk=disk, migrate_data=migrate_data)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 169, in call
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     retry=self.retry)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/transport.py", line 123, in _send
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     timeout=timeout, retry=retry)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 566, in send
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     retry=retry)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 557, in _send
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     raise result
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d] DestinationDiskExists_Remote: The supplied disk path (/var/lib/nova/instances/aa58095d-7027-488e-901e-f3259353de0d) already exists, it is expected not to exist.
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d] Traceback (most recent call last):
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     res = self.dispatcher.dispatch(message)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     return self._do_dispatch(endpoint, method, ctxt, args)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     result = func(ctxt, **new_args)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/exception_wrapper.py", line 76, in wrapped
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     function_name, call_dict, binary)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     self.force_reraise()
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     six.reraise(self.type_, self.value, self.tb)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/exception_wrapper.py", line 67, in wrapped
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     return f(self, context, *args, **kw)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/utils.py", line 880, in decorated_function
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     return function(self, context, *args, **kwargs)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 218, in decorated_function
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     kwargs['instance'], e, sys.exc_info())
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     self.force_reraise()
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     six.reraise(self.type_, self.value, self.tb)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 206, in decorated_function
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     return function(self, context, *args, **kwargs)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 5507, in pre_live_migration
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     migrate_data)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]   File "/openstack/venvs/nova-16.0.16/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7072, in pre_live_migration
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]     raise exception.DestinationDiskExists(path=instance_dir)
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d] DestinationDiskExists: The supplied disk path (/var/lib/nova/instances/aa58095d-7027-488e-901e-f3259353de0d) already exists, it is expected not to exist.
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:04.143 8785 ERROR nova.compute.manager [instance: aa58095d-7027-488e-901e-f3259353de0d]
2018-07-25 19:42:06.908 8785 WARNING nova.compute.manager [req-cda02056-75c7-463c-ac7e-2925ba2cd29c 78ca592a663a487d982a4c412ce4d52e d0d9c227e2d34d3a941e3cd16dea06ed - default default] [instance: aa58095d-7027-488e-901e-f3259353de0d] Received unexpected event network-vif-unplugged-7a356ab1-e0d3-4a69-9aa9-f71329caa17f for instance
2018-07-25 19:42:07.821 8785 ERROR nova.virt.libvirt.driver [req-cda02056-75c7-463c-ac7e-2925ba2cd29c 78ca592a663a487d982a4c412ce4d52e d0d9c227e2d34d3a941e3cd16dea06ed - default default] [instance: aa58095d-7027-488e-901e-f3259353de0d] Live Migration failure: Domain not found: no domain with matching name 'instance-00000056': libvirtError: Domain not found: no domain with matching name 'instance-00000056'
2018-07-25 19:42:08.294 8785 WARNING nova.compute.manager [req-5d3abd38-c8a9-499d-b97f-1d9e748a19d3 78ca592a663a487d982a4c412ce4d52e d0d9c227e2d34d3a941e3cd16dea06ed - default default] [instance: aa58095d-7027-488e-901e-f3259353de0d] Received unexpected event network-vif-plugged-7a356ab1-e0d3-4a69-9aa9-f71329caa17f for instance
2018-07-25 19:42:14.934 8785 WARNING nova.compute.manager [req-e9442ba9-1700-4221-a495-31d76a2b5bf0 78ca592a663a487d982a4c412ce4d52e d0d9c227e2d34d3a941e3cd16dea06ed - default default] [instance: aa58095d-7027-488e-901e-f3259353de0d] Received unexpected event network-vif-plugged-7a356ab1-e0d3-4a69-9aa9-f71329caa17f for instance
linux kvm

1 个回答

  1. Best Answer

    原始错误来自 libvirt 驱动程序调用guest.migrate_configure_max_speed这里：

    https://github.com/openstack/nova/blob/stable/pike/nova/virt/libvirt/driver.py#L6944

    直到 Pike ( https://github.com/openstack/nova/commit/23446a9552b5be3b040278646149a0f481d0a005 )之后，该方法才添加到来宾模块中，并且它似乎没有被反向移植到 Pike，因为它在这里丢失：

    https://github.com/openstack/nova/blob/stable/pike/nova/virt/libvirt/guest.py

    在此提交中将调用guest.migrate_configure_max_speed添加到驱动程序：https ://github.com/openstack/nova/commit/ff747792b8f5aefe1bebb01bdf49dacc01353348

    您可以使用 no-op 手动修补 guest.py 以解决该错误：

    def migrate_configure_max_speed(self, bandwidth):
    pass
    • 1

相关问题