====== QEMU/KVM/libvirtをCentOS 7にインストール ====== {{tag>qemu-kvm libvirt centos7}} ===== パッケージのインストール ===== === 必須パッケージ === yum install centos-release-qemu-ev yum install qemu-kvm-ev libvirt systemctl start libvirtd FUSEの設定が必要。 ++++ ESXi 6.0の上のCentOS 7.9にインストールする場合 | ''centos-release-qemu-ev''を使用するとゲストOSのブートに失敗したケース有り。 その際は、''qemu-kvm-ev''ではなく古い''qemu-kvm''を使用することで回避できた。 ++++ === CUI用のツール === yum install qemu-kvm-tools-ev virt-install virt-top virt-dib virt-v2v libguestfs-tools libguestfs-bash-completion libguestfs-rsync libguestfs-xfs perl-XML-XPath === GUI用のツール === yum install virt-manager virt-viewer === インストールするパッケージの概要 === ^ パッケージ名 ^^ 主要なコマンド ^ 概要 ^ | qemu-kvm-ev || qemu-kvm 2.12.0 | 必須 | | | qemu-img-ev | qemu-img | 仮想ディスクイメージの操作 | | libvirt || libvirt 4.5.0 | 必須 | | | libvirt-client | virsh, virt-host-validate, libvirt-guests.service | 基本ツール | | qemu-kvm-tools-ev || kvm_stat | | | virt-install || virt-install | 仮想マシンの作成 | | virt-top || virt-top | | | virt-dib || virt-dib | 仮想ディスクイメージの作成 | | virt-v2v || virt-v2v | 仮想マシンの変換 | | libguestfs-tools || guestfish, virt-sysprep, virt-df, virt-ls | ゲスト(の中のファイル)を操作するツール | | libguestfs-rsync || | ディスクイメージをrsyncで操作 | | libguestfs-xfs || | XFSのディスクイメージ操作 | | perl-XML-XPath || xpath | | ''centos-release-qemu-ev''を使用しない場合のQEMUのパッケージ名とバージョン。 ^ パッケージ名 ^^ 主要なコマンド ^ 概要 ^ | qemu-kvm || qemu-kvm 1.5.3 | 必須 | | | qemu-img | qemu-img | 仮想ディスクイメージの操作 | | qemu-kvm-tools || kvm_stat | | ===== 仮想化環境の設定 ===== virt-host-validate 全て''PASS''、あるいは''WARN''であること。 === FUSEの有効化 === ''LXC: Checking if device /sys/fs/fuse/connections exists : FAIL (Load the 'fuse' module to enable /proc/ overrides)''と表示された場合。 ''modprobe fuse''を実行して改善されるか確認。 改善されるなら設定を永続化する。 echo fuse > /etc/modules-load.d/fuse.conf === IOMMUの有効化 === ''QEMU: Checking if IOMMU is enabled by kernel : WARN (IOMMU appears to be disabled in kernel. Add intel_iommu=on to kernel cmdline arguments)''と表示された場合。 vi /etc/sysconfig/grub CPUがIntel製なら''GRUB_CMDLINE_LINUX''に''intel_iommu=on iommu=pt''を追加。 AMD製なら''iommu=pt''を追加。 UEFIブートなら''/etc/grub2-efi.cfg''に出力、BIOSブートなら''/etc/grub2.cfg''に出力。 grub2-mkconfig -o /etc/grub2-efi.cfg reboot * [[https://access.redhat.com/documentation/ja-jp/red_hat_enterprise_linux/7/html/virtualization_deployment_and_administration_guide/sect-device-gpu]] * [[https://access.redhat.com/documentation/ja-jp/red_hat_enterprise_linux/8/html/configuring_and_managing_virtualization/assembly_managing-gpu-devices-in-virtual-machines_configuring-and-managing-virtualization]] ===== サポートOSの一覧 ===== ''virt-install''の''%%--os-variant%%''に設定可能な値。 osinfo-query os 一部抜粋 ^ Short ID ^ Name ^ CentOS 7 ^ | centos7.0 | CentOS 7 | x | | centos8 | CentOS 8 | x | | debian10 | Debian 10 | x | | freebsd12.1 | FreeBSD 12.1 | x | | rhel7.9 | Red Hat Enterprise Linux 7.9 | x | | rhel8-unknown | Red Hat Enterprise Linux 8 Unknown | x | ===== CentOS 7のlibvirtが作成するデフォルトNATネットワークの設定内容 ===== {{:articles:kvm-virbr0.png?nolink|}} [root@kvmhost ~]# virsh net-dumpxml default default 7ee64691-b5b4-4f7e-affe-df96fe6268bd [root@kvmhost ~]# nmcli con NAME UUID TYPE DEVICE eth0 e1a84776-5009-44b4-a597-47e8707ec615 ethernet eth0 virbr0 ff8c7f60-bdc3-45d4-8aab-b67d03d431fc bridge virbr0 [root@kvmhost ~]# nmcli -f connection.zone con show eth0 connection.zone: -- [root@kvmhost ~]# nmcli -f connection.zone con show virbr0 connection.zone: -- [root@kvmhost ~]# nmcli dev DEVICE TYPE STATE CONNECTION eth0 ethernet connected eth0 virbr0 bridge connected virbr0 lo loopback unmanaged -- virbr0-nic tun unmanaged -- [root@kvmhost ~]# nmcli dev show virbr0 GENERAL.DEVICE: virbr0 GENERAL.TYPE: bridge GENERAL.HWADDR: 52:54:00:FB:05:99 GENERAL.MTU: 1500 GENERAL.STATE: 100 (connected) GENERAL.CONNECTION: virbr0 GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/2 IP4.ADDRESS[1]: 192.168.122.1/24 IP4.GATEWAY: -- IP4.ROUTE[1]: dst = 192.168.122.0/24, nh = 0.0.0.0, mt = 0 IP6.GATEWAY: -- [root@kvmhost ~]# nmcli dev show virbr0-nic GENERAL.DEVICE: virbr0-nic GENERAL.TYPE: tun GENERAL.HWADDR: 52:54:00:FB:05:99 GENERAL.MTU: 1500 GENERAL.STATE: 10 (unmanaged) GENERAL.CONNECTION: -- GENERAL.CON-PATH: -- [root@kvmhost ~]# brctl show bridge name bridge id STP enabled interfaces virbr0 8000.525400fb0599 yes virbr0-nic [root@kvmhost ~]# brctl showstp virbr0 virbr0 bridge id 8000.525400fb0599 designated root 8000.525400fb0599 root port 0 path cost 0 max age 20.00 bridge max age 20.00 hello time 2.00 bridge hello time 2.00 forward delay 2.00 bridge forward delay 2.00 ageing time 300.00 hello timer 1.46 tcn timer 0.00 topology change timer 0.00 gc timer 235.90 flags virbr0-nic (1) port id 8001 state disabled designated root 8000.525400fb0599 path cost 100 designated bridge 8000.525400fb0599 message age timer 0.00 designated port 8001 forward delay timer 0.00 designated cost 0 hold timer 0.00 flags [root@kvmhost ~]# ip netns [root@kvmhost ~]# ip -detail link 1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 2: eth0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 link/ether 00:15:5d:00:24:2a brd ff:ff:ff:ff:ff:ff promiscuity 0 addrgenmode none numtxqueues 64 numrxqueues 64 gso_max_size 62780 gso_max_segs 65535 3: virbr0: mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000 link/ether 52:54:00:fb:05:99 brd ff:ff:ff:ff:ff:ff promiscuity 0 bridge forward_delay 200 hello_time 200 max_age 2000 ageing_time 30000 stp_state 1 priority 32768 vlan_filtering 0 vlan_protocol 802.1Q bridge_id 8000.52:54:0:fb:5:99 designated_root 8000.52:54:0:fb:5:99 root_port 0 root_path_cost 0 topology_change 0 topology_change_detected 0 hello_timer 0.34 tcn_timer 0.00 topology_change_timer 0.00 gc_timer 224.78 vlan_default_pvid 1 vlan_stats_enabled 0 group_fwd_mask 0 group_address 01:80:c2:00:00:00 mcast_snooping 1 mcast_router 1 mcast_query_use_ifaddr 0 mcast_querier 0 mcast_hash_elasticity 4 mcast_hash_max 512 mcast_last_member_count 2 mcast_startup_query_count 2 mcast_last_member_interval 100 mcast_membership_interval 26000 mcast_querier_interval 25500 mcast_query_interval 12500 mcast_query_response_interval 1000 mcast_startup_query_interval 3125 mcast_stats_enabled 0 mcast_igmp_version 2 mcast_mld_version 1 nf_call_iptables 0 nf_call_ip6tables 0 nf_call_arptables 0 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 4: virbr0-nic: mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT group default qlen 1000 link/ether 52:54:00:fb:05:99 brd ff:ff:ff:ff:ff:ff promiscuity 1 tun bridge_slave state disabled priority 32 cost 100 hairpin off guard off root_block off fastleave off learning on flood on port_id 0x8001 port_no 0x1 designated_port 32769 designated_cost 0 designated_bridge 8000.52:54:0:fb:5:99 designated_root 8000.52:54:0:fb:5:99 hold_timer 0.00 message_age_timer 0.00 forward_delay_timer 0.00 topology_change_ack 0 config_pending 0 proxy_arp off proxy_arp_wifi off mcast_router 1 mcast_fast_leave off mcast_flood on addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 [root@kvmhost ~]# ip addr 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:15:5d:00:24:2a brd ff:ff:ff:ff:ff:ff inet 172.31.0.120/24 brd 172.31.0.255 scope global noprefixroute eth0 valid_lft forever preferred_lft forever inet6 fe80::8364:c19b:ca0b:310a/64 scope link noprefixroute valid_lft forever preferred_lft forever 3: virbr0: mtu 1500 qdisc noqueue state DOWN group default qlen 1000 link/ether 52:54:00:fb:05:99 brd ff:ff:ff:ff:ff:ff inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0 valid_lft forever preferred_lft forever 4: virbr0-nic: mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000 link/ether 52:54:00:fb:05:99 brd ff:ff:ff:ff:ff:ff [root@kvmhost ~]# ip route default via 172.31.0.1 dev eth0 proto static metric 100 172.31.0.0/24 dev eth0 proto kernel scope link src 172.31.0.120 metric 100 192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 [root@kvmhost ~]# ps ax | grep [d]nsmasq 1389 ? S 0:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper 1390 ? S 0:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper [root@kvmhost ~]# cat /var/lib/libvirt/dnsmasq/default.conf ##WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE ##OVERWRITTEN AND LOST. Changes to this configuration should be made using: ## virsh net-edit default ## or other application using the libvirt API. ## ## dnsmasq conf file created by libvirt strict-order pid-file=/var/run/libvirt/network/default.pid except-interface=lo bind-dynamic interface=virbr0 dhcp-range=192.168.122.2,192.168.122.254 dhcp-no-override dhcp-authoritative dhcp-lease-max=253 dhcp-hostsfile=/var/lib/libvirt/dnsmasq/default.hostsfile addn-hosts=/var/lib/libvirt/dnsmasq/default.addnhosts [root@kvmhost ~]# cat /var/lib/libvirt/dnsmasq/default.hostsfile [root@kvmhost ~]# cat /var/lib/libvirt/dnsmasq/default.addnhosts [root@kvmhost ~]# cat /proc/sys/net/ipv4/ip_forward 1 [root@kvmhost ~]# iptables -S -t filter -P INPUT ACCEPT -P FORWARD ACCEPT -P OUTPUT ACCEPT -N FORWARD_IN_ZONES -N FORWARD_IN_ZONES_SOURCE -N FORWARD_OUT_ZONES -N FORWARD_OUT_ZONES_SOURCE -N FORWARD_direct -N FWDI_public -N FWDI_public_allow -N FWDI_public_deny -N FWDI_public_log -N FWDO_public -N FWDO_public_allow -N FWDO_public_deny -N FWDO_public_log -N INPUT_ZONES -N INPUT_ZONES_SOURCE -N INPUT_direct -N IN_public -N IN_public_allow -N IN_public_deny -N IN_public_log -N OUTPUT_direct -A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT -A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT -A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT -A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -j INPUT_direct -A INPUT -j INPUT_ZONES_SOURCE -A INPUT -j INPUT_ZONES -A INPUT -m conntrack --ctstate INVALID -j DROP -A INPUT -j REJECT --reject-with icmp-host-prohibited -A FORWARD -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT -A FORWARD -i virbr0 -o virbr0 -j ACCEPT -A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable -A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A FORWARD -i lo -j ACCEPT -A FORWARD -j FORWARD_direct -A FORWARD -j FORWARD_IN_ZONES_SOURCE -A FORWARD -j FORWARD_IN_ZONES -A FORWARD -j FORWARD_OUT_ZONES_SOURCE -A FORWARD -j FORWARD_OUT_ZONES -A FORWARD -m conntrack --ctstate INVALID -j DROP -A FORWARD -j REJECT --reject-with icmp-host-prohibited -A OUTPUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT -A OUTPUT -o lo -j ACCEPT -A OUTPUT -j OUTPUT_direct -A FORWARD_IN_ZONES -i eth0 -g FWDI_public -A FORWARD_IN_ZONES -g FWDI_public -A FORWARD_OUT_ZONES -o eth0 -g FWDO_public -A FORWARD_OUT_ZONES -g FWDO_public -A FWDI_public -j FWDI_public_log -A FWDI_public -j FWDI_public_deny -A FWDI_public -j FWDI_public_allow -A FWDI_public -p icmp -j ACCEPT -A FWDO_public -j FWDO_public_log -A FWDO_public -j FWDO_public_deny -A FWDO_public -j FWDO_public_allow -A INPUT_ZONES -i eth0 -g IN_public -A INPUT_ZONES -g IN_public -A IN_public -j IN_public_log -A IN_public -j IN_public_deny -A IN_public -j IN_public_allow -A IN_public -p icmp -j ACCEPT -A IN_public_allow -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW,UNTRACKED -j ACCEPT [root@kvmhost ~]# iptables -S -t nat -P PREROUTING ACCEPT -P INPUT ACCEPT -P OUTPUT ACCEPT -P POSTROUTING ACCEPT -N OUTPUT_direct -N POSTROUTING_ZONES -N POSTROUTING_ZONES_SOURCE -N POSTROUTING_direct -N POST_public -N POST_public_allow -N POST_public_deny -N POST_public_log -N PREROUTING_ZONES -N PREROUTING_ZONES_SOURCE -N PREROUTING_direct -N PRE_public -N PRE_public_allow -N PRE_public_deny -N PRE_public_log -A PREROUTING -j PREROUTING_direct -A PREROUTING -j PREROUTING_ZONES_SOURCE -A PREROUTING -j PREROUTING_ZONES -A OUTPUT -j OUTPUT_direct -A POSTROUTING -s 192.168.122.0/24 -d 224.0.0.0/24 -j RETURN -A POSTROUTING -s 192.168.122.0/24 -d 255.255.255.255/32 -j RETURN -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535 -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535 -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE -A POSTROUTING -j POSTROUTING_direct -A POSTROUTING -j POSTROUTING_ZONES_SOURCE -A POSTROUTING -j POSTROUTING_ZONES -A POSTROUTING_ZONES -o eth0 -g POST_public -A POSTROUTING_ZONES -g POST_public -A POST_public -j POST_public_log -A POST_public -j POST_public_deny -A POST_public -j POST_public_allow -A PREROUTING_ZONES -i eth0 -g PRE_public -A PREROUTING_ZONES -g PRE_public -A PRE_public -j PRE_public_log -A PRE_public -j PRE_public_deny -A PRE_public -j PRE_public_allow [root@kvmhost ~]# iptables -S -t mangle -P PREROUTING ACCEPT -P INPUT ACCEPT -P FORWARD ACCEPT -P OUTPUT ACCEPT -P POSTROUTING ACCEPT -N FORWARD_direct -N INPUT_direct -N OUTPUT_direct -N POSTROUTING_direct -N PREROUTING_ZONES -N PREROUTING_ZONES_SOURCE -N PREROUTING_direct -N PRE_public -N PRE_public_allow -N PRE_public_deny -N PRE_public_log -A PREROUTING -j PREROUTING_direct -A PREROUTING -j PREROUTING_ZONES_SOURCE -A PREROUTING -j PREROUTING_ZONES -A INPUT -j INPUT_direct -A FORWARD -j FORWARD_direct -A OUTPUT -j OUTPUT_direct -A POSTROUTING -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill -A POSTROUTING -j POSTROUTING_direct -A PREROUTING_ZONES -i eth0 -g PRE_public -A PREROUTING_ZONES -g PRE_public -A PRE_public -j PRE_public_log -A PRE_public -j PRE_public_deny -A PRE_public -j PRE_public_allow [root@kvmhost ~]# iptables -S -t raw -P PREROUTING ACCEPT -P OUTPUT ACCEPT -N OUTPUT_direct -N PREROUTING_ZONES -N PREROUTING_ZONES_SOURCE -N PREROUTING_direct -N PRE_public -N PRE_public_allow -N PRE_public_deny -N PRE_public_log -A PREROUTING -j PREROUTING_direct -A PREROUTING -j PREROUTING_ZONES_SOURCE -A PREROUTING -j PREROUTING_ZONES -A OUTPUT -j OUTPUT_direct -A PREROUTING_ZONES -i eth0 -g PRE_public -A PREROUTING_ZONES -g PRE_public -A PRE_public -j PRE_public_log -A PRE_public -j PRE_public_deny -A PRE_public -j PRE_public_allow [root@kvmhost ~]# iptables -S -t security -P INPUT ACCEPT -P FORWARD ACCEPT -P OUTPUT ACCEPT -N FORWARD_direct -N INPUT_direct -N OUTPUT_direct -A INPUT -j INPUT_direct -A FORWARD -j FORWARD_direct -A OUTPUT -j OUTPUT_direct [root@kvmhost ~]# ebtables-save # Generated by ebtables-save v1.0 *nat :PREROUTING ACCEPT :OUTPUT ACCEPT :POSTROUTING ACCEPT :PREROUTING_direct ACCEPT :POSTROUTING_direct ACCEPT :OUTPUT_direct ACCEPT -A PREROUTING -j PREROUTING_direct -A OUTPUT -j OUTPUT_direct -A POSTROUTING -j POSTROUTING_direct -A PREROUTING_direct -j RETURN -A POSTROUTING_direct -j RETURN -A OUTPUT_direct -j RETURN *broute :BROUTING ACCEPT :BROUTING_direct ACCEPT -A BROUTING -j BROUTING_direct -A BROUTING_direct -j RETURN *filter :INPUT ACCEPT :FORWARD ACCEPT :OUTPUT ACCEPT :INPUT_direct ACCEPT :OUTPUT_direct ACCEPT :FORWARD_direct ACCEPT -A INPUT -j INPUT_direct -A FORWARD -j FORWARD_direct -A OUTPUT -j OUTPUT_direct -A INPUT_direct -j RETURN -A OUTPUT_direct -j RETURN -A FORWARD_direct -j RETURN [root@kvmhost ~]# firewall-cmd --state running [root@kvmhost ~]# firewall-cmd --get-default-zone public [root@kvmhost ~]# firewall-cmd --get-active-zones public interfaces: eth0 [root@kvmhost ~]# firewall-cmd --info-zone=public public (active) target: default icmp-block-inversion: no interfaces: eth0 sources: services: dhcpv6-client ssh ports: protocols: masquerade: no forward-ports: source-ports: icmp-blocks: rich rules: [root@kvmhost ~]# firewall-cmd --get-zone-of-interface=virbr0 no zone [root@kvmhost ~]# firewall-cmd --get-zone-of-interface=virbr0-nic no zone [root@kvmhost ~]# firewall-cmd --get-zones block dmz drop external home internal public trusted work