Arch Linux Infrastructure - Brouter Inception - Part 4 - Virtual Router

alt text

Index

Part 01 - Network Switch VLANs

Part 02 - Hypervisor OS Install

Part 03 - Hypervisor OS Setup

Part 04 - Virtual Router - You Are Here!

Part 05 - VoIP Server

Part 06 - Automation Server

Part 07 - NAS

Part 08 - NFTables Transparent TOR Proxy / SSH / IRC

Virtual Router Setup

Start i3-WM

startx

Open Terminal (urxvt)

Alt + Enter

Tip: If you want your right IP showing in the i3status bar “E: no IP (1000 Mbit/s)” edit /etc/i3status.conf

sudo nano /etc/i3status.conf

order += "ipv6"
order += "disk /"
order += "wireless _first_"
order += "ethernet _first_"
order += "battery all"
order += "load"
order += "tztime local"

ethernet _first_ {
        # if you use %speed, i3status requires root privileges
        format_up = "E: %ip (%speed)"
        format_down = "E: down"
}

Change to

order += "ipv6"
order += "disk /"
order += "wireless _first_"
order += "ethernet brv200"
order += "battery all"
order += "load"
order += "tztime local"

ethernet brv200 {
        # if you use %speed, i3status requires root privileges
        format_up = "E: %ip (10 Gbit/s)"
        format_down = "E: down"
}

Now in the i3status bar you should see your IP address getting dhcp inception from the virtual router if it was setup properly and working.

xx##::#x:#xxx:xx#x:#### | 9001 TiB | W: down | E: 192.168.1.100 (10 Gbit/s) | Fully Charged  | 1.18 | 2017-06-28 19:48:50

Make ISO Directory For Hypervisor Media Installs

sudo mkdir /var/lib/libvirt/images/iso
cd /var/lib/libvirt/images/iso

Download Some Install Media

sudo pacman -S wget

Find the latest version

pfSense

Note: RELEASE only works with Firmware: BIOS mode not UEFI! :(

https://www.pfsense.org/download/

sudo wget https://nyifiles.pfsense.org/mirror/downloads/pfSense-CE-2.3.4-RELEASE-amd64.iso.gz
sudo gunzip pfSense-CE-2.3.4-RELEASE-amd64.iso.gz

Note: pfSense 2.4 BETA works with Firmware: UEFI.

https://snapshots.pfsense.org/amd64/pfSense_master/installer/?C=M;O=D

sudo wget https://snapshots.pfsense.org/amd64/pfSense_master/installer/pfSense-CE-2.4.0-BETA-amd64-latest.iso.gz
sudo gunzip pfSense-CE-2.4.0-BETA-amd64-latest.iso.gz

OPNsense

https://mirrors.nycbug.org/pub/opnsense/releases/mirror/

sudo wget http://mirrors.nycbug.org/pub/opnsense/releases/mirror/OPNsense-17.1.4-OpenSSL-cdrom-amd64.iso.bz2
sudo bzip2 -d OPNsense-17.1.4-OpenSSL-cdrom-amd64.iso.bz2

PIAF

https://sourceforge.net/projects/pbxinaflash/files/

sudo wget https://downloads.sourceforge.net/project/pbxinaflash/IncrediblePBX13-12%20with%20Incredible%20PBX%20GUI/IncrediblePBX13.2.iso

Arch Linux

http://mirror.rackspace.com/archlinux/iso/

sudo wget http://mirror.rackspace.com/archlinux/iso/2017.06.01/archlinux-2017.06.01-x86_64.iso

Grab any other distribution install media you require for your hypervisor…

Caution: you will no longer have interenet access after this part until you get the vm router working, so make sure you have everything you need. :)

Setup Network VLAN interfaces

Heres a bash script to create all the interfaces below, run with sudo sh networkcfgs.sh Script Download

Hardware NIC interface eno1

10-eno1.network

sudo nano /etc/systemd/network/10-eno1.network
[Match]
Name=eno1
 
[Network]
VLAN=eno1.100
VLAN=eno1.200
VLAN=eno1.300
VLAN=eno1.400
VLAN=eno1.450
VLAN=eno1.500

WAN VLAN 100

eno1.100.netdev

sudo nano /etc/systemd/network/eno1.100.netdev
[NetDev]
Name=eno1.100
Kind=vlan
 
[VLAN]
Id=100

eno1.100.network

sudo nano /etc/systemd/network/eno1.100.network
[Match]
Name=eno1.100
 
[Network]
Bridge=brv100

brv100.netdev

sudo nano /etc/systemd/network/brv100.netdev
[NetDev]
Name=brv100
Kind=bridge

brv100.network

sudo nano /etc/systemd/network/brv100.network
[Match]
Name=brv100
 
[Network]

LAN VLAN 200

eno1.200.netdev

sudo nano /etc/systemd/network/eno1.200.netdev
[NetDev]
Name=eno1.200
Kind=vlan
 
[VLAN]
Id=200

eno1.200.network

sudo nano /etc/systemd/network/eno1.200.network
[Match]
Name=eno1.200
 
[Network]
Bridge=brv200

brv200.netdev

sudo nano /etc/systemd/network/brv200.netdev
[NetDev]
Name=brv200
Kind=bridge

brv200.network

Note: Setting DHCP=yes here gives the hypervisor a network connection from our LAN VLAN 200 so it can do updates and install software.

sudo nano /etc/systemd/network/brv200.network
[Match]
Name=brv200
 
[Network]
DHCP=yes

Automation VLAN 300

eno1.300.netdev

sudo nano /etc/systemd/network/eno1.300.netdev
[NetDev]
Name=eno1.300
Kind=vlan
 
[VLAN]
Id=300

eno1.300.network

sudo nano /etc/systemd/network/eno1.300.network
[Match]
Name=eno1.300
 
[Network]
Bridge=brv300

brv300.netdev

sudo nano /etc/systemd/network/brv300.netdev
[NetDev]
Name=brv300
Kind=bridge

brv300.network

sudo nano /etc/systemd/network/brv300.network
[Match]
Name=brv300
 
[Network]

GUEST WiFi VLAN 400

eno1.400.netdev

sudo nano /etc/systemd/network/eno1.400.netdev
[NetDev]
Name=eno1.400
Kind=vlan
 
[VLAN]
Id=400

eno1.400.network

sudo nano /etc/systemd/network/eno1.400.network
[Match]
Name=eno1.400
 
[Network]
Bridge=brv400

brv400.netdev

sudo nano /etc/systemd/network/brv400.netdev
[NetDev]
Name=brv400
Kind=bridge

brv400.network

sudo nano /etc/systemd/network/brv400.network
[Match]
Name=brv400
 
[Network]

Main WiFi 450

eno1.450.netdev

sudo nano /etc/systemd/network/eno1.450.netdev
[NetDev]
Name=eno1.450
Kind=vlan
 
[VLAN]
Id=450

eno1.450.network

sudo nano /etc/systemd/network/eno1.450.network
[Match]
Name=eno1.450
 
[Network]
Bridge=brv450

brv450.netdev

sudo nano /etc/systemd/network/brv450.netdev
[NetDev]
Name=brv450
Kind=bridge

brv450.network

sudo nano /etc/systemd/network/brv450.network
[Match]
Name=brv450
 
[Network]

Voice VLAN 500

sudo nano /etc/systemd/network/eno1.500.netdev
[NetDev]
Name=eno1.500
Kind=vlan
 
[VLAN]
Id=500

eno1.500.network

sudo nano /etc/systemd/network/eno1.500.network
[Match]
Name=eno1.500
 
[Network]
Bridge=brv500

brv500.netdev

sudo nano /etc/systemd/network/brv500.netdev
[NetDev]
Name=brv500
Kind=bridge

brv500.network

sudo nano /etc/systemd/network/brv500.network
[Match]
Name=brv500
 
[Network]

Enable systemd-networkd Service

Note: Don’t enable and start this if you still need to setup your virtual router.

sudo systemctl enable systemd-networkd
sudo systemctl start systemd-networkd

Verify Interfaces Have Been Created

ip link

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether f0:1f:af:32:66:4f brd ff:ff:ff:ff:ff:ff
3: brv500: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 9a:de:fd:00:4c:9c brd ff:ff:ff:ff:ff:ff
4: brv450: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether b2:40:b2:19:5d:b2 brd ff:ff:ff:ff:ff:ff
5: brv400: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 82:f9:79:52:52:f5 brd ff:ff:ff:ff:ff:ff
6: brv300: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether ce:0c:c4:f6:9b:41 brd ff:ff:ff:ff:ff:ff
7: brv200: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 56:9c:3a:04:c4:f8 brd ff:ff:ff:ff:ff:ff
8: brv100: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether ae:df:f9:a5:92:e9 brd ff:ff:ff:ff:ff:ff
9: eno1.200@eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master brv200 state UP mode DEFAULT group default qlen 1000
    link/ether f0:1f:af:32:66:4f brd ff:ff:ff:ff:ff:ff
10: eno1.300@eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master brv300 state UP mode DEFAULT group default qlen 1000
    link/ether f0:1f:af:32:66:4f brd ff:ff:ff:ff:ff:ff
11: eno1.400@eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master brv400 state UP mode DEFAULT group default qlen 1000
    link/ether f0:1f:af:32:66:4f brd ff:ff:ff:ff:ff:ff
12: eno1.500@eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master brv500 state UP mode DEFAULT group default qlen 1000
    link/ether f0:1f:af:32:66:4f brd ff:ff:ff:ff:ff:ff
13: eno1.450@eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master brv450 state UP mode DEFAULT group default qlen 1000
    link/ether f0:1f:af:32:66:4f brd ff:ff:ff:ff:ff:ff
14: eno1.100@eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master brv100 state UP mode DEFAULT group default qlen 1000
    link/ether f0:1f:af:32:66:4f brd ff:ff:ff:ff:ff:ff

Configure qemu.conf and libvirtd.conf

Change these settings so your local user can login without sudo.

sudo nano /etc/libvirt/qemu.conf

# The user for QEMU processes run by the system instance. It can be
# specified as a user name or as a user id. The qemu driver will try to
# parse this value first as a name and then, if the name doesn't exist,
# as a user id.
#
# Since a sequence of digits is a valid user name, a leading plus sign
# can be used to ensure that a user id will not be interpreted as a user
# name.
#
# Some examples of valid values are:
#
#       user = "qemu"   # A user named "qemu"
#       user = "+0"     # Super user (uid=0)
#       user = "100"    # A user named "100" or a user with uid=100
#
user = "root"

# The group for QEMU processes run by the system instance. It can be
# specified in a similar way to user.
group = "root"

sudo nano /etc/libvirt/libvirtd.conf

# Network connectivity controls
#

# Flag listening for secure TLS connections on the public TCP/IP port.
# NB, must pass the --listen flag to the libvirtd process for this to
# have any effect.
#
# It is necessary to setup a CA and issue server certificates before
# using this capability.
#
# This is enabled by default, uncomment this to disable it
listen_tls = 0

#################################################################
#
# UNIX socket access controls
#

# Set the UNIX domain socket group ownership. This can be used to
# allow a 'trusted' set of users access to management capabilities
# without becoming root.
#
# This is restricted to 'root' by default.
unix_sock_group = "libvirt"

# Set the UNIX socket permissions for the R/O socket. This is used
# for monitoring VM status only
#
# Default allows any user. If setting group ownership, you may want to
# restrict this too.
unix_sock_ro_perms = "0777"

# Set the UNIX socket permissions for the R/W socket. This is used
# for full management of VMs
#
# Default allows only root. If PolicyKit is enabled on the socket,
# the default will change to allow everyone (eg, 0777)
#
# If not using PolicyKit and setting group ownership for access
# control, then you may want to relax this too.
unix_sock_rw_perms = "0770"


# Set the UNIX socket permissions for the admin interface socket.
#
# Default allows only owner (root), do not change it unless you are
# sure to whom you are exposing the access to.
#unix_sock_admin_perms = "0700"

# Set the name of the directory in which sockets will be found/created.
unix_sock_dir = "/var/run/libvirt"

#################################################################
#
# Authentication.
#
#  - none: do not perform auth checks. If you can connect to the
#          socket you are allowed. This is suitable if there are
#          restrictions on connecting to the socket (eg, UNIX
#          socket permissions), or if there is a lower layer in
#          the network providing auth (eg, TLS/x509 certificates)
#
#  - sasl: use SASL infrastructure. The actual auth scheme is then
#          controlled from /etc/sasl2/libvirt.conf. For the TCP
#          socket only GSSAPI & DIGEST-MD5 mechanisms will be used.
#          For non-TCP or TLS sockets, any scheme is allowed.
#
#  - polkit: use PolicyKit to authenticate. This is only suitable
#            for use on the UNIX sockets. The default policy will
#            require a user to supply their own password to gain
#            full read/write access (aka sudo like), while anyone
#            is allowed read/only access.
#
# Set an authentication scheme for UNIX read-only sockets
# By default socket permissions allow anyone to connect
#
# To restrict monitoring of domains you may wish to enable
# an authentication mechanism here
auth_unix_ro = "none"

# Set an authentication scheme for UNIX read-write sockets
# By default socket permissions only allow root. If PolicyKit
# support was compiled into libvirt, the default will be to
# use 'polkit' auth.
#
# If the unix_sock_rw_perms are changed you may wish to enable
# an authentication mechanism here
auth_unix_rw = "none"


# Listen for unencrypted TCP connections on the public TCP/IP port.
# NB, must pass the --listen flag to the libvirtd process for this to
# have any effect.
#
# Using the TCP socket requires SASL authentication by default. Only
# SASL mechanisms which support data encryption are allowed. This is
# DIGEST_MD5 and GSSAPI (Kerberos5)
#
# This is disabled by default, uncomment this to enable it.
listen_tcp = 1

# Override the default configuration which binds to all network
# interfaces. This can be a numeric IPv4/6 address, or hostname
#
# If the libvirtd service is started in parallel with network
# startup (e.g. with systemd), binding to addresses other than
# the wildcards (0.0.0.0/::) might not be available yet.
#
#listen_addr = "192.168.0.1"
listen_addr = "0.0.0.0"

# Change the authentication scheme for TCP sockets.
#
# If you don't enable SASL, then all TCP traffic is cleartext.
# Don't do this outside of a dev/test scenario. For real world
# use, always enable SASL and use the GSSAPI or DIGEST-MD5
# mechanism in /etc/sasl2/libvirt.conf
#auth_tcp = "sasl"
auth_tcp = "none"

If you start virt-manager without sudo and it gives you “failed to initialize KVM: Permission denied” error try this:

sudo rmmod kvm_intel
sudo rmmod kvm
sudo modprobe kvm
sudo modprobe kvm_intel

The above commands reload the kvm_intel and kvm kernel modules.

Start virt-manager

Note: If you require to run virt-manager from a Windows box checkout the guide here: virt-manager-windows

sudo virt-manager

virt-manager should now be on screen…

Note: If we don’t connect to 0.0.0.0 (local machine) it will make the vm but you won’t be able to see the vnc session.

Click File => Add Connection…

Hypervisor: QEMU/KVM X Connect to remote host Method: SSH Username: root Hostname: 0.0.0.0 Autoconnect: Check Generated URI: qemu+ssh://root@0.0.0.0/system

Click Connect

Now you should be connected to your hypervisor.

Click File => New Virtual Machine

New VM
Create a new virtual machine
Step 1 of 5

Connection: QEMU/KVM

Choose how you would like to install the operating system
* Local install media (ISO image or CDROM)
* Network Install (HTTP,FTP, or NFS)
* Network Boot (PXE)
* Import existing disk image

Choose local install media and click Forward

New VM
Create a new virtual machine
Step 2 of 5

Locate your install media
* Use CDROM or DVD
  No device present
* Use ISO image:
  ComboBox V Browse...
Automatically detect operating system based on install media
OS type: -
Version: -

Click Browse

Choose Storage Volume

On the left bottom click the +

Add a New Storage Pool
Create storage pool
Step 1 of 2

Select the storage pool type you would like to configure.
Name: iso
Type: dir:Filesystem Directory

Click Forward

Add a New Storage Pool
Create storage pool
Step 2 of 2

Target Path: /var/lib/libvirt/images/iso

Click Finish

Now you should see the iso storage pool on the left, click it.

Then click your Virtual Router ISO.

Then click “Choose Volume”.

Leave Automatically detect checked and click Forward.

New VM
Create a new virtual machine
Step 3 of 5

Choose Memory and CPU settings
Memory (RAM): 1024 - + MiB
CPUs: 1 - +
Up to 4 available

Click Forward

New VM
Create a new virtual machine
Step 4 of 5

Enable storage for this virtual machine
* Create a disk image for the virtual machine
  10.0 - + GiB
  100 GiB available in the default location
* Select or create custom storage
  Manage...

Click Forward

New VM
Create a new virtual machine
Step 5 of 5

Ready to begin the installation
Name: VirtualRouter
OS: Generic
Install: Local CDROM/ISO
Memory: 1024 MiB
CPUs: 1
Storage: 10.0 GiB /var/lib/libvirt/images/VirtualRouter.qcow2
X Customize configuration before install

Network selection
* Specify shared device name
  Bridge name: brv100

Click Finish

Now you should see the full virtual machine settings window.

FirewallRouter on QEMU/KVM
Begin Installation    Cancel Installaion

Overview
CPUs
Memory
Boot Options
IDE Disk 1
IDE CDROM 1
NIC :be:ef:10
Mouse
Display Spice
Sound: ich6
Console
Channel spice
Video QXL
Controller USB
USB Redirector 1
USB Redirector 2

Add Hardware

Basic Details
   Name: FirewallRouter
   UUID: xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
   Status: Shutoff (Shut Down)
   Title:
   Description:
Hypervisor Details
   Hypervisor: KVM
   Achitecture: x86_64
   Emulator: /usr/sbin/qemu-system-x86_64
   Firmware: UEFI x86_64: /usr/share/ovmf/ovmf_code_x64.bin
   Chipset: i440FX
   
Cancel Apply

Click Firmware change BIOS to UEFI x86_64: /usr/share/ovmf/ovmf_code_x64.bin

Note: You only get one chance to change this Firmware field, it will go readonly after you Begin the Installaion.

Virtual Network Interface
Network source: Specify shared device name
                Bridge name: brv100
Device model: virtio
MAC address: X 42:de:ad:be:ef:10

Note: Changing the MAC to something you can recognise makes it easier to sort out in the VM Virtual Router we will see that :10 and know it’s for our wan connection and is vlan 100 :20 is our LAN VLAN 200 etc.

Click Apply.

Click Add Hardware in the lower left.

Add New Virtual Hardware
Network
Network source: Specify shared device name
                Bridge name: brv200
MAC address: X 42:de:ad:be:ef:20
Device model: virtio

Click Finish.