X

Lumen help

Connecting a CentOS bare metal server to Lumen Network Storage

After you have configured a node on a Lumen® Network Storage service, you can connect it to a bare metal server. Below are the steps and terminal commands necessary to mount the node to the server.
 

Using an Ubuntu bare metal server? Learn how to connect an Ubuntu bare metal server to Network Storage

There are seven steps to connect a CentOS bare metal server to Network Storage:

Step 1. Update the system and software

  1. Check for updates.
                yum check-update 

yum update -y 
            
                yum -y install traceroute
  1. Install NFS tools.
                yum -y install nfs-utils
  1. Install iSCSI tools.
  1. Install traceroute.
                rpm -qa |grep device-mapper 

lsmod |grep dm_multipath 

yum -y install device-mapper-multipath 

/sbin/mpathconf --enable 

multipath -ll 

vi /etc/multipath.conf 

chkconfig multipathd on 

yum -y install iscsi-initiator-utils

Step 2: Set up /etc/hosts for Network Storage

Example /etc/hosts for NFS only:

                10.10.10.11      SNC01-NFS1
10.10.10.12      SNC02-NFS2

#YOUR ASSIGNED IP ADDRESS
10.10.10.21     LOCALHOST-NFS

Example /etc/hosts for NFS and iSCSI:

                10.10.10.11      SNC01-NFS1
10.10.10.12      SNC02-NFS2
10.10.10.16      SNC01-ISCSI1 
10.10.10.15      SNC02-ISCSI1 
10.10.10.14      SNC01-ISCSI2 
10.10.10.13      SNC02-ISCSI2 

#YOUR ASSIGNED IP ADDRESS
10.10.10.21     LOCALHOST-NFS

Step 3: Create the /etc/sysconfig/network-scripts/ifcfg-bond0.<VLAN#> configuration

Example of bond for vlan4 for IP 10.10.10.21:

                cd /etc/sysconfig/network-scripts 

more ifcfg-bond0.4 

VLAN=yes 
TYPE=Vlan 
PHYSDEV=bond0 
DEVICE=bond0.4 
VLAN_ID=4 
REORDER_HDR=yes 
GVRP=no 
MVRP=no 
HWADDR= 
PROXY_METHOD=none 
BROWSER_ONLY=no 
BOOTPROTO=none 
IPADDR=10.10.10.21 
PREFIX=24 
DEFROUTE=no 
IPV4_FAILURE_FATAL=no 
IPV6INIT=no 
IPV6_AUTOCONF=no 
IPV6_DEFROUTE=no 
IPV6_FAILURE_FATAL=no 
IPV6_ADDR_GEN_MODE=stable-privacy 
NAME=bond0.4 
ONBOOT=no

Important: Leave ONBOOT=no until you are sure the configuration operates properly. Change the value to ONBOOT=yes after proper functionality is confirmed.

                # Start vlan 4
ifup bond0.4

# If you need stop vlan 4 at a later date
ifdown bond0.4

Step 4: Verify network connectivity by pinging the local, NFS, and iSCSI IP addresses

Example:

                ping -c 3 LOCALHOST-LNS 

PING LOCALHOST-LNS (10.10.10.21) 56(84) bytes of data. 

64 bytes from LOCALHOST-LNS (10.10.10.21): icmp_seq=1 ttl=64 time=0.019 ms 
64 bytes from LOCALHOST-LNS (10.10.10.21): icmp_seq=2 ttl=64 time=0.008 ms 
64 bytes from LOCALHOST-LNS (10.10.10.21): icmp_seq=3 ttl=64 time=0.006 ms 

--- LOCALHOST-LNS ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2002ms 
rtt min/avg/max/mdev = 0.006/0.011/0.019/0.005 ms 

ping -c 3 SNC01-NFS1 

PING SNC01-NFS1 (10.10.10.11) 56(84) bytes of data. 

64 bytes from SNC01-NFS1 (10.10.10.11): icmp_seq=1 ttl=64 time=0.069 ms 
64 bytes from SNC01-NFS1 (10.10.10.11): icmp_seq=2 ttl=64 time=0.064 ms 
64 bytes from SNC01-NFS1 (10.10.10.11): icmp_seq=3 ttl=64 time=0.060 ms 

--- SNC01-NFS1 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 1999ms 
rtt min/avg/max/mdev = 0.060/0.064/0.069/0.007 ms 

ping -c 3 SNC02-NFS2 

PING SNC02-NFS2 (10.10.10.12) 56(84) bytes of data. 

64 bytes from SNC02-NFS2 (10.10.10.12): icmp_seq=1 ttl=64 time=0.084 ms 
64 bytes from SNC02-NFS2 (10.10.10.12): icmp_seq=2 ttl=64 time=0.072 ms 
64 bytes from SNC02-NFS2 (10.10.10.12): icmp_seq=3 ttl=64 time=0.062 ms 

--- SNC02-NFS2 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2003ms 
rtt min/avg/max/mdev = 0.062/0.072/0.084/0.013 ms 

ping -c 3 SNC01-ISCSI1 

PING SNC01-ISCSI1 (10.10.10.16) 56(84) bytes of data. 

64 bytes from SNC01-ISCSI1 (10.10.10.16): icmp_seq=1 ttl=64 time=0.080 ms 
64 bytes from SNC01-ISCSI1 (10.10.10.16): icmp_seq=2 ttl=64 time=0.075 ms 
64 bytes from SNC01-ISCSI1 (10.10.10.16): icmp_seq=3 ttl=64 time=0.065 ms 

--- SNC01-ISCSI1 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2002ms 
rtt min/avg/max/mdev = 0.065/0.073/0.080/0.009 ms 

ping -c 3 SNC02-ISCSI1 

PING SNC02-ISCSI1 (10.10.10.15) 56(84) bytes of data. 

64 bytes from SNC02-ISCSI1 (10.10.10.15): icmp_seq=1 ttl=64 time=0.083 ms 
64 bytes from SNC02-ISCSI1 (10.10.10.15): icmp_seq=2 ttl=64 time=0.075 ms 
64 bytes from SNC02-ISCSI1 (10.10.10.15): icmp_seq=3 ttl=64 time=0.068 ms 

--- SNC02-ISCSI1 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2002ms 
rtt min/avg/max/mdev = 0.068/0.075/0.083/0.009 ms 

ping -c 3 SNC01-ISCSI2 

PING SNC01-ISCSI2 (10.10.10.14) 56(84) bytes of data. 

64 bytes from SNC01-ISCSI2 (10.10.10.14): icmp_seq=1 ttl=64 time=0.080 ms 
64 bytes from SNC01-ISCSI2 (10.10.10.14): icmp_seq=2 ttl=64 time=0.077 ms 
64 bytes from SNC01-ISCSI2 (10.10.10.14): icmp_seq=3 ttl=64 time=0.058 ms 

--- SNC01-ISCSI2 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2000ms 
rtt min/avg/max/mdev = 0.058/0.071/0.080/0.013 ms 

ping -c 3 SNC02-ISCSI2 

PING SNC02-ISCSI2 (10.10.10.13) 56(84) bytes of data. 

64 bytes from SNC02-ISCSI2 (10.10.10.13): icmp_seq=1 ttl=64 time=0.078 ms 
64 bytes from SNC02-ISCSI2 (10.10.10.13): icmp_seq=2 ttl=64 time=0.073 ms 
64 bytes from SNC02-ISCSI2 (10.10.10.13): icmp_seq=3 ttl=64 time=0.065 ms 

--- SNC02-ISCSI2 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2002ms 
rtt min/avg/max/mdev = 0.065/0.072/0.078/0.005 ms

Step 5: Set up NFS

  1. Show the available mounts.
                showmount --exports <ip_address/Host_name of the nfs-server>

Example: 

                showmount --exports <ip_address/Host_name of the nfs-server> 

Example output:  

showmount --exports SNC01-NFS1 

Export list for SNC01-NFS1: 

/LNS2ML4ECTST00006VOL01 (everyone) 

/LNS2ML4ECTST00006VOL02 (everyone) 

/LNS2ML4ECTST00006VOL03 (everyone)
  1. Create mount points for each volume.
                mkdir -p /nfs1 

mkdir -p /nfs2 

mkdir -p /nfs3
  1. Mount a NFS volume. 
                mount -t nfs -o options <nfs-server:/remote/share_directory>  </local/mount_directory>

Example:

                mount -t nfs -o clientaddr=10.10.10.21 SNC01-NFS1:/LNS2ML4ECTST00006VOL01 /nfs1 

mount -t nfs -o clientaddr=10.10.10.21 SNC02-NFS2:/LNS2ML4ECTST00006VOL02 /nfs2 

mount -t nfs -o clientaddr=10.10.10.21 SNC01-NFS1:/LNS2ML4ECTST00006VOL03 /nfs3 

df -BG 

Filesystem                                      1G-blocks  Used Available Use% Mounted on 

SNC01-NFS1:/LNS2ML4ECTST00006VOL01       72G    1G       72G   1% /nfs1 

SNC02-NFS2:/LNS2ML4ECTST00006VOL02       74G    1G       74G   1% /nfs2 

SNC01-NFS1:/LNS2ML4ECTST00006VOL03       87G    1G       87G   1% /nfs3
  1. Ensure the NFS mounts are writable.
                cd /nfs1 

dd if=/dev/urandom of=./zz1 bs=10k count=10000 

ls -al 

cd /nfs2 

dd if=/dev/urandom of=./zz2 bs=100k count=10000 

ls -al 

cd /nfs3 

dd if=/dev/urandom of=./zz3 bs=200k count=10000 

ls -al
  1. If you choose, you can unmount the NFS mount.
                cd /root 

umount /nfs1 

umount /nfs2 

umount /nfs3

Example of current mounts in the FSTAB NFS entries:

                Filesystem                                      1G-blocks  Used Available Use% Mounted on 

<SNIP> 

ML4ECTSTS0201SNC01-NFS1:/LNS2ML4ECTST00006VOL01       72G    1G       72G   1% /nfs1 

ML4ECTSTS0201SNC02-NFS2:/LNS2ML4ECTST00006VOL02       74G    1G       73G   2% /nfs2 

ML4ECTSTS0201SNC01-NFS1:/LNS2ML4ECTST00006VOL03       87G    2G       85G   3% /nfs3

Step 6: Set up iSCSI

  1. Show your wwids.
                more /etc/multipath/wwids

Important: wwids will be empty immediately after setup.

  1. Check or fix multiplat.cond.

Example:

                defaults { 

        user_friendly_names yes 

        find_multipaths yes 

}
  1. Set up iSCSI service.
                # Setup 

mkdir /iscsi0 

cd /iscsi0 

ls -al 

df -k 

cd
systemctl restart iscsid.service
  1. Locate your client machine's IQN located at /etc/iscsi/initiatorname.iscsi. You will need this to add an iSCSI volume in the Network Storage portal (located within Control Center), This value will be used in the Mapped Client IQN field when you create a volume.

Example:

                InitiatorName=iqn.1994-05.com.redhat:fff7ba98df65
  1. Create a volume before proceeding to the next steps.
  1. Discover the target. This will list the volumes you created.
                iscsiadm -m discovery -t sendtargets -p <IP Address/Hostname>

Example:

                iscsiadm -m discovery -t sendtargets -p SNC01-ISCSI1 
10.10.10.16:3260,1041 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.13:3260,1044 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.14:3260,1043 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.15:3260,1042 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
iscsiadm -m discovery -t sendtargets -p SNC02-ISCSI1 
10.10.10.15:3260,1042 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.13:3260,1044 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.14:3260,1043 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.16:3260,1041 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
iscsiadm -m discovery -t sendtargets -p SNC01-ISCSI2 
10.10.10.14:3260,1043 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.13:3260,1044 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.15:3260,1042 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.16:3260,1041 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
iscsiadm -m discovery -t sendtargets -p SNC02-ISCSI2 
10.10.10.13:3260,1044 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.14:3260,1043 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.15:3260,1042 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.16:3260,1041 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39
  1. Discover any blocked devices.
                lsblk -e7

Example:

                lsblk -e7 

NAME    MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT 

sda       8:0    0   1.8T  0 disk 

sdb       8:16   0   1.8T  0 disk 

sdc       8:32   0   1.8T  0 disk 

sdd       8:48   0   1.8T  0 disk 

sde       8:64   0 223.5G  0 disk 

├─sde1    8:65   0   512M  0 part /boot/efi 

└─sde2    8:66   0   223G  0 part / 

nvme0n1 259:0    0   1.5T  0 disk 

nvme1n1 259:1    0   1.5T  0 disk
  1. Use iscsiadmin to list all target information. (When running this command, you may experience timeouts. If this occurs, run the command again.)

Example:

                iscsiadm --mode node -l all

Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39, portal: 10.10.10.14,3260] (multiple) 
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39, portal: 10.10.10.16,3260] (multiple) 
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39, portal: 10.10.10.15,3260] (multiple) 
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39, portal: 10.10.10.13,3260] (multiple) 
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39, portal: 10.10.10.14,3260] successful. 
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39, portal: 10.10.10.16,3260] successful. 
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39, portal: 10.10.10.15,3260] successful. 
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39, portal: 10.10.10.13,3260] successful.
  1. Verify the disks are visible.

Example:

                lsblk -e7 

NAME    MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT 

sda       8:0    0   1.8T  0 disk 

sdb       8:16   0   1.8T  0 disk 

sdc       8:32   0   1.8T  0 disk 

sdd       8:48   0   1.8T  0 disk 

sde       8:64   0 223.5G  0 disk 

├─sde1    8:65   0   512M  0 part /boot/efi 

└─sde2    8:66   0   223G  0 part / 

sdf       8:80   0   105G  0 disk 

sdg       8:96   0   105G  0 disk 

sdh       8:112  0   105G  0 disk 

sdi       8:128  0   105G  0 disk 

nvme0n1 259:0    0   1.5T  0 disk 

nvme1n1 259:1    0   1.5T  0 disk
  1. Install the 7.1 version of the Linux Unified Host Utilities.
  1. Register with NetApp.

Example:

                rpm -ivh netapp_linux_unified_host_utilities-7-1.x86_64.rpm 

Preparing...                          ################################# [100%] 

Warning: libnl.so library not found, some sanlun commands may not work. Refer Linux Host Utilities Installation and Set up Guide for more details 
Warning: libHBAAPI.so library not found, some sanlun commands may not work. Refer Linux Host Utilities Installation and Setup Guide for more details 
Updating / installing... 
1:netapp_linux_unified_host_utiliti################################# [100%]
  1. Restart, enable, and start multipath.
                systemctl restart iscsid.service 

systemctl enable multipathd 

systemctl start multipathd
  1. Recheck multipath.
                multipath -r 

reload: 3600a098038314630452b532f46714a50 undef NETAPP  ,LUN C-Mode 

size=105G features='3 queue_if_no_path pg_init_retries 50' hwhandler='0' wp=undef 

|-+- policy='service-time 0' prio=50 status=undef 

| |- 18:0:0:0 sdg     8:96  active ready running 

| `- 21:0:0:0 sdh     8:112 active ready running 

`-+- policy='service-time 0' prio=10 status=undef 

  |- 19:0:0:0 sdf     8:80  active ready running 

  `- 20:0:0:0 sdi     8:128 active ready running 

multipath -ll 

3600a098038314630452b532f46714a50 dm-0 NETAPP  ,LUN C-Mode 

size=105G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw 

|-+- policy='service-time 0' prio=50 status=enabled 

| |- 18:0:0:0 sdg     8:96  active ready running 

| `- 21:0:0:0 sdh     8:112 active ready running 

`-+- policy='service-time 0' prio=10 status=enabled 

  |- 19:0:0:0 sdf     8:80  active ready running 

  `- 20:0:0:0 sdi     8:128 active ready running 

 

lsblk -e7 

NAME                                MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT 

sda                                   8:0    0   1.8T  0 disk 

sdb                                   8:16   0   1.8T  0 disk 

sdc                                   8:32   0   1.8T  0 disk 

sdd                                   8:48   0   1.8T  0 disk 

sde                                   8:64   0 223.5G  0 disk 

├─sde1                                8:65   0   512M  0 part  /boot/efi 

└─sde2                                8:66   0   223G  0 part  / 

sdf                                   8:80   0   105G  0 disk 

└─3600a098038314630452b532f46714a50 253:0    0   105G  0 mpath 

sdg                                   8:96   0   105G  0 disk 

└─3600a098038314630452b532f46714a50 253:0    0   105G  0 mpath 

sdh                                   8:112  0   105G  0 disk 

└─3600a098038314630452b532f46714a50 253:0    0   105G  0 mpath 

sdi                                   8:128  0   105G  0 disk 

└─3600a098038314630452b532f46714a50 253:0    0   105G  0 mpath 

nvme0n1                             259:0    0   1.5T  0 disk 

nvme1n1                             259:1    0   1.5T  0 disk 

more /etc/multipath/wwids
  1. Set the multipath.
                multipath -f /dev/mapper/<WWID> 

Important: The multipath wwids are located in /etc/multipath/wwids.  

Example:  

multipath -f /dev/mapper/3600a098038314630452b532f46714a50 

 

multipath -r 

 

Example output: 

create: 3600a098038314630452b532f46714a50 undef NETAPP  ,LUN C-Mode 

size=105G features='3 queue_if_no_path pg_init_retries 50' hwhandler='0' wp=undef 

|-+- policy='service-time 0' prio=50 status=undef 

| |- 18:0:0:0 sdg     8:96  undef ready running 

| `- 21:0:0:0 sdh     8:112 undef ready running 

`-+- policy='service-time 0' prio=10 status=undef 

  |- 19:0:0:0 sdf     8:80  undef ready running 

  `- 20:0:0:0 sdi     8:128 undef ready running 

 

Example output: 

3600a098038314630452b532f46714a50 dm-0 NETAPP  ,LUN C-Mode 

size=105G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw 

|-+- policy='service-time 0' prio=50 status=enabled 

| |- 18:0:0:0 sdg     8:96  active ready running 

| `- 21:0:0:0 sdh     8:112 active ready running 

`-+- policy='service-time 0' prio=10 status=enabled 

  |- 19:0:0:0 sdf     8:80  active ready running 

  `- 20:0:0:0 sdi     8:128 active ready running 

 

Example devices for multipath:  

ls -al /dev/dm* 

brw-rw----. 1 root disk 253, 0 Mar 23 21:43 /dev/dm-0 

ls -al /dev/mapper/3600a098038314630452b532f46714a50 

lrwxrwxrwx. 1 root root 7 Mar 23 21:43 /dev/mapper/3600a098038314630452b532f46714a50 -> ../dm-0
  1. Put an ext4 filesystem on /dev.dm-0.
                mkfs -t ext4 /dev/dm-0
  1. Create a mountpoint.
                mkfs -t ext4 /dev/dm-0
  1. Add the mountpoint to fstab.
                more /etc/fstab
  1. Mount the iSCSI volume.
                mount /iscsi0

Step 7: Display the available storage space

                df -BG 

Filesystem                                      1G-blocks  Used Available Use% Mounted on 

<SNIP> 

/dev/mapper/3600a098038314630452b532f46714a50        104G    1G       98G   1% /iscsi0

Step 8: Write test data to /iscsi0

                cd /iscsi0/ 

dd if=/dev/urandom of=./zz0 bs=1000k count=4000 

4000+0 records in 

4000+0 records out 

4096000000 bytes (4.1 GB) copied, 23.7238 s, 173 MB/s 

# Note - /dev/urandom max is about ~150 MB/s 

df -BG /iscsi0 

Filesystem                                    1G-blocks  Used Available Use% Mounted on 

/dev/mapper/3600a098038314630452b532f46714a50      104G    4G       95G   4% /iscsi0