21 Apr 17 How to create a sparse file from a block device
This is how to create a sparse file dump from a block device:
cp --sparse=always <(dd if=/dev/vg0/vmservice1 bs=8M iflag=direct | pv -pre --size=20G) /opt/backups/vmservice1.dat
04 Nov 15 Rescan iSCSI volume after resize on linux
If you need to resize a iSCSI volume you need to:
- resize the volume on the iSCSI target (ietd)
- rescan the volume on the iSCSI initiator (open-iscsi)
- resize the fs, if any
I’ll skip the resize procedure on the target, because it depends on how it’s made (lvresize, dd, etc).
The procedure to rescan the volume on the initiator (open-iscsi) is very simple and can be accomplished online.
iscsiadm -m node -R
Then, you can grow the filesystem, if any (xfs_grofs, resize_reiserfs, resize2fs, depending on your fs type).
27 Gen 15 How to get the device mapper name associated to LVM logical volumes
This is how to get the device mapper name (dm-1, dm-2, etc) associated to each LVM logical volume:
lvdisplay|awk '/LV Name/{n=$3} /Block device/{d=$3; sub(".*:","dm-",d); print d,n;}'
15 Nov 14 Dump and restore block device data on the fly by the network
Sometimes you may need to copy data from a block device (or LVM logical volume or snapshot) from one server to another., but you don’t want to dump the image to disk, move to the other server, then import. You may need (or just want) to copy on the fly, transfering data on the net.
To do this, and have ETA on the operation you need the pv executable. The command nc is used to stream data on the network, while pigz is used to compress data (gzip uses just one CPU, while pigz uses all available CPU, and it’s much faster).
On the origin server (server1) you have a block device (lvm logical volume in this case) called /dev/vg0/vm-111-disk-1, while on the destination server (server2) you want to overwrite a LVM logical volume called /dev/vg0/vm-112-disk-1 with data coming from the origin server.
To do this, assuming the device is big 20GB, you may run those commands:
Server side (destination server, server2, ip 192.168.0.2):
nc -l -n -p 2102 -q 2 | pigz -d | pv -pre –size=20G | dd iflag=fullblock bs=512k of=/dev/vg0/vm-112-disk-1
Client side (origin server, server1, 192.168.0.1):
dd if=/dev/vg0/vm-111-disk-1 bs=512k | pv -pre –size=20G | pigz | nc -q 2 192.168.0.2 2102
Data will be read, compressed, transfered on the network on (port TCP 2102 on our case, from 192.168.0.1 to 192.168.0.2), uncompressed on the destination server and restored on disk, and you’ll have ETA and progress indication:
Output server side (destination server, server2):
root@server2 ~ # nc -l -n -p 2102 -q 2 | pigz -d | pv -pre –size=20G | dd iflag=fullblock bs=512k of=/dev/vg0/vm-112-disk-1
[71.2MB/s] [=========================================================================================================================================>] 100%
40960+0 records in
40960+0 records out
21474836480 bytes (21 GB) copied, 296.436 s, 72.4 MB/s
Output client side (origin server, server1):
root@server1 ~ # dd if=/dev/vg0/vm-111-disk-1 bs=512k | pv -pre –size=20G | pigz | nc -q 2 192.168.0.2 2102
[72.2MB/s]
[=========================================================================================================================================>]
100%
40960+0 records
40960+0 records out
21474836480 bytes (21 GB) copied, 283.531 s, 75.7 MB/s
14 Nov 13 LVM Hot backups with snapshot
As you may know, LVM make it possible to create live snapshots of running logical volumes.
Imagine a guest virtual machine that has its virtual disk backed on a LVM logical volume on the host system.
You may create a live hot backup of your virtual machine on the fly, while it is working.
To do this, I created a small script that makes a compressed backup of all the logical volumes on the /dev/vg0 volume group.
The script make use of the standard LVM utilities to have the snapshot, the pv utility to get a cool progress bar and pigz utility to compress (gzip) using all of your processors.
If everything went ok, when the script finishes you’ll find your LVM hot backups on the /backups directory, and the temporary lvm snapshots removed.
This is how I make hot backups of some of my virtual machines (lvm_hot_backup.sh):
#!/bin/bash
for lv in `lvdisplay /dev/vg0 | grep ‘LV Name’ | awk ‘{print $3}’`
do
LV_SIZE=”`lvs –units m –noheadings –nosuffix $lv | cut -d’ ‘ -f7 | cut -d. -f 1`” # LV size in MB
LV_UUID=”`lvdisplay $lv | grep ‘LV UUID’ | awk -F’LV UUID’ ‘{print $2}’ | sed ‘s/^ *//g’`”
LV_SNAPNAME=”SNAP_`basename $lv`”echo “LVM Logical Volume: $lv”
echo “Size: $LV_SIZE MB”
echo “UUID: $LV_UUID”
echo “Snapshot name: $LV_SNAPNAME”
echo “Removing old snapshot (if any)…”
lvremove -f “/dev/vg0/$LV_SNAPNAME”
echo “Creating snapshot…”
lvcreate -L+2G –snapshot -n”$LV_SNAPNAME” “$lv”
sleep 4
echo “Backing up snapshot…”
dd if=”/dev/vg0/$LV_SNAPNAME” bs=512k of=/dev/stdout | pv -pterbW -i 2 –buffer-size 512k –size “$LV_SIZE”m | /usr/bin/pigz -9 -b 256 > “/backups/$LV_SNAPNAME.lv.gz”
echo “Removing snapshot…”
lvremove -f “/dev/vg0/$LV_SNAPNAME”
echo “–”
done
Follow me!
Stack Overflow
IPv6 Certified (he.net)