 | Level: Intermediate Daniel Robbins (drobbins@gentoo.org), President/CEO, Gentoo Technologies, Inc.
01 Apr 2001 In this article, Daniel shares his experiences converting cvs.gentoo.org's /home filesystem to an LVM logical volume. After the transition, we get to see the benefits of LVM when cvs.gentoo.org's /home partition is dynamically resized in real-time, without rebooting, unmounting /home, or even dropping to runlevel 1. All processes continue to work without any interruption. Daniel's step-by-step details of the conversion will help anyone interested in peforming a similiar transition on their own machine.
In my first LVM article, I explained the concepts behind
LVM. Now it's time to put LVM into action. In this article, I'm going to set
up LVM on the official Gentoo Linux web/cvs/email server, cvs.gentoo.org.
Although cvs.gentoo.org has only one hard drive, LVM's flexibility still
provides an incredible improvement over the standard static partitioning
approach. I'll show you all the steps of the LVM conversion process, so that
if you're interested you can perform a similar conversion on one of your
machines.
Before we begin, a warning. Because implementing LVM is a major change to the
system (involving the creation of new partitions and other potentially
hazardous actions) it's a really good idea to perform a full system
backup before beginning this process. If you're not going to perform a backup,
I hope you're using a test box with no important data on it :) I should
mention that I didn't experience any problems while converting to LVM, but it's
best to be prepared in case something goes wrong.
That said, let's continue. Before starting the conversion process,
I upgraded cvs.gentoo.org so that it was using the following packages. At the
time I performed the LVM transition, these were the latest versions available (see Resources later in this article):
- Linux kernel 2.4.1-ac19
- LVM 0.9.1_beta5
- reiserfs-utils 3.6.25
Now, for the hard drive. cvs.gentoo.org had a nice new IBM 45 GB hard drive
sitting in it; however, when I installed Gentoo Linux on cvs, I only
partitioned about 10 Gigabytes of the drive, keeping the remaining 35 GB for
"future partitions". Such are the little tricks you need to employ when not
using LVM -- leaving part of the drive unpartitioned is a primitive but
effective way to allow for future expansion. However, with LVM there is a
better approach.
The space problem
In the past few weeks, I had been noticing that my root ReiserFS partition had
been slowly filling up, as you can see from this "df" output:
Filesystem 1k-blocks Used Available Use% Mounted on
/dev/hda3 9765200 6989312 2775888 72% /
tmpfs 269052 0 269052 0% /dev/shm
|
Now, a 72% full root partition isn't exactly a crisis, but it isn't
a wonderful situation either. ReiserFS, like many other filesystems, starts
slowing down as it gets more and more full, and it was just a matter of time
before my root filesystem would fill up completely and filesystem performance would take a hit.
I decided to fix
this problem by using LVM to create a new logical volume out of the 35 GB of
currently unpartitioned space at the end of my hard drive. Then, I'd create a
filesystem on this volume and move a good chunk of the contents of /dev/hda3 to
it.
If you're thinking of making a similar transition on one of your machines, the
first thing you need to do is find a suitable piece of your root filesystem to
move to a logical volume. For me, the choice was easy -- my /home tree was
taking up around 5.7 GB. By moving /home to its own LVM logical volume, my root
filesystem would then be at about 20% capacity. Since most new data is being
added to /home, my root filesystem would likely stay at around 20% capacity as
well -- a very healthy situation.
The beginnings of a solution
To begin the conversion, I first had to partition the unused space at the end of
my hard drive. Using cfdisk, I created a 35 GB partition (/dev/hda5) and set the
partition type of the partition to "8E" (the official LVM partition type).
After this change, I rebooted to force a reread of my partition table.
After the reboot, my partition table looked like this:
# sfdisk -l
Disk /dev/hda: 89355 cylinders, 16 heads, 63 sectors/track
Units = cylinders of 516096 bytes, blocks of 1024 bytes, counting from 0
Device Boot Start End #cyls #blocks Id System
/dev/hda1 * 0+ 247 248- 124960+ 83 Linux
/dev/hda2 248 743 496 249984 82 Linux swap
/dev/hda3 744 20119 19376 9765504 83 Linux
/dev/hda4 20120 89354 69235 34894440 5 Extended
/dev/hda5 20120+ 89354 69235- 34894408+ 8e Linux LVM
|
Now that I had an empty 35 GB partition, I was ready to initialize it for LVM.
Here's the procedure -- first, I would initialize the 35 Gigabytes
as a physical volume; then, I would create a volume group using
this physical volume, and finally, I would allocate some of the extents on the volume group,
creating a logical volume that would contain my new filesystem and house
all the files currently in /home.
To begin the process, I used the pvcreate command to initialize /dev/hda5 as a
physical volume:
# pvcreate /dev/hda5
pvcreate -- physical volume "/dev/hda5" successfully created
|
pvcreate set up a special "accounting" area on /dev/hda5, called the VGDA
("volume group descriptor area"). LVM uses this area to keep track of how the
physical extents are allocated, among other things.
My next step was to create a volume group and add /dev/hda5 to this group. The
volume group would act as a pool of extents (chunks of storage blocks). Once
the volume group was created, I could create as many logical volumes as I
wanted. I decided that my volume group would be called "main":
# vgcreate main /dev/hda5
vgcreate -- INFO: using default physical extent size 4 MB
vgcreate -- INFO: maximum logical volume size is 255.99 Gigabyte
vgcreate -- doing automatic backup of volume group "main"
vgcreate -- volume group "main" successfully created and activated
|
The vgcreate command did a couple of things. In addition to creating the "main"
volume group, it also set up /dev/hda5 to use 4 MB extents, the default extent size.
This means that any logical volumes I create from this volume group can
be expanded and shrunk in 4 MB increments.
Due to kernel limitations, the extent size determines the maximum size that
a logical volume can be. As you can see from the above output, a 4 MB extent
size imposes a logical volume size limitation of 256 Gigabytes, which is an
easily attainable logical volume size if you're adding several high-capacity
drives to your volume group. If your volumes could end up being greater than
256 GB apiece, I recommend specifying a larger extent size at vgcreate time.
Extents can range anywhere from 8 KB to 512 MB, and must always be a multiple of
two. By increasing the extent size above 4 MB, the maximum physical volume size
will be scaled accordingly, up to a maximum of 1 Petabyte (although the current
real-world size limit is 2 Terabytes on x86 systems). For example, if I wanted
to create a volume group with 32 Megabyte extents, I'd type:
# vgcreate -s 32M main /dev/hda5
|
32 MB is a good extent size, since a 32 MB granuarity is still manageable and
pushes the maximum logical volume size to 2 Terabytes to boot. Once your
volume group is created, you can view its information by typing "vgdisplay":
# vgdisplay
--- Volume group ---
VG Name main
VG Access read/write
VG Status available/resizable
VG # 0
MAX LV 256
Cur LV 0
Open LV 0
MAX LV Size 255.99 GB
Max PV 256
Cur PV 1
Act PV 1
VG Size 33.28 GB
PE Size 4 MB
Total PE 8519
Alloc PE / Size 0 / 0
Free PE / Size 8519 / 33.28 GB
VG UUID 2qC2H2-iA8s-qW6F-cwXx-JVIh-I6VC-VVCGmn
|
Now that I had my volume group, I was ready to create a logical volume. I
decided to initially make it 8 Gigabytes in size and call it "lv_home":
# lvcreate -L8G -nlv_home main
lvcreate -- doing automatic backup of "main"
lvcreate -- logical volume "/dev/main/lv_home" successfully created
|
Then, I created a filesystem on the volume:
# mkreiserfs /dev/main/lv_home
<----------- MKREISERFSv2 ----------->
Block size 4096 bytes
Block count 2097152
Used blocks 8275
Journal - 8192 blocks (18-8209), journal header is in block 8210
Bitmaps: 17, 32768, 65536, 98304, 131072, 163840,
196608, 229376, 262144, 294912, 327680, 360448,
393216, 425984, 458752, 491520, 524288, 557056,
589824, 622592, 655360, 688128, 720896, 753664,
786432, 819200, 851968, 884736, 917504, 950272,
983040, 1015808, 1048576, 1081344, 1114112,
1146880, 1179648, 1212416, 1245184, 1277952,
1310720, 1343488, 1376256, 1409024, 1441792,
1474560, 1507328, 1540096, 1572864, 1605632,
1638400, 1671168, 1703936, 1736704, 1769472,
1802240, 1835008, 1867776, 1900544, 1933312,
1966080, 1998848, 2031616, 2064384
Root block 8211
Hash function "r5"
ATTENTION: ALL DATA WILL BE LOST ON '/dev/main/lv_home'! (y/n)y
journal size 8192 (from 18)
Initializing journal - 0%....20%....40%....60%....80%....100%
Syncing..done. |
Now that the filesystem was created, I could mount it at /mnt/newhome:
# mkdir /mnt/newhome
# mount /dev/main/lv_home /mnt/newhome
# df
Filesystem 1k-blocks Used Available Use% Mounted on
/dev/hda3 9765200 6989840 2775360 72% /
tmpfs 291388 0 291388 0% /dev/shm
/dev/main/lv_home 8388348 32840 8355508 1% /mnt/newhome
|
As you can see above, I was almost ready to copy over all my data in /home.
Before I began, I dropped to runlevel 1 to ensure that no users or processes
would be accessing or modifying files in /home as they were being copied over:
Then, I began copying the files:
# cp -avx /home/* /mnt/newhome
|
The copy completed in about ten minutes. Then, I backed up my original /home
to /home.old, just in case something was wrong with my copy. I created a new
mount point, and remounted the new home at /home:
# cd /
# mv home home.old
# mkdir home
# umount /mnt/newhome
# mount /dev/main/lv_home /home
|
Then, it was time to set up the server so that my new /home partition would be
available every time the machine started up. First, I modified my /etc/fstab
so that it included a new /home entry:
# /etc/fstab: static file system information.
#
# fs mountpoint type opts dump/pass
/dev/hda3 / reiserfs defaults 1 1
/dev/main/lv_home /home reiserfs defaults 2 2
/dev/hda2 none swap sw 0 0
/dev/hda1 /boot reiserfs noauto 0 0
/dev/cdrom /mnt/cdrom iso9660 noauto,ro 0 0
proc /proc proc defaults 0 0
none /dev/pts devpts mode=620 0 0
tmpfs /dev/shm tmpfs defaults 0 0
|
Then, I made minor modifications to my initialization scripts. I modified my
"checkroot" startup script so that the following commands would run immediately
after my root partition was remounted read/write:
/sbin/vgscan
/sbin/vgchange -a y
|
Then, I modified my filesystem unmounting script that gets run on shutdown, so
that the following command would run immediately after all filesystems
were unmounted:
Once I had completed these steps, I rebooted the machine, and to my delight
everything worked perfectly. After a day or so of absolutely no problems, I
deleted /home.old to free up some space on my root filesystem. Yay! The
transition to LVM was a success.
The beauty of LVM
While the transition to LVM is a bit of an ordeal, once the transition is
complete, managing filesystems becomes tremendously easier. As an example, I
decided to resize my new /home logical volume, adding about 2 Gigabytes worth
of space to the end of the filesystem. First, I added additional capacity to
my "lv_home" logical volume, and then I used the resize_reiserfs utility to
expand the filesystem so that it would use this additional capacity. Here are
the two commands that did all this:
# lvextend -L+2G /dev/main/lv_home
# resize_reiserfs -f /dev/main/lv_home
|
In about a second, I had enlarged my /home filesystem by 2 GB; amazingly, I
didn't need to reboot, drop to runlevel 1, or even unmount /home to perform the
resize. Everything continued to work as it had before. Isn't that great?
Here's the current state of my filesystems:
# df
Filesystem 1k-blocks Used Available Use% Mounted on
/dev/hda3 9765200 1413340 8351860 15% /
/dev/main/lv_home 10485436 5609836 4875600 54% /home
|
You
can see how LVM really can make an administrator's work a whole lot easier. In
the future, I hope to move additional parts of my root filesystem over to LVM,
and eventually even convert my root filesystem over to an LVM logical volume.
The resources below will help you learn even more about LVM.
Resources
- For LVM concepts and advice on get the latest kernel patches
and tools installed on your system, see Daniel's previous article, Learning Linux LVM, Part 1 on developerWorks.
- Download the LVM tarball from Sistina Software.
- Sistina now has an excellent LVM HOWTO available (it's closer to a full-blown manual than a simple HOWTO).
- Be sure to check out the Linux LVM FAQ.
- The impatient will want to check out Heinz Mauelshagen's LVM quick-start document, which contains more examples on how to set up volume groups and logical volumes.
- There's also an interesting HOWTO that shows you
how to set up your root filesystem on a logical volume. Once LVM-0.9.1_final is out, I may try doing this.
- Andreas Dilger is involved with the Linux LVM project and has a nice-looking
online ext2 filesystem resizer.
-
ReiserFS is an excellent filesystem (especially in combination with LVM). If you're using ReiserFS, you'll want to grab the reiserfs-utils tarball, which contains a program called "reiserfs_resize" -- allowing online resizing of ReiserFS filesystems.
- For more information on setting up Linux software RAID volumes, see Part 1 and Part 2 of Daniel's developerWorks series on software RAID.
- For a refresher, see the developerWorks
tutorial on compiling the Linux kernel.
About the author | | | Daniel Robbins lives in Albuquerque, New Mexico. He is the President/CEO of Gentoo Technologies Inc., the Chief Architect of the Gentoo Project and a contributing
author of several books published by MacMillan: Caldera OpenLinux Unleashed, SuSE Linux
Unleashed, and Samba Unleashed. Daniel has been involved with computers in some fashion since the second grade when he was first exposed to the Logo programming language and a potentially lethal dose of Pac Man. This probably explains why he has since served as a Lead Graphic Artist at SONY Electronic Publishing/Psygnosis. Daniel enjoys spending time with his wife Mary and his new baby daughter, Hadassah. You can contact Daniel at drobbins@gentoo.org. |
Rate this page
| |
