This article explains how to move a KVM virtual disk image file from a plain file to a raw hard disk partition. It then explains how to grow the virtual disk to use the full partition size.

Why using a disk partition for the virtual machine image

Using a plain file for a virtual machine disk image is the easiest configuration when you setup some virtual machine environment. It allows to start quickly for the setup and you can copy easily the virtual machine for a backup.

However, using a raw disk partition for the virtual machine provides better performance in general. The overhead of the guest file system is avoided as we have a direct access to the partition.

Copy the virtual machine image on the partition

To copy the virtual machine image on our partition, the easiest way is to use the dd command. This step assumes that the virtual machine is stopped. In the example, the partition is /dev/sdb10, this partition is bigger than the image file (if this is not the case, the image will be truncated).

$ sudo dd if=windows-xp.img of=/dev/sdb10 bs=1048576
5120+1 records in
5120+1 records out
5368709121 bytes (5.4 GB) copied, 331.51 s, 16.2 MB/s

Resize the virtual disk to the full partition size

The virtual disk partition should be changed to use the full disk space provided by our /dev/sdb10 partition. For this, we can use the fdisk command:

$ sudo fdisk /dev/sdb10
Command (m for help): p
Disk /dev/sdb10: 22.0 GB, 22019042304 bytes
...
     Device Boot      Start         End      Blocks   Id  System
/dev/sdb10p1   *           1         651     5229126    7  HPFS/NTFS

You can easily change the partition to use the full disk by deleting the partition and creating it again so that you get something such as:

Now, we have to resize the file system on the virtual disk partition /dev/sdb10p1. For this, we will use kpartx to get access to the disk partitions provided by our /dev/sdb10 partition:

$ sudo kpartx -v -a /dev/sdb10
add map sdb10p1 (251:1): 0 42989877 linear /dev/sdb10 63

After the partitions are mapped, we can look at the filesystem before resizing it with the ntfsresize command. We use this command to know the good size for resizing the file system.

$ sudo ntfsresize --info /dev/mapper/sdb10p1
ntfsresize v2.0.0 (libntfs 10:0:0)
Device name        : /dev/mapper/sdb10p1
NTFS volume version: 3.1
Cluster size       : 4096 bytes
Current volume size: 5354623488 bytes (5355 MB)
Current device size: <b>22010817024</b> bytes (22011 MB)
Checking filesystem consistency ...
100.00 percent completed
Accounting clusters ...
Space in use       : 4786 MB (89.4%)
Collecting resizing constraints ...
You might resize at 4785565696 bytes or 4786 MB (freeing 569 MB).
Please make a test run using both the -n and -s options before real resizing!

And we can do the resize by using the Current device size as the new file system size.

$ sudo ntfsresize -s 22010817024 /dev/mapper/sdb10p1
ntfsresize v2.0.0 (libntfs 10:0:0)
Device name        : /dev/mapper/sdb10p1
NTFS volume version: 3.1
Cluster size       : 4096 bytes
Current volume size: 5354623488 bytes (5355 MB)
Current device size: 22010817024 bytes (22011 MB)
New volume size    : 22010810880 bytes (22011 MB)
Checking filesystem consistency ...
100.00 percent completed
Accounting clusters ...
Space in use       : 4786 MB (89.4%)
Collecting resizing constraints ...
WARNING: Every sanity check passed and only the dangerous operations left.
Make sure that important data has been backed up! Power outage or computer
crash may result major data loss!
Are you sure you want to proceed (y/[n])? y
Schedule chkdsk for NTFS consistency check at Windows boot time ...
Resetting $LogFile ... (this might take a while)
Updating $BadClust file ...
Updating $Bitmap file ...
Updating Boot record ...
Syncing device ...
Successfully resized NTFS on device '/dev/mapper/sdb10p1'.

At this stage, our virtual machine disk image was moved from a plain file to a raw disk partition that it uses entirely.

Change the virtual machine definition

The virtual machine definition must now be changed to use our partition. You can do this by copying the XML definition to another file, thus creating a new virtual machine. This is the best thing to do so that you can still use the old configuration. If you do such copy, you have to change the uuid as well as the network mac address.

The disk type parameter must be changed to block and the dev parameter must now point to the device partition.

<domain type='kvm'>
  ...
    <disk type='block' device='disk'>
      <source dev='/dev/sdb10'/>
      <target dev='hda' bus='ide'/>
    </disk>
    ...
</domain>

After this, start the virtual machine!

The next step is to setup virtio to boost performance by using paravirtualization.

Having acquired a ReadyNAS duo for my new backup system, I wanted to explore the system that runs on it and see if I could run more services on it. There is nothing terrific in this article as many people have already done

After installing mysql server on a Ready NAS duo, it is necessary to tune the configuration to make the server run well on this small hardware. This article describes a possible configuration for tuning the Mysql server.

Mysql Temporary directory

Mysql uses files in the temporary directory to store temporary tables. Depending on your database and your queries, temporary tables could be quite large. To avoid problems in the /tmp partition becomming full, the best thing is to use a directory in the /c partition

tmpdir          = /c/backup/tmp

Make sure the directory exist before starting mysql:

# mkdir -p /c/backup/tmp

Mysql storage engine

After playing with a reasonably big database and the MyISAM storage engine, it turns out that the mysql server was sometimes crashing and barking at some corrupted myisam tables. I switched to the InnoDB storage engine, which is better for transactions anyway. Since the readynas does not have a lot of memory I've used the following configuration:

default_storage_engine = InnoDB
thread_cache_size = 0

innodb_buffer_pool_size = 6M
innodb_thread_concurrency = 1

Other mysql settings

To reduce the resources used by the mysql server to the minimum, I changed the max number of connections to a small number.

key_buffer_size = 16k
sort_buffer_size = 100k
max_connection = 10

I'm using these settings for almost 6 months now; my bacula database now contains a table with 5 milions of rows. Of course you can't expect big performance but the mysql server is stable.

Being able to connect to my ReadyNAS duo using SSH (See Connecting to a ReadyNAS duo using SSH), the next step for setting up a Bacula backup solution was to setup a MySQL server. Th

When a system lock up occurs, we often blame applications but when you look carefully you may see that despite your multi-core CPU, your applications are sleeping! No cpu activity! So what happens then? Check the I/Os, it could be the root cause!

With Ubuntu 10.04, my desktop computer was freezing when the ReadyNAS Bacula backup was running. Indeed, the Bacula daemon was performing intensive disk operations (on a fast SATA hard disk). The situation was such that it was impossible to use the system, the interface was freezing for a several seconds then working for a few seconds and freezing again.

Linux I/O Scheduler

The I/O scheduler is responsible for organizing the order in which disk operations are performed. Some algorithms allow to minimize the disk head moves, other algorithms tend to anticipate read operations,

When I/O operations are not scheduled correctly, an interactive application such as a desktop or a browser can be blocked until its I/O operations are scheduled and executed (the situation can be even worse for those applications that use the O_SYNC writing mode).

By default, the Linux kernel is configured to use the Completely Fair Queuing scheduler. This I/O scheduler does not provide any time guarantee but it gives in general good performances. Linux provides other I/O schedulers such as the Noop scheduler, the Anticipatory scheduler and the Deadline scheduler.

The deadline scheduler puts an execution time limit to requests to make sure the I/O operation is executed before an expiration time. Typically, a read operation will wait at most 500 ms. This is the I/O scheduler we need to avoid the system lock up.

Checking the I/O Scheduler

To check which I/O scheduler you are using, you can use the following command:

$ cat /sys/block/sda/queue/scheduler
noop anticipatory deadline [cfq]

where sda is the device name of your hard disk (or try hda).

The result indicates the list of supported I/O scheduler as well as the current scheduler used (here the Completely Fair Queuing).

Changing the I/O Scheduler

To change the scheduler, you can echo the desired scheduler name to activate it (you must be root):

# echo deadline >  /sys/block/sda/queue/scheduler

To make sure the I/O scheduler is configured after each system startup, you can add the following lines to your /etc/rc.local startup script:

test -f /sys/block/sda/queue/scheduler &&
  echo deadline > /sys/block/sda/queue/scheduler

test -f /sys/block/sdb/queue/scheduler &&
   echo deadline > /sys/block/sdb/queue/scheduler

test -f /sys/block/hda/queue/scheduler &&
   echo deadline > /sys/block/hda/queue/scheduler

You may have to change the sda and sdb into hda and hdb if you have an IDE hard disk.

Conclusion

After changing the I/O scheduler to use the Deadline scheduler, the desktop was not freezing any more when backups are running.

Tracking changes in a server configuration can be critical to understand problems, identify security breaches and repair a server. When several people are in charge of administering one or several servers, sharing the configuration changes is helpful to i

On Ubuntu 8.04 running a Linux 2.6.24 kernel, you may see some strange error logs reported by dmesg. First you will look at them, you'll wonder where they come from and you will soon ignore them. You should better fix the problem, in most cases they p

Last day the main disk of by computer stopped to work. My Western Digital 150Gb raptor hard disk was no longer recognized by the system: it was simply dead after one year of work. The 10000 rpm di

After having used a RedHat distribution (1998-2002), a Mandrake distribution (2002-2005) and then a Debian distribution (2005-2006), I have switch my systems to an Ubuntu Server and Ubuntu Desktop distribution. Is it worth to switch, well may be. But, st

You have created a Web application using a JBoss application server and you are going to put it in production. Great!

But deploying your application with JBoss serving the Web requests directly may not be the optimal solution. First because the Tomcat