Migrating from Amazon Web Services PV to Google Cloud Engine

Migrating from Amazon Web Services PV to Google Cloud Engine

At ATADATA, we are always looking for ways to solve potential migration problems, even before they occur. Most members of our technical team have more than a decade of experience working on server migrations, and we have deep empathy for our customers having been in their shoes exploring the same migration challenges for years.

Sometime ago, when we started to code our Linux migrations software for ATAmotion, we faced the challenge of migrating from paravirtualized (PV) [1] guests. PV instances are sometimes difficult to migrate because they are special in some ways.

Firstly, the virtualization is lightweight, not fully emulated, the instances share a large part of the kernel resources with the host, and they usually have either a customized kernel for PV or special kernel modules that allows for the interaction with the host kernel. Moreover, they usually run on low-end or old hardware that does not have hardware-assisted virtualization support. To further complicate the problem, AWS only supports PV instances on 5 instance types and in some new regions, like Ohio (US East), they do not support PV at all.  In our opinion, since its support is very limited, Amazon might want to get rid of dealing with this kind of virtualization sometime in future.

Secondly, the bootloader is usually not installed on the guest OS because Xen (the hypervisor AWS uses) boots the kernel directly using a small bootloader called pygrub [2]. It does not boot from the MBR as traditional guests would.

After identifying the challenges above, our team formulated a plan to add the necessary code in ATAmotion to fix these problems, on the fly, while the server was migrated to the target cloud with zero changes in the source server.

Our Approach:

1. For the customized kernel or special kernel modules, we ensure the target has a kernel compatible with the target hypervisor. Moreover, we regenerate the file that stores the kernel drivers used at boot time adding all the necessary drivers for the target hardware.

2. We install the bootloader that is appropriate for the Linux distribution that is being migrated and the target cloud. Furthermore, we update the boot loader configuration to have the recommended settings for the target cloud.

To deliver some context, here is an example:

We have seen challenges with Google Cloud Engine. Some boot loader parameters that would be set by any Linux installer in any normal VM, like the picture that is displayed when the server boots, will simply not work in GCE. Updating the boot loader configuration here is completely necessary otherwise, the instance will not boot.

We tested migrations from AWS PV to AWS HVM, VMware ESX, IBM Softlayer, Google Cloud Engine (GCE), etc.

Our team recently had the privilege to help R9 [3] move their servers from AWS PV to GCE. GCE uses KVM as hypervisor. KVM does not support PV, instead they emulate the full hardware of the guest VM. This is more like Xen Hardware Virtual Machine (HVM).

Using our technology, they could move around 20 servers in a week with absolutely zero downtime, nor performance impact on their servers. ATAmotion applied all the GCE tweaks and they could move from PV seamlessly, since we had this feature already built in.

We had to update our tool to support Debian PVs. Since this use case was not previously tested, we had to update our logic to handle Debian boot loader configuration. However, we provided  an updated version of our tool within hours and before the migration wave even started.

We also worked closely with the client to ensure they succeeded. Any small issues faced during the migrations were fixed jointly since the client was very knowledgeable. They were very happy with the product and the quality of our Linux engineering and support team. 

ATAmotion is the only product in the market supporting migrations from paravirtualized guests and a substantial reason R9 trusted ATADATA to do their server migrations.

Written By: Matias (Matt) Kreder

Head of the Linux Engineering team


[1] https://wiki.xen.org/wiki/Paravirtualization_(PV)

[2] https://wiki.xen.org/wiki/PyGrub

[3] http://revolution9.fr/

Roman Grimaldi