Adding support for PVFS2 to the Kernel
From Debian Clusters
Contents |
Adding Support For PVFS2 to the Kernel
First off, you might be wondering with the FLA (five-letter acronym) "PVFS2" stands for. PVFS2 stands for the Parallel Virtual Filesystem, version 2."
It's name represents what it does. Namely, it adds parallel filesystem support to your kernel, and the backend to distribute that filesystem across various systems in your cluster. This adds higher I/O throughput capabilities to your distributed applications that are hungry to read in a lot of data.
Adding PVFS support to your kernel requires building the PVFS kernel module. This task is relatively easy to do if you leverage the PVFS code base and have your kernel headers readily available. This process is described in further detail below. However, the best and most up-to-date installation instructions are available from the PVFS Documentation Page.
Getting PVFS2
The download site on the PVFS site should direct you to where you need to go. Fetch the tarball for the current release:
root@vulture:/usr/local/src# wget ftp://ftp.parl.clemson.edu/pub/pvfs2/pvfs-2.7.1.tar.gz
Configuring PVFS2
Once downloaded, extract the tar file, cd into the directory and configure the PVFS code base for your system.
root@vulture:/usr/local/src# tar xfz pvfs-2.7.1.tar.gz root@vulture:/usr/local/src# cd pvfs-2.7.1 root@vulture:/usr/local/src/pvfs-2.7.1# ./configure --prefix=/opt/pvfs2 --with-kernel=/usr/src/linux-source-2.6.24
After configuration completes, you should see something like this:
***** Displaying PVFS2 Configuration Information ***** ------------------------------------------------------ PVFS2 configured to build karma gui : no PVFS2 configured to use epoll : yes PVFS2 configured to perform coverage analysis : no PVFS2 configured for aio threaded callbacks : yes PVFS2 configured for the 2.6.x kernel module : yes PVFS2 configured for the 2.4.x kernel module : no PVFS2 configured for using the mmap-ra-cache : no PVFS2 configured for using trusted connections : no PVFS2 configured for a thread-safe client library : yes PVFS2 will use workaround for redhat 2.4 kernels : no PVFS2 will use workaround for buggy NPTL : no PVFS2 server will be built : yes PVFS2 version string: 2.7.1 root@vulture:/usr/local/src/pvfs-2.7.1#
If you get an error about missing database libraries, you'll need to install the Berkely Database development libraries:
root@vulture:/usr/local/src/pvfs-2.7.1# apt-get install libdb-dev
Building PVFS2
Once configured, the PVFS2 code base is ready to be compiled for your system using make:
root@vulture:/usr/local/src/pvfs-2.7.1# make root@vulture:/usr/local/src/pvfs-2.7.1# make install
This builds the system and installs PVFS to the target directory, which is /opt/pvfs2 in our case. In order to enable support in the kernel, the kernel module for PVFS is built with make kmod:
root@vulture:/usr/local/src/pvfs-2.7.1# make kmod root@vulture:/usr/local/src/pvfs-2.7.1# make kmod_install
One step is still needed to grease the wheels. The make kmod_install command above puts the kernel module in the right place, but in this particular release, does not build the module dependencies required for automating kernel module insertion. This can be done by hand:
root@vulture:/usr/local/src/pvfs-2.7.1# depmod -a # assuming that you're running the kernel that you just built the module for.
Firing up PVFS2 support
Once the PVFS2 utilitites and modules have been built and installed, the easiest way to see if PVFS2 was compiled correctly is to insert the pvfs2 kernel module:
root@vulture:/usr/local/src/pvfs-2.7.1# modprobe -a pvfs2
dmesg should indicate that the module loaded correctly:
root@vulture:/usr/local/src/pvfs-2.7.1# dmesg ... pvfs2: module version 2.7.1 loaded root@vulture:/usr/local/src/pvfs-2.7.1#
Setting up and deploying PVFS2 is the content for another section, but the PVFS Documentation Page is still the best place to go for setup and deployment instructions.
